Video stream transmission method and device in video surveillance system, and storage medium

Abstract

The present invention relates to the technical field of video surveillances. Disclosed are a video stream transmission method and device in a video surveillance system, and a storage medium. A backup access apparatus can continuously receive a video stream when an access apparatus is in a fault state, and send the video stream to a processing apparatus on the basis of a first connection instruction sent by the processing apparatus, so that the processing apparatus can receive a complete video stream and perform data processing, thereby avoiding affecting the completion of a video surveillance task and avoiding impacting user experience.

Description

A video stream transmission method, device, and storage medium in a video surveillance system. Technical Field

[0001] This invention relates to the field of video surveillance technology, and in particular to a video stream transmission method, device and storage medium in a video surveillance system. Background Technology

[0002] Video surveillance systems are playing an increasingly prominent role in transportation command and production operations, and the requirements for their functions and performance are becoming more and more stringent.

[0003] A video surveillance system can communicate with cameras, continuously receive video streams from the cameras, and process these streams to complete video monitoring tasks. If the system malfunctions and cannot receive video streams from the cameras, it can quickly resolve the issue and resume receiving and processing video streams after the malfunction is resolved.

[0004] However, during fault handling, video surveillance systems may be unable to receive and process video streams, resulting in incomplete video streams. This can affect the completion of video surveillance tasks and impact user experience. For example, when a user views live monitoring in a video surveillance system, the system can establish a communication link between the camera and the user terminal, continuously sending the video stream captured by the camera to the user terminal so that the user can view the live monitoring. When the communication link fails, the reception and transmission of the video stream will be interrupted. In this case, the video surveillance system can rebuild a new communication link and then continue sending the video stream captured by the camera to the user terminal. Technical issues

[0005] However, the video surveillance system is unable to receive and transmit video streams during the establishment of a new communication link, causing users to be unable to view the surveillance video during that period. This may result in users perceiving a video interruption and affecting the user experience. Technical solutions

[0006] This invention provides a video stream transmission method, device, and storage medium in a video surveillance system to address the shortcomings of related technologies where video surveillance systems cannot receive and process video streams during fault handling, resulting in incomplete video streams that may affect the completion of video surveillance tasks and user experience. The backup access device can continuously receive video streams during access device malfunctions and send video streams to the processing device according to a first connection command sent by the processing device, enabling the processing device to receive complete video streams and perform data processing, thus avoiding impact on the completion of video surveillance tasks and user experience.

[0007] In a first aspect, the present invention provides a video stream transmission method in a video surveillance system, the system comprising an access device, a backup access device, and a processing device; the method comprising:

[0008] The backup access device parses the video stream continuously sent by the camera device to obtain multiple frame images in sequence, and caches the latest N frame groups; wherein, the multiple frame images include multiple frame groups arranged in sequence, each frame group includes a first keyframe and at least one prediction frame arranged in sequence, the at least one prediction frame includes each frame image located after the first keyframe and before the second keyframe, the second keyframe being the keyframe following the first keyframe; N is an integer not less than 1;

[0009] For any obtained frame, the backup access device determines that the frame is a key frame. Then, it generates a corresponding hash value based on a preset hash algorithm and the frame, and uses it as a frame group identifier. Starting from a preset initial order, it records the acquisition order of the frame and the frame in the same group to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group. It then creates frame information for the frame and the frame in the same group, including the frame group identifier and the acquisition order of the frame in the corresponding frame group.

[0010] The backup access device receives a first connection instruction sent by the processing device; wherein, the first connection instruction is generated when the processing device determines that the access device has failed and disconnects from the access device during data processing of the video stream received through the access device, and the first connection instruction includes a target hash value and a target acquisition order, wherein the target hash value is generated by the processing device according to the preset hash algorithm and the target keyframe, the target acquisition order is the acquisition order of the target frame in the corresponding frame group, and the target keyframe and the target frame are respectively the latest keyframe and the latest frame obtained and processed by the processing device according to the video stream before disconnecting from the access device;

[0011] The backup access device determines the next frame of the target frame from the N currently cached frame groups based on the target hash value, the order in which the target is acquired, and the frame information in the first connection instruction, and sends the video stream to the processing device to continue data processing starting from the next frame.

[0012] Optionally, before caching the most recently acquired N frame groups, the following is also included:

[0013] The backup access device determines the handover duration, maximum frame transmission delay, frame transmission rate, and number of frames in the frame group; wherein, the handover duration is the time required for the processing device to successfully switch to the backup access device after determining that the access device has failed.

[0014] The backup access device inputs the switching duration, the maximum frame transmission delay, the frame transmission rate, and the number of frames contained in the frame group into a preset model for calculation, obtains the output value of the preset model, determines an integer that is not less than the output value and is closest to the output value, and uses it as N;

[0015] The preset model is: M = (S0 + S τ )×FPS / GOP;

[0016] M is the variable corresponding to the output value, S0 is the variable corresponding to the switching duration, and S τ The variable corresponding to the maximum frame transmission delay is denoted by FPS, the variable corresponding to the frame transmission rate is denoted by FPS, and the variable corresponding to the number of frames is denoted by GOP.

[0017] Optionally, when the processing device is a distribution device, the first connection instruction is generated when the distribution device determines that the access device has failed and disconnected from the access device during the process of forwarding the video stream received through the access device to the downstream device;

[0018] Starting from the next frame, sending the video stream to the processing device for continued data processing includes:

[0019] The backup access device sends the video stream to the distribution device starting from the next frame after the target frame, so that the distribution device continues to forward the received video stream to the downstream device.

[0020] Optionally, when the processing device is a storage device, the first connection instruction is generated when the storage device determines that the access device has malfunctioned and disconnected from the access device during the storage of the video stream received through the access device;

[0021] Starting from the next frame, sending the video stream to the processing device for continued data processing includes:

[0022] The backup access device sends the video stream to the storage device starting from the next frame after the target frame, so that the storage device continues to store the received video stream.

[0023] Optionally, during data processing of the video stream received through the access device, if the processing device determines that the access device has malfunctioned and disconnected from the access device, it generates the first connection instruction, configured as follows:

[0024] When the processing device obtains the first keyframe from the video stream received through the access device and has processed the first keyframe, it generates a corresponding first hash value according to the preset hash algorithm and the first keyframe, determines the first acquisition order of the first keyframe in the corresponding screen group, and saves the first hash value and the first acquisition order to a preset information record table.

[0025] When the processing device acquires a new keyframe from the video stream received through the access device and has processed the new keyframe, it generates a new hash value corresponding to the new keyframe based on the preset hash algorithm, replaces the hash value currently stored in the information record table with the new hash value, and saves it; and when the processing device acquires a new frame from the video stream received through the access device and has processed the new frame, it determines the new acquisition order of the new frame in the corresponding frame group, replaces the acquisition order currently stored in the information record table with the new acquisition order, and saves it.

[0026] During the data processing of the video stream received through the access device, when the processing device determines that the access device has malfunctioned and is disconnected from the access device, it uses the hash value currently stored in the information record table and the acquisition order as the target hash value and the target acquisition order, respectively.

[0027] The processing device generates a first connection instruction that includes the target hash value and the order in which the targets are acquired.

[0028] Optionally, the system further includes a backup processing device and a service monitoring device; after sending the video stream to the processing device for continued data processing starting from the next frame, the method further includes:

[0029] The backup access device stops sending the video stream to the processing device in response to a stop command sent by the service monitoring device; wherein the stop command is generated by the service monitoring device when it determines that the processing device has malfunctioned based on a status message sent by the processing device.

[0030] The backup access device receives a second connection instruction sent by the backup processing device; wherein, the second connection instruction is generated by the backup processing device in response to a start instruction sent by the service monitoring device, based on the hash value currently stored in the information record table and the order in which they are acquired; the start instruction is generated by the service monitoring device when it determines that the processing device has failed based on a status message sent by the processing device.

[0031] The backup access device sends the video stream to the backup processing device for further data processing according to the second connection instruction.

[0032] Secondly, the present invention provides a video stream transmission device in a video surveillance system, the system including an access device, a backup access device, and a processing device; the video stream transmission device is applied to the backup access device.

[0033] The video stream transmission device includes:

[0034] A parsing unit is used to parse the video stream continuously sent by the camera device to obtain multiple frame images in sequence; wherein, the multiple frame images include multiple frame groups arranged in sequence, each frame group includes a first keyframe and at least one prediction frame arranged in sequence, the at least one prediction frame includes each frame image located after the first keyframe and before a second keyframe, the second keyframe being the keyframe following the first keyframe; N is an integer not less than 1.

[0035] The cache unit is used to cache the most recently acquired N frame groups;

[0036] A creation unit is configured to, for any obtained frame, determine that the frame is a keyframe, generate a corresponding hash value based on a preset hash algorithm and the frame, and use it as a frame group identifier, record the acquisition order of the frame and the frame in the same group starting from a preset initial order, so as to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group, and create frame information of the frame and the frame in the same group, the frame information including the frame group identifier and the acquisition order of the frame in the corresponding frame group;

[0037] The first receiving unit is configured to receive a first connection instruction sent by the processing device; wherein the first connection instruction is generated when the processing device determines that the access device has malfunctioned and disconnects from the access device during data processing of the video stream received through the access device; the first connection instruction includes a target hash value and a target acquisition order; the target hash value is generated by the processing device according to the preset hash algorithm and the target keyframe; the target acquisition order is the acquisition order of the target frame in the corresponding frame group; and the target keyframe and the target frame are respectively the latest keyframe and the latest frame obtained and processed by the processing device according to the video stream before disconnecting from the access device.

[0038] The first determining unit is configured to determine the next frame of the target frame in the currently cached N frame groups based on the target hash value, the order in which the target is acquired, and the frame information in the first connection instruction.

[0039] The first sending unit is used to send the video stream to the processing device for continued data processing starting from the next frame.

[0040] Optionally, when the processing device is a distribution device, the first connection instruction is generated when the distribution device determines that the access device has failed and disconnected from the access device during the process of forwarding the video stream received through the access device to the downstream device;

[0041] The first transmitting unit is further configured to:

[0042] Starting from the next frame after the target frame, the video stream is sent to the distribution device so that the distribution device continues to forward the received video stream to the downstream device.

[0043] Optionally, when the processing device is a storage device, the first connection instruction is generated when the storage device determines that the access device has malfunctioned and disconnected from the access device during the storage of the video stream received through the access device;

[0044] The first transmitting unit is further configured to:

[0045] Starting from the next frame after the target frame, the video stream is sent to the storage device so that the storage device continues to store the received video stream.

[0046] Thirdly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the video stream transmission method in a video surveillance system according to the first aspect or any corresponding embodiment described above. Beneficial effects

[0047] This invention provides a video stream transmission method, device, and storage medium for a video surveillance system. The video surveillance system includes an access device, a backup access device, and a processing device. The backup access device can parse the continuously transmitted video stream from the camera device to sequentially obtain multiple frame images, cache the latest acquired N frame groups, and create frame information for each frame image. The frame information includes the corresponding frame group identifier and the acquisition order of the frame image within the corresponding frame group. The backup access device receives a first connection instruction sent by the processing device. The first connection instruction is generated when the processing device determines that the access device has failed and disconnects from the access device during data processing of the video stream received through the access device. According to the first connection instruction, the backup access device determines the next frame image of the target frame image from the currently cached N frame groups. The target frame image is the latest frame image obtained and processed by the processing device based on the video stream before determining that the access device has failed and disconnecting from the access device. The backup access device then transmits the video stream to the processing device starting from this next frame image to continue data processing. The backup access device can continuously receive video streams during the period when the access device is in failure, and send video streams to the processing device according to the first connection command sent by the processing device, so that the processing device can receive the complete video stream and perform data processing, avoiding the impact on the completion of video surveillance tasks and avoiding the impact on user experience. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 is a flowchart of a video stream transmission method in a video surveillance system provided by an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a video stream structure provided in an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of the structure of multiple frame information provided in an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of the structure of an information recording table provided in an embodiment of the present invention;

[0053] Figure 5 is a schematic diagram of another information recording table provided in an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of the distribution of screen groups according to an embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of the structure of a video stream transmission device in a video surveillance system provided by an embodiment of the present invention;

[0056] Figure 8 is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. The best embodiment of the present invention

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0058] The video stream transmission method in the video surveillance system of the present invention is described below with reference to Figures 1-6.

[0059] As shown in Figure 1, this embodiment proposes a first method for video stream transmission in a video surveillance system. The system includes an access device, a backup access device, and a processing device. The method may include the following steps:

[0060] S101, The backup access device parses the video stream continuously sent by the camera device to obtain multiple screen frames in sequence. Among them, the multiple screen frames include multiple screen groups arranged in sequence, each screen group includes a first keyframe and at least one prediction frame arranged in sequence, and the at least one prediction frame includes each screen frame located after the first keyframe and before the second keyframe, the second keyframe being the keyframe following the first keyframe.

[0061] Optionally, the video surveillance system can be used for transportation command, public security, production operations, or safety monitoring. For example, the video surveillance system can be a comprehensive railway video surveillance system.

[0062] The access device and the backup access device can have identical performance. Both the access device and the backup access device are communicatively connected to the camera device and synchronously receive the continuously transmitted video stream from the camera device. The access device can forward the continuously transmitted video stream from the camera device to the processing device in real time. The backup access device can replace the access device and transmit the video stream transmitted by the camera device to the processing device when the access device fails.

[0063] Specifically, keyframes and prediction frames within a Group of Pictures (GOP) are both picture frames, with the first picture frame in the GOP being the keyframe. A video sequence, or video stream, can include multiple GOPs arranged sequentially. Each GOP contains one keyframe (I-frame) and other non-keyframes. The non-keyframes are the prediction frames, which can be either P-frames (forward prediction frames) or B-frames (bidirectional interpolation frames).

[0064] As shown in Figure 2, when the video surveillance system is a railway integrated video surveillance system, each Group of Pictures (GOP) in the video sequence (i.e., the video stream) can be composed of one I-frame followed by 49 P-frames, corresponding to a 2-second video stream. It should be noted that in the railway integrated video surveillance system, the program processing the video stream is divided into video access, recording and storage, and video forwarding programs. When a user views the video, the system program establishes a communication link after signaling negotiation to perform video stream storage or forwarding. During the user's viewing process, if a program on this communication link malfunctions or needs to be restarted or upgraded, the link will be interrupted, requiring renegotiation to establish a new link. The user terminal will inevitably be affected by the temporary lack of data. A characteristic of the video sequence is that the correct decoding of a P-frame in a GOP depends on the preceding I-frames and P-frames. Once data is lost, data after this GOP may not be correctly decoded and viewed, and recovery can only begin from the next received I-frame.

[0065] Specifically, the processing device can receive video streams forwarded by the access device or backup access device, and process the video streams accordingly, such as storing the video streams or forwarding the video streams to the user terminal.

[0066] Specifically, the processing device can first receive the video stream forwarded by the access device. When the access device fails and cannot continue to forward the video stream to the processing device, the processing device can switch to the backup access device and receive the video stream forwarded by the backup access device.

[0067] S102. The backup access device caches the latest N screen groups. N is an integer not less than 1.

[0068] It should be noted that when the processing device determines that the access device has malfunctioned, it will disconnect from the access device. The processing device will no longer receive the video stream forwarded by the access device, and the process of switching to the backup access device to receive the video stream requires a certain switching time. If the backup access device only forwards the video stream sent by the camera to the processing device in real time, the processing device will miss the video stream sent by the camera during the aforementioned switching time. In this embodiment, the backup access device can save the video stream during the aforementioned switching time so that when sending a video stream to the processing device, it can start sending from the video stream within the aforementioned switching time, so that the processing device can obtain the complete video stream.

[0069] Specifically, the backup access device can parse the received video stream, extract multiple frame images carried in the video stream in sequence, and cache the latest N extracted frame images, that is, only store the latest N extracted frame images, and do not store other frame images.

[0070] It is understood that the size of N must ensure that the backup access device retains the video stream within the aforementioned switching time. In this embodiment, technicians can set N according to actual conditions to ensure that when the backup access device sends a video stream to the processing device, it can start sending from the video stream within the aforementioned switching time, thereby ensuring that the processing device receives the complete video stream, avoiding impact on the completion of the video surveillance task, and avoiding impact on the user experience.

[0071] S103. For any obtained frame, the backup access device determines that the frame is a key frame. Then, it generates a corresponding hash value based on the preset hash algorithm and the frame and uses it as the frame group identifier. Starting from the preset initial order, it records the acquisition order of the frame and the frame in the same group to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group. It creates frame information of the frame and the frame in the same group. The frame information includes the frame group identifier and the acquisition order of the frame in the corresponding frame group.

[0072] The preset initial order can be set by technicians according to the actual situation, such as 0 or 1, but this embodiment does not limit it.

[0073] The frames in the same frame group can include other frames that are in the same frame group as the keyframe. This means that the frames in the same frame group can include every frame that follows the keyframe in the frame group, i.e., every predicted frame in the frame group. For example, when a frame group includes keyframe A, predicted frame B, predicted frame C, and predicted frame D, the frames in the same frame group can include predicted frames B, C, and D that are in the same frame group as keyframe A.

[0074] Specifically, for any frame, the backup access device can determine whether it is a keyframe. If it is a keyframe, it can process the frame using a preset hash algorithm to generate a corresponding hash value, which is then used as the identifier for the corresponding frame group. The backup access device can then record the acquisition order of the frame and other frames in the same group, starting from a preset initial order, thus recording the acquisition order of the frame and other frames in the corresponding frame group. If the frame is not a keyframe, this embodiment continues recording the acquisition order of the frame in the corresponding frame group from the currently recorded order.

[0075] It is understandable that the backup access device can record the acquisition order of each frame within its corresponding frame group. The backup access device can also create frame information for each frame. The frame information of any frame can include the frame group identifier of the frame group to which the frame belongs, and the acquisition order of the frame within the corresponding frame group. The frame information of frames within the same frame group can include the frame information of each frame following the keyframe within that frame group.

[0076] It is understandable that the frame information of a frame, including the frame group identifier and the order in which the frame was acquired, can be used to uniquely identify that frame.

[0077] Optionally, the backup access device can also record information such as the video stream ID, the time of receiving the frame, and the original data of the frame in the frame information. As shown in Figure 3, the GOP HASH in Figure 3 is the frame group identifier, the time in Figure 3 is the time information of receiving the frame, which can be used to identify the receiving time of the frame, the sequence number is the order in which the frame is acquired in the corresponding frame group (sequence number 0 indicates that the frame is acquired first in the corresponding frame group, sequence number 1 indicates that the frame is acquired second in the corresponding frame group, and so on), and the data is the original data of the frame.

[0078] S104. The backup access device receives a first connection instruction sent by the processing device; wherein, the first connection instruction is generated when the processing device determines that the access device has failed and disconnects from the access device during data processing of the video stream received through the access device, and the first connection instruction includes a target hash value and a target acquisition order, the target hash value is generated by the processing device according to a preset hash algorithm and a target key frame, the target acquisition order is the acquisition order of the target frame in the corresponding frame group, and the target key frame and the target frame are the latest key frame and the latest frame obtained and processed by the processing device according to the video stream before disconnecting from the access device.

[0079] Specifically, when the processing device determines that the access device has malfunctioned, it can disconnect the communication connection with the access device, and determine the latest key frame and the latest picture frame that were received from the access device and processed (e.g., stored or forwarded to the user terminal) before the connection with the access device was disconnected. The processing device can then use a preset hash algorithm to process the latest key frame to generate the corresponding hash value, i.e., the target hash value, and determine the acquisition order of the latest picture frame in the corresponding picture group, i.e., the target acquisition order.

[0080] Understandably, the processing device can also generate a corresponding hash value based on the new keyframe each time it receives and processes it, or each time it receives and processes a new frame, and determine the acquisition order of the frame in the corresponding frame group, and save the hash value and the acquisition order. Alternatively, the processing device can save only the hash value corresponding to the latest keyframe and the acquisition order of the latest frame in the corresponding frame group, without saving the hash values ​​corresponding to other keyframes or the acquisition order of other frames in the corresponding frame group.

[0081] Specifically, the processing device can generate a first connection instruction based on the target hash value and the order in which the targets are acquired, and send it to the backup access device.

[0082] S105. The backup access device determines the next frame of the target frame from the N currently cached frame groups based on the target hash value, the order in which the target is acquired, and the frame information in the first connection instruction.

[0083] Specifically, the backup access device can extract the target hash value and the target acquisition order from the first connection command, determine the matching frame information based on the target hash value and the target acquisition order, and determine the next frame information of that frame information, that is, the frame information of the next frame of the corresponding video frame. Then, in this embodiment, based on the next frame information, the corresponding video frame can be determined from all video frames of the currently cached N video groups, that is, the next frame of the target video frame in the video stream.

[0084] S106. Starting from the next frame, the backup access device sends a video stream to the processing device to continue data processing.

[0085] Specifically, the backup access device can forward the video stream to the processing device starting from the next frame after the target frame in the video stream, that is, forward the subsequent frames of the target frame in the video stream to the processing device.

[0086] This embodiment proposes a video stream transmission method for a video surveillance system. The video surveillance system includes an access device, a backup access device, and a processing device. The backup access device can parse the continuously transmitted video stream from the camera device to sequentially obtain multiple frame images, and cache the latest acquired N frame groups. It creates frame information for each frame image, which includes the corresponding frame group identifier and the acquisition order of the frame image within the corresponding frame group. The backup access device receives a first connection instruction sent by the processing device. This first connection instruction is generated when the processing device determines that the access device has failed and disconnects from the access device during data processing of the video stream received through the access device. Based on the first connection instruction, the backup access device determines the next frame image of the target frame image from the currently cached N frame groups. The target frame image is the latest frame image obtained and processed by the processing device based on the video stream before determining that the access device has failed and disconnecting from the access device. The backup access device then transmits the video stream to the processing device starting from this next frame image to continue data processing. The backup access device can continuously receive video streams during the period when the access device is in failure, and send video streams to the processing device according to the first connection command sent by the processing device, so that the processing device can receive the complete video stream and perform data processing, avoiding the impact on the completion of video surveillance tasks and avoiding the impact on user experience. Embodiments of the present invention

[0087] It should be noted that among related technologies, the railway integrated video surveillance system has significant application value in the railway transportation sector. This system has already achieved large-scale network coverage of railway work areas, stations, railway bureaus / companies, and the China Railway Corporation. With the development of railway transportation, the role of video surveillance systems in transportation command, public security, production operations, and safety monitoring is becoming increasingly prominent. The pressure on railway operations to utilize integrated video surveillance systems is also increasing, thus leading to increasingly stringent requirements for the functionality and performance of these systems.

[0088] In the current railway integrated video surveillance system, the use of cloud computing technology has greatly improved system stability. However, due to the characteristics of video systems, although the system automatically recovers after a failure in the background service program, there will still be short-term interruptions in the video stream. This results in users perceiving preview and recording interruptions, affecting user experience and related business. Furthermore, normal system software upgrades and maintenance can also cause real-time preview and recording interruptions, so upgrades and maintenance must be performed during nighttime maintenance windows, which also causes inconvenience to maintenance personnel. Specifically, the current railway integrated video surveillance system uses single-point access, with single-point forwarding between background service programs. When viewing video from the core node, there is a unique route from the core node to the cameras under the access nodes. In this software architecture, when a program malfunctions, even if it recovers quickly, the link still needs to be re-established, inevitably disrupting the continuity of video data and causing interruptions in the upstream video preview. This embodiment, through the method shown in Figure 1, considers technologies such as system-related interfaces and cloud computing to improve system stability and solve the problem of real-time preview and recording interruptions caused by program instability or upgrades in the railway integrated video surveillance system. In the event of individual service program failures or upgrades and maintenance, it can achieve a user-unobtrusive effect and ensure user experience.

[0089] Based on Figure 1, this embodiment proposes a second video stream transmission method in a video surveillance system. This method, before buffering the latest N video frames, includes:

[0090] The backup access device determines the handover duration, maximum frame transmission delay, frame transmission rate, and number of frames in a frame group; whereby the handover duration is the time required for the processing device to successfully switch to the backup access device after determining that the access device has failed.

[0091] The backup access device determines N based on the handover duration, maximum frame transmission delay, frame transmission rate, and the number of frames contained in the frame group.

[0092] Specifically, in this embodiment, the value of N can be determined based on the switching duration, maximum frame transmission delay, frame transmission rate, and the number of frames contained in the frame group. This is to minimize the number of frames buffered in the frame group while ensuring the transmission of the complete video stream to the processing device, thereby reducing the consumption of buffer resources and improving the utilization rate of buffer resources.

[0093] In practical applications, the video stream transmission rate must be greater than the video bitrate to enable real-time viewing. The video stream transmission rate does not significantly affect video latency. For example, in a video stream with a keyframe every 2 seconds, a frame rate of 25 frames per second, and a bitrate of 4Mb / s, the transmission speed must be greater than 4Mb / s. The average latency per frame cannot exceed 40 milliseconds (25 frames per second). However, due to the varying size of each frame, keyframes, with their larger data volume, will experience slightly higher latency.

[0094] Optionally, the backup access device determines N based on the handover duration, maximum frame transmission delay, frame transmission rate, and the number of frames contained in the frame group, including:

[0095] The backup access device inputs the switching duration, maximum frame transmission delay, frame transmission rate, and number of frames in the frame group into the preset model for calculation, obtains the output value of the preset model, determines the integer that is not less than the output value and is closest to the output value, and uses it as N;

[0096] The preset model is: M = (S0 + S τ )×FPS / GOP;

[0097] M is the variable corresponding to the output value, S0 is the variable corresponding to the switching duration, and S τ is the variable corresponding to the maximum frame transmission delay, FPS is the variable corresponding to the frame transmission rate, and GOP is the variable corresponding to the number of frames.

[0098] Specifically, in this embodiment, the switching duration, maximum frame transmission delay, frame transmission rate, and number of frames in the frame group can be input into a preset model for calculation to obtain the corresponding output value. The integer closest to the output value and not less than the output value is determined and used as N.

[0099] The video stream transmission method in the video surveillance system proposed in this embodiment can determine N based on the switching duration, maximum frame transmission delay, frame transmission rate, and the number of frames in the frame group. While effectively ensuring the transmission of the complete video stream to the processing device, it can reduce the consumption of video stream buffer resources in the backup access device and improve the utilization rate of buffer resources.

[0100] Based on Figure 1, this embodiment proposes a third method for video stream transmission in a video surveillance system, wherein the processing device is a storage device or a distribution device.

[0101] Optionally, when the processing device is a distribution device, the first connection instruction is generated when the distribution device determines that the access device has failed and disconnected from the access device during the forwarding of the video stream received through the access device to the downstream device. In this case, the aforementioned sending of the video stream to the processing device to continue data processing from the next frame includes:

[0102] The backup access device sends a video stream to the distribution device starting from the next frame after the target frame, so that the distribution device can continue to forward the received video stream to the downstream device.

[0103] Specifically, when the processing device is a distribution device, this embodiment, through the setup of the access device, the backup access device, and the distribution device, can continuously forward and completely forward the video stream captured by the camera device to the user terminal, thereby meeting the user's need to view real-time monitoring and avoiding the user's perception of video interruption, which would affect the user experience.

[0104] Optionally, for any group of frames in the received video stream, the distribution device can split each frame in the group of frames into multiple Real-Time Transport Protocol (RTP) packets and send the multiple RTP packets corresponding to each frame to the downstream device.

[0105] Optionally, when the processing device is a storage device, the first connection instruction is generated when the storage device determines that the access device has malfunctioned and disconnected from the access device during the storage of the video stream received through the access device. In this case, the aforementioned sending the video stream to the processing device to continue data processing from the next frame includes:

[0106] The backup access device sends a video stream to the storage device starting from the next frame after the target frame, so that the storage device can continue to store the received video stream.

[0107] Downstream devices can be user terminals or downstream devices with heavy video surveillance tasks.

[0108] Optionally, when the processing device is a storage device, a distribution device can also be provided in the video surveillance system of this embodiment. In this case, the storage device can store the complete video stream sent by the access device or the backup access device, and can send the stored video stream to downstream devices through the distribution device.

[0109] Specifically, when the processing device is a storage device, this embodiment, through the setup of an access device, a backup access device, and a storage device, sends the video stream captured by the camera device to the storage device for storage in its entirety. This can satisfy the user's need to view historical monitoring data when needed, and can prevent the user from perceiving video interruption and affecting the user experience.

[0110] The video stream transmission method in the video surveillance system proposed in this embodiment can effectively store the full video stream sent by the camera device, and can also forward the video stream of the camera device to the user terminal in real time, so that the user can view historical video surveillance or real-time video surveillance. In addition, when the access device fails, the backup access device can be activated to work, so as to avoid the user's perception of video interruption during the viewing of video surveillance, ensure user experience, and enhance user stickiness.

[0111] This embodiment proposes a fourth video stream transmission method in a video surveillance system. During data processing of the video stream received through the access device, if the processing device determines that the access device has malfunctioned and disconnected, it generates a first connection command, configured as follows:

[0112] When the processing device obtains the first keyframe from the video stream received through the access device and has processed the first keyframe, it generates the corresponding first hash value according to the preset hash algorithm and the first keyframe, determines the first acquisition order of the first keyframe in the corresponding screen group, and saves the first hash value and the first acquisition order to the preset information record table.

[0113] When the processing device acquires a new keyframe from the video stream received through the access device and has processed the new keyframe, it generates a new hash value based on a preset hash algorithm and the new keyframe, replaces the hash value currently stored in the information record table with the new hash value, and saves it; and when the processing device acquires a new frame from the video stream received through the access device and has processed the new frame, it determines the new acquisition order of the new frame in the corresponding frame group, replaces the acquisition order currently stored in the information record table with the new acquisition order, and saves it.

[0114] During the data processing of the video stream received through the access device, if the processing device determines that the access device has failed and is disconnected from the access device, it uses the hash value and the acquisition order currently stored in the information record table as the target hash value and the target acquisition order, respectively.

[0115] The processing device generates a first connection instruction that includes the target hash value and the order in which the targets are retrieved.

[0116] Specifically, the processing device can save and update the identification information of the latest keyframe and the latest frame in the information record table, and record in real time the identification information of the latest keyframe and the latest frame that has been acquired and processed, that is, the hash value corresponding to the latest keyframe and the acquisition order of the latest frame in the corresponding frame group. In the event of a failure of the access device, the identification information can be sent to the backup access device, so that the backup access device can continue to send subsequent frame to the processing device from the point where the video stream is interrupted, thus ensuring the integrity of the video stream received by the processing device.

[0117] Specifically, when the processing device is a storage device, the storage device can initiate a storage task and create a storage session, i.e., an information record table, when it starts receiving the video stream. This information record table saves and updates the hash value of the latest keyframe received by the storage device and the acquisition order of the latest frame. The information record table can also record information such as the storage task session ID and the video stream ID. As shown in Figure 4, the information record table can store the storage task session ID, the video stream ID, and the device identifiers of all access devices (i.e., access programs). It can also store the stored data frame GOP HASA (i.e., the hash value of the latest keyframe received and stored by the storage device), the stored data frame timestamp (i.e., the timestamp of the latest keyframe), the stored data frame sequence number (i.e., the acquisition order of the latest frame received and stored by the storage device in the corresponding GOP), and the last storage time (i.e., the timestamp of the latest frame).

[0118] Optionally, when the processing device is a storage device, the storage device can, upon starting the storage task (i.e., starting to receive video streams), obtain the addresses of all access programs (i.e., all access devices), select an access program to retrieve the video stream, determine whether the video stream has timed out, and if it is determined that it has not timed out, store the video stream and continue receiving video streams. If it is determined that the video stream has timed out, i.e., no video stream sent by the access device is received within a certain period of time, then another access device (i.e., a backup access device) is selected to retrieve the stream.

[0119] Specifically, when storing video streams, the storage device needs to update information such as the time, sequence number, and GOP hash of the video frames in the storage session, i.e., the first record information table. When switching to the backup access device, it can send the hash value, timestamp, and frame sequence number of the last complete video frame received to the backup access device. The backup access device can then use this information to find the next frame to be sent to the storage device. The storage device must ensure the integrity of each stored video frame.

[0120] Specifically, there can be multiple distribution devices. Multiple distribution devices can jointly complete a sub-forwarding session, sharing the same task synchronization block, i.e., creating a jointly maintained information record table. In this case, the information record table can also include information such as the sub-forwarding task session ID, task connection information, and the identifier of the currently working distribution device. As shown in Figure 5, the information record table includes the sub-forwarding task session ID, video stream ID, task connection information, all sub-forwarding program IDs (i.e., device identifiers of all distribution devices), the currently working sub-forwarding program ID (i.e., the device identifier of the currently working distribution device), the sent data frame GOP HASH (i.e., the hash value of the latest keyframe sent by the distribution device to the downstream device), the sent data frame timestamp (i.e., the sending timestamp of the latest keyframe), the sent data frame sequence number (i.e., the acquisition order of the latest frame in the corresponding GOP sent by the distribution device to the downstream device), the sent data frame RTP sequence number (i.e., the sorting of the latest sent RTP in the corresponding frame), and the sent data frame RTP timestamp (i.e., the sending timestamp of the latest RTP).

[0121] The distribution device can operate in three modes: User Datagram Protocol (UDP), Transmission Control Protocol (TCP) active, and TCP passive. Depending on the operating mode, the distribution device employs different strategies. For UDP and TCP active modes, a contention-based strategy is used. Once a forwarding session times out, other distribution devices automatically enter contention mode. Distributed locking can be used to ensure that only one distribution device gains permission, and the device with permission immediately enters the forwarding state to provide the stream. When using TCP passive forwarding mode, because the terminal or upstream streaming media service actively initiates a connection, the distribution device immediately enters the working state upon detecting the connection establishment.

[0122] Optionally, when starting a split-forwarding task, the distribution device can obtain the addresses of all access programs, i.e., access devices, and determine whether it has the permission and status to obtain the task. If it does not have permission, it can wait and update the address of the access device. After obtaining permission, it selects an access device to retrieve the video stream, determines whether the video stream has timed out, and if it has, it can reselect an access device to retrieve the stream. If it has not timed out, it determines whether it has split-forwarding permission. If it does not, it re-obtains the address of the access device. If it has permission, it can split-forward the video stream, update the split-forwarding task status and the task permission timeout time, and continue to receive video streams.

[0123] Optionally, the system also includes a backup processing unit and a service monitoring unit. After sending the video stream to the processing unit for continued data processing starting from the next frame, the method further includes:

[0124] The backup access device responds to the stop command sent by the service monitoring device and stops sending video streams to the processing device; the stop command is generated by the service monitoring device when it determines that the processing device has failed based on the status message sent by the processing device.

[0125] The backup access device receives a second connection instruction sent by the backup processing device; wherein, the second connection instruction is generated by the backup processing device in response to the start instruction sent by the service monitoring device, based on the hash value currently stored in the information record table and the order in which they are acquired; the start instruction is generated by the service monitoring device when it determines that the processing device has failed based on the status message sent by the processing device.

[0126] The backup access device sends the video stream to the backup processing device for continued data processing according to the second connection command.

[0127] Specifically, this embodiment can be equipped with a backup processing device. When the processing device fails and cannot continue processing the video stream, the backup processing device is activated to continue receiving and processing the video stream, so as to ensure that the complete video stream is processed, such as stored and forwarded to the user terminal, to ensure the normal implementation of the video surveillance task and to ensure the user experience.

[0128] The second connection instruction may include the hash value and acquisition order currently stored in the information record table, namely, the hash value corresponding to the latest keyframe received by the processing device, and the acquisition order of the latest received video frame in the corresponding video group. The backup access device can, according to the second connection instruction, determine the next video frame after the latest received video frame from the currently cached N video frames, and transmit the video stream to the backup processing device for data processing starting from the next video frame, ensuring the video surveillance system can transmit and process the complete video stream.

[0129] Specifically, when the processing device is a storage device, the backup processing device is also the backup storage device. In this case, the system can monitor the status of the storage device. Once a timeout occurs, it will notify the access device or backup access device to stop sending streams to the storage device, deactivate the storage device, and quickly start a backup storage device. The backup storage device can continue to store the video stream according to the information in the storage session, i.e., the information record table, to ensure the continuity and integrity of the video stream storage.

[0130] Specifically, when the processing device is a distribution device, the backup processing device is the standby processing device. Upon startup, the standby processing device can continue receiving video frames from the video stream based on the information in the information log table, and then split the received frames into multiple RTP packets and send them to downstream devices. As shown in Figure 6, frames 1, 2, and 3 each correspond to multiple RTP packets. Forwarding device 1 and forwarding device 2 are the distribution device and the backup distribution device, respectively. When the distribution device sends frame 2 to the downstream device and the transmission is interrupted, the backup distribution device can, starting from frame 2, send multiple RTP packets corresponding to frame 2 and subsequent frames to the downstream device.

[0131] It should be noted that this embodiment allows for multiple access points to the video stream from the camera device. This embodiment can set up multiple mutually redundant access devices to ensure uninterrupted access to the camera device. Specifically, for camera devices requiring uninterrupted access, at least two video stream access programs on different servers must be running to ensure the system can reliably acquire video source data.

[0132] It should also be noted that this embodiment can ensure that multiple access devices can forward video streams to the distribution device by communicating between the distribution device and multiple access devices, so that the distribution device can correctly and completely acquire and process the video streams.

[0133] It should also be noted that when multiple distribution devices send the same video stream, they can use the same buffer information block, i.e., the aforementioned information record table, to ensure the consistency of the sending status. User terminals can use the RTP protocol to receive the stream, assembling consecutive frames based on RTP-related information, ensuring uninterrupted real-time preview. The distribution devices record the same data during operation, ensuring that multiple distribution devices have a consistent working status. This guarantees that when the same frame of data is sent via RTP, the other party can correctly identify the frame data for correct decoding and display. Distributed locking technology ensures the atomicity of the same frame of data.

[0134] The video stream transmission method in the video surveillance system proposed in this embodiment can meet the user's needs for viewing historical or real-time monitoring, and avoid the user's perception of video interruption due to system malfunctions during the monitoring process, thereby avoiding the impact on the user's viewing experience.

[0135] As shown in Figure 7, this embodiment proposes a video stream transmission device for a video surveillance system. The system includes an access device, a backup access device, and a processing device; the video stream transmission device is applied to the backup access device. The video stream transmission device includes:

[0136] The parsing unit 101 is used to parse the video stream continuously sent by the camera device to obtain multiple screen frames in sequence; wherein, the multiple screen frames include multiple screen groups arranged in sequence, each screen group includes a first keyframe and at least one prediction frame arranged in sequence, and the at least one prediction frame includes each screen frame located after the first keyframe and before the second keyframe, the second keyframe being the keyframe following the first keyframe; N is an integer not less than 1.

[0137] Cache unit 102 is used to cache the latest N frame groups;

[0138] The creation unit 103 is used to determine that any obtained frame is a key frame, generate a corresponding hash value according to a preset hash algorithm and the frame and use it as a frame group identifier, record the acquisition order of the frame and the frame in the same group starting from a preset initial order, so as to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group, and create frame information of the frame and the frame in the same group, including the frame group identifier and the acquisition order of the frame in the corresponding frame group.

[0139] The first receiving unit 104 is used to receive a first connection instruction sent by the processing device; wherein, the first connection instruction is generated when the processing device determines that the access device has failed and disconnects from the access device during data processing of the video stream received through the access device, and the first connection instruction includes a target hash value and a target acquisition order, the target hash value is generated by the processing device according to a preset hash algorithm and a target key frame, the target acquisition order is the acquisition order of the target frame in the corresponding frame group, and the target key frame and the target frame are the latest key frame and the latest frame obtained and processed by the processing device according to the video stream before disconnecting from the access device;

[0140] The first determining unit 105 is used to determine the next frame of the target frame in the currently cached N frame groups based on the target hash value, the order in which the target is acquired and the frame information in the first connection instruction.

[0141] The first sending unit 106 is used to send a video stream to the processing device for continued data processing starting from the next frame.

[0142] It should be noted that the processing procedures and beneficial effects of the parsing unit 101, the buffering unit 102, the creation unit 103, the first receiving unit 104, the first determining unit 105, and the first sending unit 106 can be referred to the relevant descriptions of steps S101 to S106 in Figure 1 in this embodiment, and will not be repeated here.

[0143] Optionally, the above-mentioned equipment also includes:

[0144] The second determining unit is used to determine the switching duration, maximum frame transmission delay, frame transmission rate, and number of frames contained in the frame group before caching the newly acquired N frame groups; wherein, the switching duration is the time required for the processing device to successfully switch to the backup access device after determining that the access device has failed; and N is determined based on the switching duration, maximum frame transmission delay, frame transmission rate, and number of frames contained in the frame group.

[0145] Optionally, the second determining unit is also used for:

[0146] Input the switching duration, maximum frame transmission delay, frame transmission rate, and number of frames in the frame group into the preset model for calculation, obtain the output value of the preset model, determine the integer that is not less than the output value and is closest to the output value, and use it as N;

[0147] The preset model is: M = (S0 + S τ )×FPS / GOP;

[0148] M is the variable corresponding to the output value, S0 is the variable corresponding to the switching duration, and S τ is the variable corresponding to the maximum frame transmission delay, FPS is the variable corresponding to the frame transmission rate, and GOP is the variable corresponding to the number of frames.

[0149] Optionally, when the processing device is a distribution device, the first connection instruction is generated when the distribution device determines that the access device has failed and disconnected from the access device during the process of forwarding the video stream received through the access device to the downstream device;

[0150] The first transmitting unit 106 is also used for:

[0151] Starting from the next frame after the target frame, a video stream is sent to the distribution device so that the distribution device continues to forward the received video stream to the downstream device.

[0152] Optionally, when the processing device is a storage device, the first connection instruction is generated when the storage device determines that the access device has failed and disconnected from the access device during the storage of the video stream received through the access device.

[0153] The first transmitting unit 106 is also used for:

[0154] Starting from the next frame after the target frame, a video stream is sent to the storage device so that the storage device can continue to store the received video stream.

[0155] Optionally, during data processing of the video stream received through the access device, if the processing device determines that the access device has malfunctioned and disconnected from the access device, it generates a first connection command, configured as follows:

[0156] When the processing device obtains the first keyframe from the video stream received through the access device and has processed the first keyframe, it generates the corresponding first hash value according to the preset hash algorithm and the first keyframe, determines the first acquisition order of the first keyframe in the corresponding screen group, and saves the first hash value and the first acquisition order to the preset information record table.

[0157] When the processing device acquires a new keyframe from the video stream received through the access device and has processed the new keyframe, it generates a new hash value based on a preset hash algorithm and the new keyframe, replaces the hash value currently stored in the information record table with the new hash value, and saves it; and when the processing device acquires a new frame from the video stream received through the access device and has processed the new frame, it determines the new acquisition order of the new frame in the corresponding frame group, replaces the acquisition order currently stored in the information record table with the new acquisition order, and saves it.

[0158] During the data processing of the video stream received through the access device, if the processing device determines that the access device has failed and is disconnected from the access device, it uses the hash value and the acquisition order currently stored in the information record table as the target hash value and the target acquisition order, respectively.

[0159] The processing device generates a first connection instruction that includes the target hash value and the order in which the targets are retrieved.

[0160] Optionally, the system also includes a backup processing unit and a service monitoring unit; the aforementioned equipment also includes:

[0161] The stop unit is used to stop sending the video stream to the processing device after the video stream has been sent to the processing device for continued data processing starting from the next frame. This stop is triggered by a stop command sent by the service monitoring device. The stop command is generated by the service monitoring device when it determines that the processing device has malfunctioned based on a status message sent by the processing device.

[0162] The second receiving unit is used to receive a second connection instruction sent by the backup processing device; wherein, the second connection instruction is generated by the backup processing device in response to the start instruction sent by the service monitoring device, based on the hash value currently stored in the information record table and the order in which they are acquired; the start instruction is generated by the service monitoring device when it determines that the processing device has failed based on the status message sent by the processing device.

[0163] The second sending unit is used to send the video stream to the backup processing device for continued data processing according to the second connection instruction.

[0164] The video stream transmission device in the video surveillance system proposed in this embodiment allows the backup access device to continuously receive video streams during the period when the access device is faulty, and send video streams to the processing device according to the first connection instruction sent by the processing device, so that the processing device can receive the complete video stream and perform data processing, avoiding the impact on the completion of video surveillance tasks and avoiding the impact on user experience.

[0165] This embodiment proposes a video surveillance system, which includes an access device, a backup access device, and a processing device; wherein:

[0166] The backup access device parses the video stream continuously sent by the camera device to obtain multiple screen frames in sequence, and caches the latest N screen groups; wherein, the multiple screen frames include multiple screen groups arranged in sequence, each screen group includes a first keyframe and at least one prediction frame arranged in sequence, and at least one prediction frame includes each screen frame located after the first keyframe and before the second keyframe, the second keyframe being the keyframe following the first keyframe; N is an integer not less than 1;

[0167] For any obtained frame, the backup access device determines that the frame is a key frame. It then generates a corresponding hash value based on a preset hash algorithm and the frame, and uses it as a frame group identifier. Starting from a preset initial order, it records the acquisition order of the frame and the frame in the same group to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group. It then creates frame information for the frame and the frame in the same group, including the frame group identifier and the acquisition order of the frame in the corresponding frame group.

[0168] During data processing of the video stream received through the access device, when the processing device determines that the access device has malfunctioned and disconnected from the access device, it generates a first connection instruction. The first connection instruction includes a target hash value and a target acquisition order. The target hash value is generated by the processing device according to a preset hash algorithm and a target key frame. The target acquisition order is the acquisition order of the target frame in the corresponding frame group. The target key frame and the target frame are the latest key frame and the latest frame obtained and processed by the processing device based on the video stream before disconnecting from the access device.

[0169] The processing device sends a first connection command to the backup access device;

[0170] The backup access device determines the next frame of the target frame from the N currently cached frame groups based on the target hash value, the order in which the target is acquired, and the frame information in the first connection instruction. Starting from the next frame, it sends the video stream to the processing device to continue data processing.

[0171] In another video surveillance system proposed in this embodiment, multiple access devices, storage devices, and distribution devices can be configured. The video surveillance system can connect to camera devices and user terminals respectively, and can be configured with signaling access programs, video stream access programs, video stream storage programs, video stream forwarding programs, and other service programs. These other service programs can include metadata management, resource and program management, configuration, and middleware, etc., with resource and program management including a registration center. It should be noted that different video stream access programs are configured in different access devices, video stream storage programs can be configured in storage devices, and different video stream forwarding programs can be configured in different distribution devices.

[0172] The video surveillance system proposed in this embodiment can switch to a backup access device to receive video streams during access device failures, so as to receive complete video streams and perform data processing, thereby avoiding the impact on the completion of video surveillance tasks and the user experience.

[0173] In this embodiment, the video streaming device in the video surveillance system is presented in the form of a functional unit. Here, a unit refers to an application-specific integrated circuit (ASIC), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0174] This invention also provides a computer device having a video stream transmission device in the video surveillance system shown in FIG7.

[0175] Please refer to Figure 8, a schematic diagram of a computer device according to an optional embodiment of the present invention. The computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 shows an example of a single processor 10.

[0176] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPRS), or any combination thereof. Memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the methods shown in the above embodiments.

[0177] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 20 may include volatile memory, such as random access memory. The memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. The memory 20 may also include combinations of the above types of memory. The computer device also includes a communication interface 30 for communication between the computer device and other devices or communication networks.

[0178] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0180] Cross-references to related applications

[0181] This application claims priority to Chinese patent application No. 202411533542.5, filed on October 31, 2024, the entire contents of which are incorporated herein by reference. Industrial applicability

[0182] This invention can be widely applied to EEG data management, primarily in clinical hospitals, to help doctors manage and access EEG data more efficiently, improving the accuracy and efficiency of diagnosis and treatment. Simultaneously, it can also be applied to scientific research, enabling researchers to access and analyze data in batches, thus promoting the development of EEG-related research.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video stream transmission method in a video surveillance system, characterized in that, The system includes an access device, a backup access device, and a processing device; the method includes: The backup access device parses the video stream continuously sent by the camera device to obtain multiple frame images in sequence, and caches the latest N frame groups; wherein, the multiple frame images include multiple frame groups arranged in sequence, each frame group includes a first keyframe and at least one prediction frame arranged in sequence, the at least one prediction frame includes each frame image located after the first keyframe and before the second keyframe, the second keyframe being the keyframe following the first keyframe; N is an integer not less than 1; For any obtained frame, the backup access device determines that the frame is a key frame. Then, it generates a corresponding hash value based on a preset hash algorithm and the frame, and uses it as a frame group identifier. Starting from a preset initial order, it records the acquisition order of the frame and the frame in the same group to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group. It then creates frame information for the frame and the frame in the same group, including the frame group identifier and the acquisition order of the frame in the corresponding frame group. The backup access device receives a first connection instruction sent by the processing device; wherein, the first connection instruction is generated when the processing device determines that the access device has failed and disconnects from the access device during data processing of the video stream received through the access device, and the first connection instruction includes a target hash value and a target acquisition order, wherein the target hash value is generated by the processing device according to the preset hash algorithm and the target keyframe, the target acquisition order is the acquisition order of the target frame in the corresponding frame group, and the target keyframe and the target frame are respectively the latest keyframe and the latest frame obtained and processed by the processing device according to the video stream before disconnecting from the access device; The backup access device determines the next frame of the target frame from the N currently cached frame groups based on the target hash value, the order in which the target is acquired, and the frame information in the first connection instruction, and sends the video stream to the processing device to continue data processing starting from the next frame.

2. The method according to claim 1, characterized in that, Before the cached N most recently acquired frame groups, the following is also included: The backup access device determines the handover duration, maximum frame transmission delay, frame transmission rate, and number of frames in the frame group; wherein, the handover duration is the time required for the processing device to successfully switch to the backup access device after determining that the access device has failed. The backup access device inputs the switching duration, the maximum frame transmission delay, the frame transmission rate, and the number of frames contained in the frame group into a preset model for calculation, obtains the output value of the preset model, determines an integer that is not less than the output value and is closest to the output value, and uses it as N; The preset model is as follows: M=(S0+S τ )×FPS / GOP; M is the variable corresponding to the output value, S0 is the variable corresponding to the switching duration, and S τ The variable corresponding to the maximum frame transmission delay is denoted by FPS, the variable corresponding to the frame transmission rate is denoted by FPS, and the variable corresponding to the number of frames is denoted by GOP.

3. The method according to claim 1, characterized in that, When the processing device is a distribution device, the first connection instruction is generated when the distribution device determines that the access device has failed and disconnects from the access device during the process of forwarding the video stream received through the access device to the downstream device; Starting from the next frame, sending the video stream to the processing device for continued data processing includes: The backup access device sends the video stream to the distribution device starting from the next frame after the target frame, so that the distribution device continues to forward the received video stream to the downstream device.

4. The method according to claim 1, characterized in that, When the processing device is a storage device, the first connection instruction is generated when the storage device determines that the access device has malfunctioned and disconnected from the access device during the storage of the video stream received through the access device; Starting from the next frame, sending the video stream to the processing device for continued data processing includes: The backup access device sends the video stream to the storage device starting from the next frame after the target frame, so that the storage device continues to store the received video stream.

5. The method according to claim 1, characterized in that, During data processing of the video stream received through the access device, if the processing device determines that the access device has malfunctioned and disconnected from the access device, it generates the first connection instruction, configured as follows: When the processing device obtains the first keyframe from the video stream received through the access device and has processed the first keyframe, it generates a corresponding first hash value according to the preset hash algorithm and the first keyframe, determines the first acquisition order of the first keyframe in the corresponding screen group, and saves the first hash value and the first acquisition order to a preset information record table. When the processing device obtains a new keyframe from the video stream received through the access device and has processed the new keyframe, it generates a corresponding new hash value according to the preset hash algorithm and the new keyframe, replaces the hash value currently stored in the information record table with the new hash value, and saves it. And, when a new frame is acquired from the video stream received by the access device and the new frame is processed, the new acquisition order of the new frame in the corresponding frame group is determined, the acquisition order currently stored in the information record table is replaced with the new acquisition order, and then saved. During the data processing of the video stream received through the access device, when the processing device determines that the access device has malfunctioned and is disconnected from the access device, it uses the hash value currently stored in the information record table and the acquisition order as the target hash value and the target acquisition order, respectively. The processing device generates a first connection instruction that includes the target hash value and the order in which the targets are acquired.

6. The method according to claim 5, characterized in that, The system also includes a backup processing device and a service monitoring device; After sending the video stream to the processing device for further data processing starting from the next frame, the method further includes: The backup access device stops sending the video stream to the processing device in response to a stop command sent by the service monitoring device; wherein the stop command is generated by the service monitoring device when it determines that the processing device has malfunctioned based on a status message sent by the processing device. The backup access device receives a second connection instruction sent by the backup processing device; wherein, the second connection instruction is generated by the backup processing device in response to a start instruction sent by the service monitoring device, based on the hash value currently stored in the information record table and the order in which they are acquired; the start instruction is generated by the service monitoring device when it determines that the processing device has failed based on a status message sent by the processing device. The backup access device sends the video stream to the backup processing device for further data processing according to the second connection instruction.

7. A video stream transmission device in a video surveillance system, characterized in that, The system includes an access device, a backup access device, and a processing device. The video streaming device is used in the backup access device; The video stream transmission device includes: A parsing unit is used to parse the video stream continuously sent by the camera device to obtain multiple frame images in sequence; wherein, the multiple frame images include multiple frame groups arranged in sequence, each frame group includes a first keyframe and at least one prediction frame arranged in sequence, the at least one prediction frame includes each frame image located after the first keyframe and before a second keyframe, the second keyframe being the keyframe following the first keyframe; N is an integer not less than 1. The cache unit is used to cache the most recently acquired N frame groups; A creation unit is configured to, for any obtained frame, determine that the frame is a keyframe, generate a corresponding hash value based on a preset hash algorithm and the frame, and use it as a frame group identifier, record the acquisition order of the frame and the frame in the same group starting from a preset initial order, so as to obtain the acquisition order of the frame and the frame in the same group in the corresponding frame group, and create frame information of the frame and the frame in the same group, the frame information including the frame group identifier and the acquisition order of the frame in the corresponding frame group; The first receiving unit is configured to receive a first connection instruction sent by the processing device; wherein the first connection instruction is generated when the processing device determines that the access device has malfunctioned and disconnects from the access device during data processing of the video stream received through the access device; the first connection instruction includes a target hash value and a target acquisition order; the target hash value is generated by the processing device according to the preset hash algorithm and the target keyframe; the target acquisition order is the acquisition order of the target frame in the corresponding frame group; and the target keyframe and the target frame are respectively the latest keyframe and the latest frame obtained and processed by the processing device according to the video stream before disconnecting from the access device. The first determining unit is configured to determine the next frame of the target frame in the currently cached N frame groups based on the target hash value, the order in which the target is acquired, and the frame information in the first connection instruction. The first sending unit is used to send the video stream to the processing device for continued data processing starting from the next frame.

8. The device according to claim 7, characterized in that, When the processing device is a distribution device, the first connection instruction is generated when the distribution device determines that the access device has failed and disconnects from the access device during the process of forwarding the video stream received through the access device to the downstream device; The first transmitting unit is further configured to: Starting from the next frame after the target frame, the video stream is sent to the distribution device so that the distribution device continues to forward the received video stream to the downstream device.

9. The device according to claim 7, characterized in that, When the processing device is a storage device, the first connection instruction is generated when the storage device determines that the access device has malfunctioned and disconnected from the access device during the storage of the video stream received through the access device; The first transmitting unit is further configured to: Starting from the next frame after the target frame, the video stream is sent to the storage device so that the storage device continues to store the received video stream.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the video stream transmission method in the video surveillance system according to any one of claims 1 to 6.

Images