Video stream transmission method and system based on adaptive stream control and protocol conversion

Abstract

The invention provides a video stream transmission method and system based on adaptive stream control and protocol conversion, and the system comprises a video stream data access unit which is used for the access of video stream data, for the obtained video stream data, based on a communication protocol adopted by a video terminal, in combination with a pre-constructed dynamic protocol library, adapter matching is carried out, and the video stream data is transmitted to a video terminal; analyzing the video stream data based on the matched adapter to realize data access; the video stream data output unit is used for displaying the accessed video stream data, and the display of the video stream data specifically comprises the steps of obtaining hardware characteristics and requirements of a display terminal, and performing automatic output control on the video stream data by combining a preset on-demand adjustment strategy based on the hardware characteristics and the requirements; and the video stream data storage unit is used for storing the accessed video stream data, and the video stream data is stored based on a preset on-demand storage strategy.

Description

Technical Field

[0001] The present invention belongs to the technical field of video transmission, and in particular relates to a video stream transmission method and system based on adaptive flow control and protocol conversion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of Internet infrastructure, especially the popularization of 5G communication technology, video content has become one of the main forms of information exchange. However, in the field of video surveillance and other remote video applications, there are still many challenges. In the existing technology, video access standards are different and there are many types of protocols, which leads to frequent compatibility problems. In addition, although bandwidth conditions have improved, how to efficiently use limited network resources during large-scale deployment and global video distribution, especially to ensure high-quality video transmission in low-bandwidth environments, is still a technical problem that needs to be solved urgently. Especially in the field of video surveillance, these problems are particularly prominent, such as the diversification of video access protocols, limited large-stream video transmission, poor transmission protocol compatibility, reduced browser plug-in support, and the need to develop special apps to view videos on mobile devices.

[0004] The inventors have found that the existing video streaming transmission solutions have the following problems: the wide variety of video access protocols leads to poor system compatibility, increases the difficulty of integration and management, and may cause communication barriers between different devices; bandwidth limitations make it impossible to publish massive large-code stream videos, limit the clarity and smoothness of the video, and make it difficult to meet users' demand for high-quality videos; the transmission protocol is limited under the Internet system, which limits the flexibility and efficiency of video transmission, and may lead to the inability to achieve effective video transmission in certain scenarios; browser upgrades have led to the gradual disabling of web page embedded plug-in mode, which has limited the display of traditional video surveillance on the web page, affecting the user experience; viewing videos on mobile phones requires the development of a special APP, which is difficult to develop and maintain, increases costs and technical barriers, and may also cause inconvenience to users. Summary of the invention

[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a video stream transmission method and system based on adaptive flow control and protocol conversion to solve the problems that traditional video stream transmission solutions have poor compatibility and cannot automatically adapt to the requirements of different scenarios.

[0006] According to a first aspect of an embodiment of the present invention, a video stream transmission system based on adaptive flow control and protocol conversion is provided, comprising:

[0007] A video stream data access unit is used for accessing video stream data, wherein, for the obtained video stream data, based on the communication protocol adopted by the video terminal and in combination with a pre-built dynamic protocol library, an adapter is matched, and the video stream data is parsed based on the matched adapter to achieve data access;

[0008] A video stream data output unit, which is used to display the received video stream data, wherein the display of the video stream data is specifically: obtaining the hardware characteristics and requirements of the display terminal, and based on the hardware characteristics and requirements, combining with a preset on-demand adjustment strategy, performing automatic output control of the video stream data;

[0009] The video stream data storage unit is used to store the received video stream data, wherein the video stream data is stored based on a preset on-demand storage strategy.

[0010] Furthermore, the dynamic protocol library stores several protocols and their matching adapters, wherein the dynamic protocol library adopts a dynamic update mechanism. When the communication protocol adopted by the connected video terminal fails to be queried in the dynamic protocol library or the protocol is updated, the current communication protocol information is automatically obtained from the device or network and updated into the dynamic protocol library.

[0011] Furthermore, the communication protocol adopted by the video terminal is combined with a pre-built dynamic protocol library to match the adapter, specifically: based on the accessed video stream data, a layered parsing strategy is used to identify the communication protocol; based on the identified communication protocol, a pre-trained machine learning-based protocol feature extraction model is used to extract the features of the communication protocol; based on the extracted features of the current communication protocol, a similarity calculation is performed with the protocol features in the dynamic protocol library, and based on the similarity calculation result, the adapter is matched.

[0012] Furthermore, the on-demand adjustment strategy is specifically as follows: obtaining the type of terminal to be displayed, device performance, network bandwidth and preset requirements of the display terminal; based on the terminal type, device performance, network bandwidth and preset requirements, using a pre-trained deep learning-based video stream data parameter adjustment model to obtain adjustment parameters; based on the adjustment parameters, automatically adjusting the video stream data on demand.

[0013] Furthermore, during the video stream data transmission process, the bit rate and resolution are adaptively adjusted based on the network conditions.

[0014] Furthermore, the access to the video stream data can also be: based on the SDK provided by the video terminal manufacturer, secondary development of the protocol is carried out through a general integration framework, and docking with different video terminal SDKs to achieve access to the video terminal data; or, access to the video terminal data is carried out through a preset open protocol.

[0015] Furthermore, the on-demand storage strategy specifically includes a method of performing scheduled storage according to a preset strategy and performing storage based on a business storage instruction.

[0016] According to a second aspect of an embodiment of the present invention, a video stream transmission method based on adaptive flow control and protocol conversion is provided, which is applied to the above-mentioned video stream transmission system based on adaptive flow control and protocol conversion, including:

[0017] For the obtained video stream data, based on the communication protocol adopted by the video terminal and combined with the pre-built dynamic protocol library, the adapter is matched, and the video stream data is parsed based on the matched adapter to achieve data access;

[0018] Acquire the hardware characteristics and requirements of the display terminal, and automatically adjust the access video stream data based on the hardware characteristics and requirements and in combination with a preset on-demand adjustment strategy;

[0019] The adjusted video stream data is displayed, and the accessed video stream data is stored.

[0020] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the above-mentioned video stream transmission method based on adaptive flow control and protocol conversion is implemented.

[0021] According to a fourth aspect of this embodiment, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, the above-mentioned video stream transmission method based on adaptive flow control and protocol conversion is implemented.

[0022] One or more of the above technical solutions have the following beneficial effects:

[0023] (1) The present invention provides a method and system for video stream transmission based on adaptive flow control and protocol conversion. The scheme is based on a pre-built dynamic protocol library, supports multiple communication protocols, and can ensure seamless access of video data from different sources. The scheme is based on a preset on-demand adjustment strategy, combined with the adaptive adjustment of bit rate and resolution based on network conditions during video stream data transmission, and can automatically adjust video encoding parameters and transmission rate to adapt to changes in the network environment and maximize the use of available bandwidth, so as to ensure that the video transmission scheme can adapt to the ever-changing Internet environment and equipment upgrades.

[0024] (2) The dynamic protocol library of the solution described in the present invention adopts a dynamic update mechanism, which can adapt to new communication protocols and updated iterations of communication protocols.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 A schematic diagram of multi-protocol access described in an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the structure of a video stream transmission system based on adaptive flow control and protocol conversion according to an embodiment of the present invention;

[0029] Figure 3 The flowchart of the video stream transmission method based on adaptive flow control and protocol conversion described in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0031] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0032] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0033] like Figure 2As shown, in one or more implementations, this embodiment provides a video stream transmission system based on adaptive flow control and protocol conversion, including:

[0034] A video stream data access unit is used for accessing video stream data, wherein, for the obtained video stream data, based on the communication protocol adopted by the video terminal and in combination with a pre-built dynamic protocol library, an adapter is matched, and the video stream data is parsed based on the matched adapter to achieve data access;

[0035] In a specific implementation, the dynamic protocol library stores several protocols and their matching adapters, wherein the dynamic protocol library adopts a dynamic update mechanism. When the communication protocol adopted by the connected video terminal fails to be queried in the dynamic protocol library or there is an update to the protocol, the current communication protocol information is automatically obtained from the device or network and updated into the dynamic protocol library.

[0036] In a specific implementation, the communication protocol adopted by the video terminal is combined with a pre-built dynamic protocol library to match the adapter, specifically: based on the accessed video stream data, a layered parsing strategy is used to identify the communication protocol; based on the identified communication protocol, a pre-trained machine learning-based protocol feature extraction model is used to extract the features of the communication protocol; based on the extracted features of the current communication protocol, a similarity calculation is performed with the protocol features in the dynamic protocol library, and based on the similarity calculation result, the adapter is matched.

[0037] In a specific implementation, the access of the video stream data specifically performs the following processing:

[0038] In order to solve the compatibility problem caused by the inconsistency of protocols when devices from different manufacturers are connected, a system that integrates efficient parsing algorithms, intelligent learning mechanisms, dynamic protocol library updates, adaptive parameter adjustments, and multi-protocol parallel processing is used to achieve rapid identification and automatic adaptation of protocol differences between different video manufacturers: First, the efficient parsing algorithm is used to quickly identify and adapt to the slight differences in various video protocols; second, the system uses machine learning technology through an intelligent learning mechanism to continuously optimize the adaptation strategy and improve the accuracy and efficiency of adaptation; third, the system has a dynamic protocol library update function that can be updated in real time to adapt to new video protocols or updates to existing protocols; fourth, the system can perform adaptive parameter adjustments and automatically adjust parameters according to the parsing results, and the adaptation can be completed without manual intervention; finally, the system uses multi-protocol parallel processing technology to process video streams of multiple protocols at the same time, significantly improving the processing power and response speed. Specifically:

[0039] (1) Efficient parsing algorithm:

[0040] The core of the algorithm is to quickly identify the basic structure and commands of the protocol, including pattern recognition, regular expression matching, and fast string analysis methods to distinguish specific patterns and command sets of different protocols.

[0041] The specific algorithm adopts a hierarchical parsing strategy, first identifying the header information of the protocol, then the command and data parts, and finally the tail information.

[0042] The efficient parsing algorithm achieves intelligent identification and automatic adaptation through a series of comprehensive optimization strategies. First, the parsing foundation is established through in-depth analysis and data modeling. Then, pre-compiled parsing rules such as regular expressions or state machines are used to reduce runtime parsing costs. Further data structure optimization, using hash tables, speeds up the search and matching of protocol fields. Parallel processing technology, combined with multi-threading or asynchronous I / O, improves data processing speed. Hardware acceleration, using GPUs and FPGAs, further improves the execution efficiency of parsing tasks. The integration of these strategies not only improves parsing performance, but also enhances the flexibility and scalability of the system.

[0043] The algorithm logic has been streamlined to reduce redundant calculations and conditional judgments, thereby speeding up the processing speed. The introduction of the fast string search algorithm Boyer-Moore significantly improves the efficiency of pattern matching. In addition, the implementation of the cache mechanism avoids multiple parsing of the same data, further improving the parsing efficiency. The abstraction and encapsulation of the protocol not only enhances the reusability of the code, but also simplifies code maintenance and expansion. Compilation optimization and memory management improvements reduce resource waste, and the frequency of memory allocation is reduced through object pool technology, thereby reducing the overhead of garbage collection. Optimization measures for specific protocols, combined with real-time performance monitoring and adaptive adjustment, ensure that the algorithm can flexibly respond to the characteristics of different protocols and maintain efficient parsing performance. These comprehensive measures work together to enable the efficient parsing algorithm to achieve fast, stable and resource-saving automatic adaptation when processing video protocols from different manufacturers.

[0044] (2) Intelligent learning mechanism:

[0045] Use machine learning or artificial intelligence techniques to train on historical data and identify patterns and differences between different protocols.

[0046] Based on the analysis results and user feedback, the algorithm parameters are continuously adjusted and the adaptation strategy is optimized.

[0047] Online learning and incremental learning mechanisms are implemented to adapt to dynamic changes in the protocol.

[0048] The intelligent learning mechanism, through the application of machine learning and artificial intelligence technologies, has become the core way for multi-bitrate streaming technology to improve video streaming quality and user experience. It first collects data, covering user behavior, network and device information, and then creates a model that can predict changing trends through pattern recognition and algorithm training. The parameter adjustment function allows the model to dynamically optimize the video bitrate and buffering strategy based on user feedback and real-time data. Online and incremental learning ensures that the system continues to evolve without repeated training, while adapting to protocol updates and changes. Performance monitoring and privacy protection are key to the operation of the mechanism, while ensuring that feedback from the user interface can be used for continuous improvement of the model. Technology integration ensures seamless collaboration between the intelligent learning mechanism and existing video streaming service technologies. The implementation of this mechanism significantly enhances the intelligence level of multi-bitrate streaming technology, automatically adjusts services to meet individual user needs, and effectively improves user satisfaction and overall service quality.

[0049] (3) Dynamic protocol library update:

[0050] By building a centralized protocol library, all known protocol formats and versions are stored.

[0051] When a new protocol is discovered or an existing protocol is updated, the new protocol information is automatically obtained from the device or network and updated to the library.

[0052] Design efficient library search and update algorithms to minimize the impact on existing services.

[0053] Dynamic protocol library update is the core function of multi-bitrate streaming technology to adapt to changes in video protocols. It is achieved by building a general database that centrally stores multiple video protocols. An automated monitoring system is used to continuously detect new protocols or updates on devices and networks. Once changes are detected, they are automatically acquired and updated to the library in real time. Efficient search algorithms speed up the retrieval and identification of protocols, while intelligent update strategies ensure seamless integration of new protocols while minimizing the impact on user services. Version control and user customization functions provide flexibility and personalization, while security checks ensure the security of updates and prevent risks. In addition, feedback mechanisms and comprehensive documentation and training support further improve the ease of use of the system and user adaptability. These comprehensive measures together ensure that video streaming services can continue to provide the latest and most stable protocol support, enhancing the overall flexibility of the system and user viewing experience.

[0054] (4) Adaptive parameter adjustment:

[0055] Based on the output of the parsing algorithm and the recommendations of the intelligent learning mechanism, the system automatically adjusts the parameters of interaction with the protocol.

[0056] For example, the system can dynamically adjust transmission and decoding parameters for different resolutions, frame rates or encoding formats.

[0057] Adaptive parameter adjustment is a key feature of multi-bitrate streaming technology to ensure the optimization of video transmission and decoding processes. This mechanism uses parsing algorithms and intelligent learning mechanisms to automatically adjust interactive parameters to adapt to the network conditions and device capabilities of different users, with the goal of providing the best viewing experience. It includes dynamically adjusting resolution to match the playback environment, adapting frame rate to maintain smooth playback, optimizing encoding format selection, adjusting transmission parameters such as bitrate and buffering strategy, and optimizing decoding parameters to adapt to hardware performance. In addition, the system achieves personalized experience by learning user preferences and continuously optimizes strategies with the help of feedback loops. These comprehensive measures enable multi-bitrate streaming technology to flexibly adapt to various conditions, ensuring that users can enjoy high-quality and smooth video playback regardless of the environment.

[0058] (5) Multi-protocol parallel processing:

[0059] Design an efficient parallel processing framework that allows the system to process video streams from multiple protocols simultaneously.

[0060] Use multithreading or distributed computing resources to increase system processing power and response speed.

[0061] Implement load balancing strategies to ensure efficient allocation of resources.

[0062] Multi-protocol parallel processing is the core feature of multi-bitrate streaming technology that improves the system's multi-protocol video stream processing capabilities. This feature is achieved by developing an efficient parallel processing framework that allows the system to process multiple video streams simultaneously. The system is able to enhance its processing capabilities by utilizing multi-threading and distributed computing resources, and to reasonably allocate resources through load balancing strategies to adapt to the real-time needs of video streams. In addition, the system has good scalability and can be horizontally expanded to cope with growing demand, and fault tolerance and fault recovery processes are designed to ensure high availability and continuity of services. Performance monitoring ensures processing speed and efficiency, while data synchronization ensures the coherence of video streams with different protocols. The system also supports encoding compatibility, can process and transmit video streams in multiple encoding formats, and provides user customization options, allowing users to select specific protocols or encoding formats according to their needs. These comprehensive strategies not only improve the overall performance and responsiveness of the system, but also greatly enrich the user's viewing experience and ensure the smoothness and diversity of video playback.

[0063] In one or more embodiments, the access to the video terminal data can also be: based on the SDK provided by the video terminal manufacturer, secondary development of the protocol is performed through a general integration framework, and docking with different video terminal SDKs to achieve access to the video terminal data; or, access to the video terminal data is performed through a preset open protocol.

[0064] In specific implementation, the solution described in this embodiment can automatically adapt to protocols of different manufacturers, which is specifically reflected in:

[0065] (1) Automatically adapt to protocols of different manufacturers:

[0066] ①Data source:

[0067] For existing platforms of other manufacturers, such as Hikvision, Dahua, Keda, H3C, Fushengxing, Uniview, etc., we can connect them through the universal GB-28181 protocol, shield the protocols of different manufacturers, and aggregate the videos into the system. This system has the experience of connecting with different manufacturers, and has made automatic adaptation for the slight differences in the protocols of different manufacturers.

[0068] ②Data reception:

[0069] The system first needs to receive audio and video stream data from different video platforms.

[0070] ③Data processing:

[0071] The system uses built-in adapters or converters to identify and parse protocols from different manufacturers and automatically adjust parameters to suit the specific requirements of different protocols.

[0072] ④Results:

[0073] It can achieve seamless connection of devices from different manufacturers without manual configuration, which improves the usability and flexibility of the system. ⑤Solve the problem:

[0074] It solves the compatibility problem caused by inconsistent protocols when devices from different manufacturers are connected.

[0075] Through a system that integrates efficient parsing algorithms, intelligent learning mechanisms, dynamic protocol library updates, adaptive parameter adjustments, and multi-protocol parallel processing, rapid identification and automatic adaptation of protocol differences among different video manufacturers are achieved.

[0076] First, it uses an efficient parsing algorithm that can quickly identify and adapt to the slight differences between various video protocols; second, the system uses an intelligent learning mechanism and machine learning technology to continuously optimize the adaptation strategy and improve the accuracy and efficiency of adaptation; third, the system has a dynamic protocol library update function that can be updated in real time to adapt to newly emerging video protocols or updates to existing protocols; fourth, the system can perform adaptive parameter adjustment and automatically adjust parameters according to the parsing results, and can complete the adaptation without human intervention; finally, the system uses multi-protocol parallel processing technology, which can process video streams of multiple protocols at the same time, significantly improving processing capabilities and response speed.

[0077] (1) SDK-level docking of mainstream manufacturers:

[0078] ①Data source:

[0079] For existing platforms of other manufacturers, such as Hikvision, Dahua, Keda, H3C, Fushengxing, Uniview, etc., you can connect them through the SDK secondary development protocol provided by the manufacturer to aggregate the video into the system. This system has experience in connecting with different manufacturers, and has made automatic adaptations for the slight differences in the protocols of different manufacturers.

[0080] ②Data reception:

[0081] The system receives audio and video data by integrating SDKs provided by mainstream manufacturers.

[0082] ③Data processing:

[0083] The system uses the API provided by the SDK to acquire, process and publish audio and video data.

[0084] ④Results:

[0085] Ensures deep integration with mainstream manufacturers' equipment and optimal performance.

[0086] ⑤Solve the problem:

[0087] It solves the technical barriers and performance issues that may be encountered when integrating with equipment from specific manufacturers.

[0088] By designing a universal integration framework, the process of connecting to different SDKs can be greatly simplified. This framework reduces the workload of customizing the integration for each SDK by providing a set of standardized interfaces and processes. Using automated tools, we can analyze the APIs of various SDKs and automatically generate an adaptation layer. This adaptation layer can unify the calling interfaces of different SDKs, allowing developers to ignore the differences in the underlying SDKs and make API calls in a unified manner, significantly improving the efficiency of integration. We have deeply optimized the API calling mode and effectively improved the overall performance of the system by reducing the number of unnecessary API calls and implementing a caching strategy for request results.

[0089] (3) Directly connect to other video sources through the open protocol onvif:

[0090] ①Data source:

[0091] For some devices with a relatively small market share and unstable SDK, this system can directly connect to this system through some open protocols, such as the onvif protocol recognized in the security field, or the more open RTSP, RTMP in the video field, or even the RTP protocol sent only through UDP, avoiding the situation where the entire system is paralyzed due to the connection of unstable devices. Some special video sources and video streams not in the security field can also be connected to this system as long as they support these protocols.

[0092] ②Data reception:

[0093] The system receives data from external video sources through open protocol standards.

[0094] ③Data processing:

[0095] The system performs format conversion and stability enhancement processing on the received data.

[0096] ④Results:

[0097] Effective management and integration of unstable equipment is achieved, avoiding the risk of system paralysis.

[0098] ⑤Solve the problem:

[0099] The impact of unstable external video source on the stability of the entire system is resolved.

[0100] (4) For other video sources:

[0101] ①Data source:

[0102] This system can connect computer desktop, microphone input, audio output, computer USB camera and external camera, add special effects synthesis and publish, and can be viewed directly by remote browser and mobile phone;

[0103] ②Data reception:

[0104] The system receives multiple audio and video data such as computer desktop, microphone input, audio output, USB camera, etc.

[0105] ③Data processing:

[0106] The system synthesizes these data, which may include image stitching, audio mixing, special effects addition, etc.

[0107] ④Results:

[0108] Generates comprehensive audio and video streams with rich visual and sound effects.

[0109] ⑤Solve the problem:

[0110] It provides an innovative way of content publishing and enhances the user experience.

[0111] (5) Extensive audio / video codec support:

[0112] ①Data source:

[0113] This system supports a wide range of encoding methods, not limited to video, but also including audio encoding methods, as listed below:

[0114] Audio: G711, G729, MP3, AAC, WMA and dozens of other mainstream formats.

[0115] Video: dozens of mainstream encoding methods such as h264, h263, h265, wmv, xvid, d ivx, mpeg2, rm, etc.

[0116] ②Data reception:

[0117] The system receives audio and video data in different encoding formats.

[0118] ③Data processing:

[0119] The system decodes and re-encodes the data to adapt to different playback environments and devices.

[0120] ④Results:

[0121] The system's compatibility and adaptability have been improved, making it capable of playing audio and video content in various formats.

[0122] ⑤Solve the problem:

[0123] This solves the problem of playing audio and video content in different encoding formats on different devices.

[0124] A video stream data output unit, which is used to display the received video stream data, wherein the display of the video stream data is specifically: obtaining the hardware characteristics and requirements of the display terminal, and based on the hardware characteristics and requirements, combining with a preset on-demand adjustment strategy, performing automatic output control of the video stream data;

[0125] In a specific implementation, the on-demand adjustment strategy is specifically as follows: obtaining the type of terminal to be displayed, device performance, network bandwidth and preset requirements of the display terminal; based on the terminal type, device performance, network bandwidth and preset requirements, using a pre-trained deep learning-based video stream data parameter adjustment model to obtain adjustment parameters; based on the adjustment parameters, performing on-demand automatic adjustment of video stream data.

[0126] Specifically, in the output of the video stream data, the following processing is specifically performed:

[0127] In terms of video output, in response to the unique needs of users due to network bandwidth and device requirements, the solution described in this embodiment provides a flexible solution to meet different playback environments.

[0128] In the specific implementation, the supported output protocols include RTMP, FLV, HLS and UDP, ensuring that the video stream can be played on multiple devices and platforms. In terms of web page playback, you can use Flash plug-ins or third-party technologies to play RTMP streams, or use the HLS protocol to achieve cross-platform playback. Video playback on mobile devices is achieved through RTMP decoding modules or HLS support.

[0129] For video content with different resolutions, the system can perform resolution compression and transcoding based on the display requirements of mobile devices and network conditions, balancing display effects and bandwidth usage. In addition, users can specify video resolution to adapt to different display devices and avoid playback anomalies.

[0130] In terms of bandwidth optimization, the system can reduce bandwidth usage by lowering the video bit rate in an environment with traffic billing or bandwidth constraints, while maintaining acceptable video quality for users. In terms of encoding, the system uses the most common H265 video encoding and AAC audio encoding on the market to ensure the compatibility and efficiency of video output.

[0131] In the specific implementation, facing the limitations of network bandwidth and device performance, it is particularly important to implement an adaptive video processing strategy. This strategy first involves real-time collection of user device information and network status, and analysis of their specific needs for video resolution, bandwidth, encoding method and communication protocol. Then, an intelligent learning algorithm is applied to dynamically adjust video parameters to adapt to user needs, and the video output settings are determined in combination with preset strategies. The system supports multiple video output protocols such as RTMP, FLV, HLS and UDP to ensure cross-platform playback compatibility. At the same time, the system automatically compresses and transcodes high-resolution videos based on device and bandwidth conditions, and uses efficient encoding methods such as H265 and AAC to optimize video quality and file size. Bandwidth optimization measures ensure that acceptable video quality can be maintained even in restricted networks. In addition, a user interface is provided to allow users to customize playback settings according to their personal preferences, and a feedback loop is set up to continuously optimize the playback strategy based on user feedback. Finally, a quality control process is used to ensure high-quality playback experience in different environments.

[0132] The video stream data storage unit is used to store the received video stream data, wherein the video stream data is stored based on a preset on-demand storage strategy.

[0133] In a specific implementation, in view of the demand for video storage, the solution described in this embodiment may be with storage or without storage, and two versions may be set.

[0134] ①Hardware requirements:

[0135] The system has low hardware requirements, which means it can run on a variety of hardware configurations and has good compatibility.

[0136] ②General format and encoding method:

[0137] The system supports a variety of common audio and video formats and encoding methods, which improves its applicability and compatibility.

[0138] ③ Storage:

[0139] The system uses MySQL database for data storage, which not only facilitates data retrieval, but also supports authentication functions and enhances data security.

[0140] ④Directory structure:

[0141] The system's directory structure is designed to be simple, making it easy for users to manage and maintain it.

[0142] ⑤ On-demand storage:

[0143] The system supports on-demand storage strategy, which means that users can store data according to actual needs, thus saving storage space in terms of time and space.

[0144] In terms of storage hardware, whether it is external NAS storage or network distributed storage, as long as the hardware meets the storage rate requirements, there are no special requirements for the physical hardware form of the storage. Next, we will introduce the file storage format, file retrieval, storage management and use, and on-demand storage.

[0145] In the specific implementation, the file storage format of the solution described in this embodiment adopts the most universal mp4 (H265 / AAC) storage format, which is very common internationally, including Apple system, Linux system, Android system and the most common Windows system. MP4 (H264 / AAC) is the most universal format and can be directly provided to HTML5 for playback. This format is a format that is natively supported by general systems and is the most universal format, including the video encoding method used for international network transmission and the file storage format.

[0146] In the specific implementation, the file retrieval uses the open source relational database MySQL, which is convenient for other developers to retrieve and manage. For the storage and management of files, since the database MySQL is used to store and retrieve video files, management is very convenient. In addition, the video files we store have the unique identification of the device and the storage time, which can be easily provided to users using the FTP service.

[0147] In one or more implementations, if for security reasons, you can manage the files yourself and save the saved files to a safer place for storage and use. We can also provide storage rules.

[0148] In one or more implementations, an on-demand storage method is adopted, specifically:

[0149] Since security equipment runs for a long time, but the files we need to store may not be for the whole period, the start and end of storage are closely related to business operation. Therefore, we can use the following two methods for storage, specifically:

[0150] (1) Scheduled storage. You can set time-related scheduled storage from Monday to Sunday. Different devices use different scheduled storage methods.

[0151] (2) Service-related storage: that is, after receiving the start storage signaling, storage starts immediately; after receiving the stop storage signaling, the ongoing storage stops.

[0152] By integrating H.265 video compression technology and deep learning algorithms, adaptive data compression is achieved, significantly optimizing the storage efficiency of video files. The high compression rate of H.265 significantly reduces the file size without sacrificing video quality. The deep learning model accurately analyzes the video content, identifies key frames and motion areas, and then dynamically adjusts quantization parameters and encoding strategies to achieve scene-adaptive compression. By supporting a wide range of video formats and encoding standards, on-demand storage is achieved, storage costs are optimized, and flexible scalability is provided to adapt to the growing amount of data, ensuring its high versatility and adaptability under a variety of storage needs and conditions.

[0153] In one or more embodiments, during the video stream data transmission process, adaptive adjustment of bit rate and resolution is performed based on network conditions, that is, on-demand transcoding.

[0154] Specifically, on-demand transcoding is a key intelligent feature in video streaming services, which allows the system to automatically decide whether the video stream needs to be transcoded based on the current network conditions and user needs. This strategy first involves the system monitoring the network bandwidth and device capabilities in real time to evaluate whether the existing encoding methods, resolutions, and bandwidth conditions for video playback are met. If these conditions are already adapted to the playback environment, no transcoding is required, thus saving resources. The system has the ability to convert multiple video streaming protocols into a single output protocol to adapt to a specific playback environment, such as uniformly converting the input to RTMP, HLS, or other protocols.

[0155] When it is detected that the existing video parameters do not meet the playback requirements, the system will intelligently transcode and adjust the video encoding, resolution or bit rate to ensure video quality and smooth playback. This intelligent transcoding decision process is driven by an algorithm that analyzes the video source and playback environment in real time to optimize video output. The system also supports real-time transcoding, allowing video parameters to be adjusted according to dynamic changes in network conditions during video streaming.

[0156] The system is designed with user-defined options, allowing users to select video quality based on their personal preferences. At the same time, optimized resource management ensures that unnecessary resource consumption is avoided during the transcoding process, and the system design takes into account compatibility and scalability with future video coding standards and network technologies. Through these measures, the system can flexibly respond to different network challenges and provide users with high-quality and stable video playback experience, no matter what network environment they are in.

[0157] In specific implementation, the system's real-time transcoding capabilities are mainly reflected in the following aspects:

[0158] (1) Dynamic response to network conditions:

[0159] During the video streaming process, it is possible to monitor changes in network conditions in real time, such as fluctuations in network bandwidth, increase or decrease in latency, etc. When the network bandwidth suddenly becomes narrow, the system can quickly reduce the video bit rate and resolution for transcoding to ensure smooth video playback; and when the network conditions improve, the video quality can be improved in a timely manner.

[0160] For example, when users are watching high-definition videos, they switch from a Wi-Fi environment to a mobile data network, and the network bandwidth drops significantly. The system can instantly transcode the video to a lower bit rate and resolution so that the video will not be stuck.

[0161] (2) Seamlessly adjust video parameters:

[0162] The video encoding parameters (such as frame rate, bit rate, resolution, etc.) can be adjusted in real time without interrupting video playback. Users will not notice obvious pauses or interruptions during viewing, and always maintain a coherent viewing experience.

[0163] For example, due to network reasons, the solution described in this embodiment can transcode a 4K video being played to 1080p or even a lower resolution within a few seconds without interruption in the playback process.

[0164] (3) Adapt to various video formats:

[0165] Regardless of the format and encoding method of the input video source, the solution described in this embodiment can transcode it into a format suitable for the current network and playback device in real time.

[0166] Assuming that the input video is in a relatively uncommon coding format, the solution described in this embodiment can immediately transcode it into a common format that can be played smoothly.

[0167] (4) Quickly handle emergencies:

[0168] When the network experiences a brief failure or instability, the solution described in this embodiment can quickly cache the current video data and quickly resume transcoding and playback after the network is restored, thereby reducing the impact on the user's viewing experience.

[0169] For example, if the network is suddenly interrupted for a few seconds, the solution described in this embodiment can use cached data to maintain playback and quickly catch up with real-time transcoding after the network is restored.

[0170] In specific implementation, the system's real-time transcoding capability has the following advantages:

[0171] (1) Providing a smooth viewing experience:

[0172] It can adjust video parameters according to real-time changing network conditions to ensure smooth video playback. For example, when users are on the move and the network signal is unstable, real-time transcoding can keep the video relatively smooth without frequent buffering or sudden drop in quality.

[0173] (2) Adapt to diverse network environments:

[0174] Whether in a high-speed and stable broadband network or a mobile network with weak signals, the normal playback of videos can be guaranteed. For example, if a user moves from an office with high-speed Wi-Fi to a subway with poor mobile network coverage, the system can adjust the transcoding strategy in real time to adapt to network changes.

[0175] (3) Improve resource utilization efficiency:

[0176] Transcoding is performed only when necessary, avoiding the waste of storage and computing resources caused by pre-transcoding all videos. Taking the video platform as an example, there is no need to prepare video versions with multiple bit rates and resolutions for each user in advance, saving a lot of storage space.

[0177] (4) Meet the needs of different devices:

[0178] It can transcode videos into formats and parameters suitable for playback on various terminal devices, so that better playback effects can be obtained on devices with different performances. For example, high-definition videos can be transcoded in real time into versions suitable for playback on old mobile phones or low-performance tablets.

[0179] (5) Reduce costs:

[0180] This reduces the time and effort required to prepare multiple pre-transcoded video versions to cope with various network and device conditions. At the same time, optimized resource utilization also reduces server and bandwidth costs.

[0181] (6) Enhance user satisfaction:

[0182] Always provide users with high-quality, uninterrupted video services to improve user satisfaction and loyalty to the service. Users will not frequently encounter video playback problems due to changes in the network environment, and will be more willing to continue using the service.

[0183] (7) Improve service flexibility and scalability:

[0184] It is easy to cope with new network technologies and equipment changes that may appear in the future. There is no need for large-scale system transformation and upgrades. New demands can be adapted simply by optimizing transcoding algorithms and strategies.

[0185] In a specific implementation, the on-demand transcoding specifically includes the following steps:

[0186] (1) The initial processing of the input video stream includes two main steps: decoding and analysis. First, the video stream is converted into raw image frames and audio data through a decoding process. This step involves parsing popular video encoding formats, such as H.264 and H.265, which ensure efficient compression and compatibility of video data. Subsequently, the decoded data is analyzed in detail to identify key information of the video, including resolution, frame rate, and color space, which determine the clarity and smoothness of the video. At the same time, audio parameters such as sampling rate and number of channels are also identified and analyzed to ensure the quality of the sound.

[0187] (2) The process by which the system determines the transcoding strategy is a collection of multi-level intelligent decisions that focus on optimizing the video to suit the target playback environment. The process begins by selecting the appropriate encoding format, H.265, to match the target device and network conditions. Next, the system intelligently adjusts the video resolution and bit rate to ensure compatibility and smooth playback on different devices and bandwidths. During the encoding process, the system performs frame type selection, motion estimation, and quantization to optimize compression efficiency and video quality. Audio data is also adaptively encoded to ensure sound quality. Real-time monitoring and feedback mechanisms ensure the stability and efficiency of transcoding, while optimization algorithms such as fast search and predictive coding algorithms further improve transcoding speed and quality. Hardware acceleration technology, using GPU acceleration, is used to increase processing speed.

[0188] (3) Bitrate control is a key step in transcoding. It uses algorithms to balance video quality and bandwidth limitations to ensure a stable playback experience under different network conditions. The real-time monitoring and adjustment mechanism ensures transcoding efficiency and stability, and adjusts transcoding parameters based on real-time feedback from system resources and network conditions.

[0189] (4) To improve transcoding efficiency, the system may adopt multi-threading or parallel processing technology, utilize multi-core CPU or distributed computing resources. Error handling capabilities ensure transcoding continuity and output quality, while optimization algorithms such as fast search and prediction algorithms further improve transcoding speed and image quality.

[0190] (5) Hardware acceleration technology accelerates the encoding and decoding process through GPU parallel computing. The transcoding engine uses efficient video encoding libraries x264 and ffmpeg, and configures and optimizes them according to requirements. Audio transcoding selects a suitable encoding algorithm and adjusts parameters such as bit rate. The entire transcoding process will make full use of computing resources, perform error handling and resource management, and ensure the stability and reliability of the system.

[0191] In one or more embodiments, Figure 3 As shown, a video stream transmission method based on adaptive flow control and protocol conversion is provided, which is applied to the above-mentioned video stream transmission system based on adaptive flow control and protocol conversion, including:

[0192] For the obtained video stream data, based on the communication protocol adopted by the video terminal and combined with the pre-built dynamic protocol library, the adapter is matched, and the video stream data is parsed based on the matched adapter to achieve data access;

[0193] Acquire the hardware characteristics and requirements of the display terminal, and automatically adjust the access video stream data based on the hardware characteristics and requirements and in combination with a preset on-demand adjustment strategy;

[0194] The adjusted video stream data is displayed, and the accessed video stream data is stored.

[0195] In further embodiments, there is also provided:

[0196] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, no further description is given here.

[0197] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0198] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0199] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0200] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0201] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in the present embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0202] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video streaming transmission system based on adaptive flow control and protocol conversion, characterized in that: include: A video stream data access unit is used for accessing video stream data, wherein, for the obtained video stream data, based on the communication protocol adopted by the video terminal and in combination with a pre-built dynamic protocol library, an adapter is matched, and the video stream data is parsed based on the matched adapter to achieve data access; A video stream data output unit, which is used to display the received video stream data, wherein the display of the video stream data is specifically: obtaining the hardware characteristics and requirements of the display terminal, and based on the hardware characteristics and requirements, combining with a preset on-demand adjustment strategy, performing automatic output control of the video stream data; The video stream data storage unit is used to store the received video stream data, wherein the video stream data is stored based on a preset on-demand storage strategy.

2. The video stream transmission system based on adaptive flow control and protocol conversion as claimed in claim 1, characterized in that: The dynamic protocol library stores several protocols and their matching adapters, wherein the dynamic protocol library adopts a dynamic update mechanism. When the communication protocol adopted by the connected video terminal fails to be queried in the dynamic protocol library or the protocol is updated, the current communication protocol information is automatically obtained from the device or network and updated into the dynamic protocol library.

3. The video stream transmission system based on adaptive flow control and protocol conversion as claimed in claim 1, characterized in that: The communication protocol adopted by the video terminal is combined with a pre-built dynamic protocol library to match the adapter, specifically: based on the accessed video stream data, a layered parsing strategy is used to identify the communication protocol; based on the identified communication protocol, a pre-trained machine learning-based protocol feature extraction model is used to extract the features of the communication protocol; Based on the extracted features of the current communication protocol, similarity calculation is performed with the protocol features in the dynamic protocol library, and based on the similarity calculation results, adapter matching is performed.

4. The video stream transmission system based on adaptive flow control and protocol conversion as claimed in claim 1, characterized in that: The on-demand adjustment strategy specifically comprises: obtaining the type of terminal to be displayed, device performance, network bandwidth and preset requirements of the display terminal; based on the terminal type, device performance, network bandwidth and preset requirements, using a pre-trained deep learning-based video stream data parameter adjustment model to obtain adjustment parameters; The video stream data is automatically adjusted on demand based on the adjustment parameters.

5. The video stream transmission system based on adaptive flow control and protocol conversion as claimed in claim 1, characterized in that: During the video streaming data transmission process, the bit rate and resolution are adaptively adjusted based on the network conditions.

6. The video stream transmission system based on adaptive flow control and protocol conversion as claimed in claim 1, characterized in that: The access to the video terminal data can also be: based on the SDK provided by the video terminal manufacturer, secondary development of the protocol is carried out through a general integration framework, and docking with different video terminal SDKs to achieve access to the video terminal data; or, access to the video terminal data is carried out through a preset open protocol.

7. The video stream transmission system based on adaptive flow control and protocol conversion as claimed in claim 1, characterized in that: The on-demand storage strategy specifically includes a method of performing scheduled storage according to a preset strategy and performing storage based on a business storage instruction.

8. A video stream transmission method based on adaptive flow control and protocol conversion, which is applied to a video stream transmission system based on adaptive flow control and protocol conversion as claimed in any one of claims 1 to 7, characterized in that: include: For the obtained video stream data, based on the communication protocol adopted by the video terminal and combined with the pre-built dynamic protocol library, the adapter is matched, and the video stream data is parsed based on the matched adapter to achieve data access; Acquire the hardware characteristics and requirements of the display terminal, and automatically adjust the access video stream data based on the hardware characteristics and requirements and in combination with a preset on-demand adjustment strategy; The adjusted video stream data is displayed, and the accessed video stream data is stored.

9. An electronic device comprising a memory and a processor and computer instructions stored in the memory and executed on the processor, characterized in that: When the computer instructions are executed by the processor, the video stream data transmission method as claimed in claim 8 is implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by the processor, the video stream data transmission method as claimed in claim 8 is implemented.

Images