Video monitoring platform streaming media optimization processing method based on video conversion service

Abstract

The application belongs to the technical field of streaming media data processing, and relates to a video monitoring platform streaming media optimization processing method based on video conversion service. The method comprises the following steps: a client initiates a streaming request, the request is sent to a streaming media module, the streaming media module transmits streaming media information to a device access service after creating a channel; the device access service calls an sdk interface according to a related protocol and sends address port information of a platform streaming to a device; the device connects the platform after receiving the information, the platform selects an optimal video conversion service according to a preset strategy, the device sends audio and video data to the selected video conversion service; the video conversion service receives the connection and the audio and video data of the device and forwards the same to the streaming media; the application realizes flexible connection of the device and the streaming media to improve streaming reliability, achieves streaming port multiplexing to reduce security risks, realizes load balancing to optimize resource utilization and improve adaptability by means of the video conversion service, and improves the overall performance of the video monitoring system.

Description

Technical Field

[0001] This invention belongs to the technical field of streaming media data processing technology in video surveillance platforms, specifically relating to a streaming media optimization processing method for video surveillance platforms based on video conversion services. Background Technology

[0002] With the booming development of the security industry, video surveillance platforms are playing an increasingly important role in maintaining public safety and realizing intelligent management. These platforms typically possess key capabilities for receiving and forwarding audio and video, and have specific workflows when dealing with common scenarios involving signaling and stream separation, such as national standard protocols and Uniview's active registration.

[0003] When a platform needs to acquire audio and video streams from numerous access devices and transmit them to the streaming media module, it must carefully select the optimal streaming media to create the receiving and forwarding channel. After receiving information about the streaming media, the device access service forwards it to the device. However, because device streaming depends on a specific selected streaming media address, the connection mode between the streaming media and the device is limited to a one-to-one stateful connection, preventing the device from freely choosing other streaming media for streaming operations.

[0004] Based on this connection method, streaming media platforms must expose ports to communicate with devices, and each platform uses a different port. As business expands, the demands of video surveillance for higher resolution and larger-scale access necessitate that streaming media platforms expand their capacity to improve processing power, while the variety of protocols they support also increases. This trend directly leads to a geometric increase in the number of ports that streaming media platforms need to expose, creating serious security vulnerabilities. It's like opening numerous doors for hacker attacks, making the system constantly vulnerable to intrusion.

[0005] To enhance network security and reduce the risks associated with exposed ports, the industry has been exploring the introduction of stateless video receiving and forwarding services. The design philosophy of this service is that for the same protocol service, only one port needs to be exposed. Devices can freely connect to any video receiving and forwarding service on the platform and send audio and video data through that service. The video receiving and forwarding service then forwards the data to a specific streaming media source according to its settings, thus realizing the platform's forwarding function. It also has the potential to deploy multiple instances in multi-domain environments and achieve load balancing.

[0006] In existing technical solutions, such as Figure 1 As shown, the streaming media service supports multi-instance deployment, and the streaming media can be independent of each other. Configure the local area network and public network information according to the deployment of the streaming media (applicable to single-domain and multi-domain deployment) so that it can be used as device connection address information later.

[0007] However, despite these improvements, the existing technology still has many obvious shortcomings.

[0008] 1. Regarding the reliability of streaming services, each stream sent by the device is targeted at a specific streaming media. If this streaming media becomes abnormal, the device's reconnection attempts often fail. This results in a temporary disruption to the streaming service, requiring the client to re-initiate the request and manually select another available streaming media to restore the service. During this period, the streaming service cannot function normally, severely interfering with the continuity of video surveillance.

[0009] 2. Regarding the issue of port exposure, due to the one-to-one nature of the connection between streaming media and devices, when multiple instances of streaming media are deployed as business needs grow, each instance needs to expose a unique receiving address port, which causes the number of exposed ports to increase exponentially with the number of instances, greatly increasing the burden on network security.

[0010] 3. Regarding load balancing, streaming media systems need to receive and forward data from multiple protocols. However, existing load balancing methods cover all streaming services and cannot accurately target the critical stage of receiving incoming data for effective load balancing. This results in the inefficient allocation of system resources, much like a traffic jam where all vehicles share the road without dedicated lanes for specific types of vehicles, leading to overall low operating efficiency.

[0011] 4. In multi-domain deployment scenarios, existing technologies require meticulous configuration of streaming media domain parameter environment variables based on complex domain distribution. This process is like finding an exit in a complex maze, full of challenges and uncertainties, greatly increasing the difficulty, complexity and cost of deployment, and severely limiting the scalability and flexibility of the system.

[0012] Therefore, in view of the above-mentioned shortcomings, there is an urgent need in this field for a more innovative and effective technical solution to overcome the limitations of existing technologies, achieve high efficiency, security and stability of streaming media processing in video surveillance platforms, and meet the growing security needs. Summary of the Invention

[0013] Given the shortcomings of existing technologies in areas such as streaming media and device connectivity, port management, load balancing, and multi-domain deployment, this invention aims to provide a streaming media optimization processing method for video surveillance platforms based on video conversion services. This method enables flexible connection between devices and streaming media to improve streaming reliability, achieves stream reception port reuse to reduce security risks, leverages video conversion services to optimize resource utilization through load balancing, and simplifies processes and improves adaptability in multi-domain deployments, ultimately enhancing the overall performance of the video surveillance system. It is widely applicable to various related application scenarios.

[0014] To achieve the above objectives, the technical solution adopted by this invention is: a streaming media optimization processing method for a video surveillance platform based on video conversion services, comprising the following steps: S1: The client initiates a streaming request, which is sent to the streaming media module. After the streaming media module creates a channel, it transmits the streaming media information to the device access service. S2: The device access service calls the SDK interface according to the relevant protocol to send the address and port information of the platform receiving the stream to the device; S3: After receiving the information, the device connects to the platform. The platform selects the optimal video conversion service according to the preset strategy. After the connection is normal, the device sends audio and video data to the selected video conversion service. S4: The video conversion service receives the device's connection and audio / video data, and forwards them to the streaming media; S5: The client connects to streaming media and receives audio and video data.

[0015] Furthermore, in step S3, the video conversion service, as a newly added proxy service, acts as a relay for audio and video data between the device and the streaming media. It adopts a K8s containerization and service port configuration deployment method, and exposes the same port to the outside world. The address information sent to the device by the device access service call interface is consistent. The device uses the address information to connect to the platform service. When the video conversion service is abnormal and cannot be used, K8s will forward the device connection to the normal video conversion service to ensure the normal operation of the business. In terms of load balancing, nginx or K8s load balancing strategy is used to forward the connection to the video conversion service, and the deployment of receiving and forwarding achieves load balancing.

[0016] Furthermore, step S3 includes a single-domain deployment implementation: In scenarios where the system has only one domain address, the platform deploys a video conversion service and configures its address and port information. After the video conversion service is deployed, its domain information is uploaded to Redis for backup or deployed to the device access service environment variables. During subsequent streaming, the device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address and port and sends audio and video data. The video conversion service can be deployed in multiple instances to achieve high availability. The port exposed by the video conversion service is always consistent. The connection is forwarded to the video conversion service using nginx or K8s load balancing strategies to ensure dynamic balance of service business.

[0017] Furthermore, step S3 includes a multi-domain deployment implementation: In scenarios where multiple domains are mixed in the system, the domain environment of the video conversion service is deployed according to the actual resource distribution. After the video conversion service is deployed, its domain information is uploaded to Redis for backup. When retrieving streams later, the device access service obtains the domain load and same-domain load data from Redis, selects the domain data according to the policy, and refreshes the domain load information synchronously to achieve load balancing across multiple domains. The device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address port and sends audio and video data. The receiving connection business of the same domain is executed in accordance with the single-domain scheme.

[0018] Furthermore, the deployment order and process of the multi-instance service are as follows: (1) Deployment of multi-instance video conversion service on the platform: The multi-instance deployment operation of video conversion service is carried out on the platform. This operation is the basis for realizing efficient and flexible data processing of the system. By deploying multiple instances, it is possible to effectively cope with business needs of different scales and possible service anomalies, and improve the overall availability and stability of the system. (2) Video conversion service interacts with Redis: The video conversion service updates its domain and load information to Redis in real time or periodically, and the amount of data matches the number of domains. This step enables subsequent modules to obtain accurate and real-time updated data from Redis when they need relevant information, providing key basis for the system's load balancing, domain selection and other operations. (3) Client initiates stream retrieval request: The client initiates a stream retrieval request, which marks the beginning of the entire business process. Its purpose is to obtain the video surveillance data that the user wants to view. For example, the request is triggered when the user operates the monitoring terminal to view the real-time monitoring screen of a certain area. (4) Request flow to user permission management service: The client's request to retrieve the video stream is first sent to the user permission management service. The user permission management service is mainly responsible for permission judgment, that is, checking whether the client that initiated the request has the permission to obtain the corresponding video stream. Only when the permission judgment is successful will the request be allowed to continue to flow down, so as to ensure the security and access legality of video surveillance data. (5) User permission management forwards the request to the streaming media scheduling service: After the permission judgment is passed, the user permission management service forwards the streaming request to the streaming media scheduling service. The streaming media scheduling service includes the steps of selecting streaming media and creating channels, as well as assisting in the selection of streaming media. They cooperate with each other to complete the task of selecting the appropriate streaming media to ensure that the audio and video data transmitted from the device can be received and processed smoothly in the future. (6) Interaction between streaming media scheduling service and device access service: After successfully selecting streaming media and creating a channel, the streaming media scheduling service will send a request to the device access service. The device access service will then determine whether the device meets the receiving conditions. Devices that meet the conditions will enter the subsequent data transmission process. (7) Device access service interacts with Redis to obtain information: In order to determine the optimal network domain for device connection, the device access service obtains the receiving network domain information from Redis. Based on this information, the device access service can select the optimal network domain according to factors such as load information, so that the device can connect to the most suitable platform part, ensuring the efficient operation and load balancing of the overall business. (8) Device access service sends domain information to device: After determining the optimal domain, the device access service calls the SDK stream retrieval interface to send the domain information to the device. After success, the stream identifier and streaming media information are cached in Redis. After receiving this information, the device knows how to connect to the platform to perform subsequent data sending operations. This step ensures that the connection between the device and the platform is established on the basis of accurate and effective information. (9) Device connection platform and video conversion service: The device connects to the platform based on the received domain address information. After the platform receives the device connection request, it uses nginx or K8s load balancing strategy to transfer the connection to the video conversion service. The load balancing strategy here can ensure that the device connects to a suitable video conversion service instance, so as to realize efficient and stable connection and streaming services between the device and the video conversion service. (10) Video conversion service performs receiving business processing and forwards data: After the device successfully connects to the video conversion service, the video conversion service starts the receiving business processing process, receives the audio and video data sent by the device, and then obtains the streaming media address information according to the stream identifier information, and forwards the data to the streaming media, thereby achieving the transmission of audio and video data from the front-end device to the streaming media inside the platform, ensuring that the monitoring data can smoothly enter the platform for subsequent processing; (11) Client receives audio and video data: The client connects to the streaming media and receives the audio and video data sent by the streaming media. This completes the entire process from the client initiating the stream request to finally receiving the required video surveillance data, realizing the normal stream retrieval service of the video surveillance platform.

[0019] Furthermore, the operations of the device access service to obtain information from Redis include: obtaining domain information when deployed in a single domain, obtaining domain information and load data of the same domain when deployed in multiple domains, and, when deployed in multiple domains, the domain load information in Redis will be refreshed synchronously after the device access service successfully processes the data. Load balancing of the multi-domain deployment scheme is achieved based on the Redis cached data. Specifically, when the device access service selects the domain address information to send to the device, it obtains the domain data of the video conversion service from Redis, selects the domain address with the least load, refreshes the domain load information after the device access service successfully processes the data, and the device connects to the platform according to the domain information. The platform uses nginx and Kubernetes strategies to forward the connection to the video conversion service.

[0020] Furthermore, a ZK-based multi-active mechanism within the group is adopted. Under this mechanism, when some video conversion services malfunction, the device can automatically retry connecting to other normal video conversion services to process the business, thereby ensuring that the business will not be affected as long as not all video conversion services within the group are offline at the same time. The device's connection to the platform's video conversion services is non-directional, and it can connect to any video conversion service that is alive.

[0021] Furthermore, in a single-domain deployment scheme, the platform uses nginx or Kubernetes load balancing strategies to transfer device connections to the video conversion service, thereby achieving a stable connection and efficient transmission and reception of streams between the devices and the video conversion service, and ultimately achieving load balancing in the single-domain deployment scheme.

[0022] Furthermore, a bidirectional connection for data forwarding is achieved using a video conversion service. This video conversion service is stateless, both in its connection to the front-end device and its connection to the platform. The domain address obtained by the device corresponds to a node rather than a pod or service, so that when the device connects to the platform's video conversion service, it can connect to any of the active video conversion services without any connection targeting.

[0023] The present invention, by adopting the above technical solution, has at least the following beneficial effects: 1. Port Management Optimization: In single-domain deployment scenarios, the multi-instance deployment mode of the video conversion service demonstrates significant advantages. By configuring domain information in the device access service and passing it to the device when calling the interface, the device can smoothly forward the data to the video conversion service after connecting to the platform to complete the business process. In this process, multi-instance deployment does not require repeated port exposure, greatly reducing the number of ports exposed to the outside world, reducing network security risks caused by port exposure, and enhancing the overall security and stability of the system.

[0024] 2. Superior Load Balancing Performance: The video conversion service excels in load balancing for streaming services. In complex environments with multiple domains and instances, it successfully achieves load balancing across domains using Redis data sharing mechanisms, ensuring reasonable and efficient resource allocation across different domains. Furthermore, in scenarios with multiple instances connected within the same domain, nginx or Kubernetes can accurately achieve load balancing based on key factors such as the number of connections, effectively avoiding performance bottlenecks caused by excessive local load. This significantly improves the overall system performance and resource utilization, enabling the video surveillance platform to operate stably even during large-scale data transmission.

[0025] 3. Flexible Automatic Scaling Capability: Based on an innovative stateless video conversion service multi-instance architecture, the system possesses powerful adaptive capabilities. When business volume shows an increasing trend, this architecture can automatically achieve horizontal scaling according to preset conditions, and no new ports need to be exposed during the entire scaling process. This feature effectively overcomes the port exposure problem caused by manually starting streaming media and configuring port mapping in traditional methods. It not only simplifies the system management process and reduces operation and maintenance costs, but also ensures that the system can maintain a highly efficient and stable operating state when business volume fluctuates, easily coping with video surveillance business needs of various scales.

[0026] 4. High Availability and Reliability Guaranteed: Thanks to the multi-instance, multi-active operating mode within the group, system reliability is greatly improved. Even in extreme cases where individual services crash or go offline, the entire business process can continue to run stably without any interruption or anomalies because other normal services can quickly take over the business. This mechanism effectively solves the problem of interruption of streaming services due to abnormal streaming services, providing a solid guarantee of high availability and high reliability for the video surveillance platform, ensuring the continuity and stability of security monitoring work, and meeting the security industry's extremely high requirements for the stability of video surveillance systems. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of existing technology interactions.

[0029] Figure 2 This is a flowchart of the video surveillance platform streaming media optimization processing method of the present invention.

[0030] Figure 3This is a schematic diagram of the streaming media optimization processing system of the video surveillance platform of the present invention.

[0031] Figure 4 This is a schematic diagram of the overall framework interaction for the streaming media optimization processing of the video surveillance platform of this invention. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0033] like Figure 1 As shown, the existing streaming media service supports multi-instance deployment. Streaming media can be independent of each other. Configure the local area network and public network information according to the deployment of the streaming media (applicable to single-domain and multi-domain deployment) so that it can be used as device connection address information later.

[0034] Business steps: 1. The client requests a stream from the platform. The streaming media scheduling service selects the streaming media as the receiver, creates a channel, and then sends the stream request to the device access service along with the address information of the streaming media.

[0035] 2. The device access service calls the interface to send streaming media information to the device, and returns responses sequentially. The address here is unique, for use by subsequent device connections and streaming.

[0036] 3. The device connects to the platform's streaming media based on the received address, and broadcasts the stream upon successful connection. Connection failure is handled according to the device's processing mechanism (retry or reconnection upon failure).

[0037] 4. The client connects to the streaming media and obtains the stream sent by the device.

[0038] Following the steps above, the connection between the streaming media and the device is stateful; the device connection will only be established one-to-one with the streaming media created.

[0039] For example, the platform has streaming media 1 (IP1, Port1) and streaming media 2 (IP2, Port2). After the streaming media scheduling service selects streaming media 1 to create a channel, it includes IP1 and Port1 in the protocol and sends them to the device access service. The device access service then calls the interface to the front-end device. After that, the device will only use IP1 and Port1 to connect to the streaming media and send streams. If the service encounters an error or the address becomes unavailable, the device will be unable to connect and send data, and the streaming service will fail.

[0040] It has at least the following shortcomings: 1. Reliability of streaming services: Each time the device sends a stream, it targets a specific streaming media. If this streaming media becomes abnormal, the device will fail to retry the connection, and the streaming service will be temporarily affected. The client needs to intervene and re-initiate the request to select another streaming media to restore the service, during which time the streaming will fail. 2. Port exposure: The connection between the streaming media and the device is one-to-one, and the streaming capacity of a single streaming media is limited, requiring the deployment of multiple instances of the streaming media. The deployment of multiple instances of the streaming media requires that the exposed receiving address port of the streaming media cannot be repeated, and the exposed port should be multiplied according to the number of instances. 3. Load statistics: The streaming media has the function of receiving and forwarding streams, involving the reception and forwarding of various protocol data (the streaming media acts as an intermediary service, receiving external data of various protocols, and also forwarding different protocol streams to meet user needs). At this time, load statistics involve all streaming services, and it is impossible to achieve load balancing only for receiving external data. 4. Multi-domain deployment: The streaming media is deployed according to the distribution of the streaming media by domain, and the domain parameter environment variables of the streaming media are configured. This increases the deployment complexity.

[0041] Example 1

[0042] Please see Figure 2 This embodiment provides a streaming media optimization processing method for a video surveillance platform based on video conversion services, including the following steps: S1: The client initiates a streaming request, which is sent to the streaming media module. After the streaming media module creates a channel, it transmits the streaming media information to the device access service. S2: The device access service calls the SDK interface according to the relevant protocol to send the address and port information of the platform receiving the stream to the device; S3: After receiving the information, the device connects to the platform. The platform selects the optimal video conversion service according to the preset strategy. After the connection is normal, the device sends audio and video data to the selected video conversion service. S4: The video conversion service receives the device's connection and audio and video data, and forwards them to the streaming media. S5: The client connects to streaming media and receives audio and video data.

[0043] In one implementation method, in step S3 of this embodiment, the video conversion service is added as a proxy service, which acts as an audio and video data relay between the device and the streaming media. It adopts a K8s containerization and service port configuration deployment method, and exposes the same port to the outside world. The address information sent to the device by the device access service call interface is consistent. The device uses the address information to connect to the platform service. When the video conversion service is abnormal and cannot be used, K8s will transfer the device connection to the normal video conversion service to ensure the normal operation of the business. In terms of load balancing, nginx or K8s load balancing strategy is used to forward the connection to the video conversion service, and the deployment of receiving and forwarding achieves load balancing.

[0044] As one implementation method, this embodiment includes a single-domain deployment method in step S3: In a scenario where the system has only one domain address, the platform deploys a video conversion service and configures the address and port information. After the video conversion service is deployed, its domain information is uploaded to Redis for backup or deployed to the device access service environment variables. When retrieving streams, the device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address and port and sends audio and video data. The video conversion service can be deployed in multiple instances to achieve high availability. The port exposed by the video conversion service is always consistent. The connection is forwarded to the video conversion service with the help of nginx or K8s load balancing strategies to ensure dynamic balance of service business.

[0045] As one implementation method, step S3 of this embodiment includes a multi-domain deployment implementation: In scenarios where multiple domains are mixed in the system, the domain environment of the video conversion service is deployed according to the actual resource distribution. After the video conversion service is deployed, its domain information is uploaded to Redis for backup. When retrieving streams later, the device access service obtains the domain load and same domain load data on Redis, selects the domain data according to the policy, and refreshes the domain load information synchronously to achieve load balancing of multiple domains. The device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service according to the received address port and sends audio and video data. The receiving connection business of the same domain is executed in accordance with the single domain scheme.

[0046] As one implementation method, the deployment order and process of the multi-instance service in this embodiment are as follows: (1) Deployment of multiple instances of video conversion service on the platform: The multi-instance deployment operation of video conversion service is carried out on the platform. This operation is the basis for realizing efficient and flexible data processing of the system. By deploying multiple instances, it is possible to effectively cope with business needs of different scales and possible service anomalies, and improve the overall availability and stability of the system. (2) Video conversion service interacts with Redis: The video conversion service updates its domain and load information to Redis in real time or periodically, and the amount of data matches the number of domains. This step enables subsequent modules to obtain accurate and real-time updated data from Redis when they need relevant information, providing key basis for the system's load balancing, domain selection and other operations. (3) Client initiates stream retrieval request: The client initiates a stream retrieval request, which marks the beginning of the entire business process. Its purpose is to obtain the video surveillance data that the user wants to view. For example, the request is triggered when the user operates the monitoring terminal to view the real-time monitoring screen of a certain area. (4) Request flow to user permission management service: The client's request to retrieve the video stream is first sent to the user permission management service. The user permission management service is mainly responsible for permission judgment, that is, checking whether the client that initiated the request has the permission to obtain the corresponding video stream. Only when the permission judgment is successful will the request be allowed to continue to flow down, so as to ensure the security and access legality of video surveillance data. (5) User permission management forwards the request to the streaming media scheduling service: After the permission judgment is passed, the user permission management service forwards the streaming request to the streaming media scheduling service. The streaming media scheduling service includes the steps of selecting streaming media and creating channels, as well as assisting in the selection of streaming media. They cooperate with each other to complete the task of selecting the appropriate streaming media to ensure that the audio and video data transmitted from the device can be received and processed smoothly in the future. (6) Interaction between streaming media scheduling service and device access service: After successfully selecting streaming media and creating a channel, the streaming media scheduling service will send a request to the device access service. The device access service will then determine whether the device meets the receiving conditions. Devices that meet the conditions will enter the subsequent data transmission process. (7) Device access service interacts with Redis to obtain information: In order to determine the optimal network domain for device connection, the device access service obtains the receiving network domain information from Redis. Based on this information, the device access service can select the optimal network domain according to factors such as load information, so that the device can connect to the most suitable platform part, ensuring the efficient operation and load balancing of the overall business. (8) Device access service sends domain information to device: After determining the optimal domain, the device access service calls the SDK stream retrieval interface to send the domain information to the device. After success, the stream identifier and streaming media information are cached in Redis. After receiving this information, the device knows how to connect to the platform to perform subsequent data sending operations. This step ensures that the connection between the device and the platform is established on the basis of accurate and effective information. (9) Device connection platform and video conversion service: The device connects to the platform based on the received domain address information. After the platform receives the device connection request, it uses nginx or K8s load balancing strategy to transfer the connection to the video conversion service. The load balancing strategy here can ensure that the device connects to a suitable video conversion service instance, so as to realize efficient and stable connection and streaming services between the device and the video conversion service. (10) Video conversion service performs receiving business processing and forwards data: After the device successfully connects to the video conversion service, the video conversion service starts the receiving business processing process, receives the audio and video data sent by the device, and then obtains the streaming media address information according to the stream identifier information, and forwards the data to the streaming media, thereby achieving the transmission of audio and video data from the front-end device to the streaming media inside the platform, ensuring that the monitoring data can smoothly enter the platform for subsequent processing; (11) Client receives audio and video data: The client connects to the streaming media and receives the audio and video data sent by the streaming media. This completes the entire process from the client initiating the stream request to finally receiving the required video surveillance data, realizing the normal stream retrieval service of the video surveillance platform.

[0047] As one implementation method, the operation of the device access service obtaining information from Redis in this embodiment includes: obtaining domain information when deployed in a single domain, obtaining domain information and load data of the same domain when deployed in multiple domains, and when deployed in multiple domains, the domain load information in Redis will be refreshed synchronously after the device access service successfully processes the data. Load balancing of the multi-domain deployment scheme is achieved based on the Redis cached data. Specifically, when the device access service selects domain address information to send to the device, it obtains the domain data of the video conversion service from Redis, selects the domain address with the least load, refreshes the domain load information after the device access service successfully processes the data, and the device connects to the platform according to the domain information. The platform uses nginx and Kubernetes strategies to forward the data to the video conversion service.

[0048] As one implementation method, this embodiment adopts a group-based multi-active mechanism based on ZK. Under this mechanism, when some video conversion services malfunction, the device can automatically retry connecting to other normal video conversion services to process the business, thereby ensuring that the business will not be affected as long as not all video conversion services in the group are offline at the same time. The device is non-directional when connecting to the platform's video conversion services and can connect to any video conversion service that is alive.

[0049] As one implementation method, in this embodiment, in a single-domain deployment scheme, the platform uses nginx or Kubernetes load balancing strategies to transfer device connections to the video conversion service, thereby achieving a stable connection and efficient sending and receiving of streaming services between the device and the video conversion service, and thus achieving load balancing in the single-domain deployment scheme.

[0050] As one implementation method, this embodiment utilizes a video conversion service to achieve bidirectional connection and data forwarding. The video conversion service is in a stateless state when connected to the front-end device and when connected to the platform. The domain address obtained by the device corresponds to a node rather than a pod or service, so that when the device connects to the platform's video conversion service, it can connect to any video conversion service that is alive and there is no connection directionality.

[0051] Example 2

[0052] like Figure 3 and Figure 4As shown, this embodiment provides a streaming media optimization processing method for a video surveillance platform based on a video conversion service. The core idea of ​​this solution is to introduce a proxy service (video conversion service) as a relay station. One end connects to the front-end device to receive audio and video data; the other end connects to the streaming media to send audio and video data to the streaming media. The connection of the front-end device is non-directional and stateless, thereby realizing the system's streaming media service.

[0053] like Figure 3 As shown, the process is explained below: The client initiates a streaming request, sends a request to the streaming media module, and after creating a channel, sends the streaming media information to the device access service.

[0054] The device access service calls the SDK interface according to the protocol (currently supported signaling offloading protocols, such as Uniview active registration, national standard, etc.) to send the address and port information received by the platform to the device.

[0055] After receiving the information, the device connects to the platform. The platform selects the optimal video conversion service for transfer according to its strategy. Once the connection is established, the device begins sending audio and video data.

[0056] The video conversion service receives the device's connection, receives audio and video data, and forwards it to the streaming media.

[0057] The client connects to the streaming media and receives audio and video data.

[0058] Video conversion service business description: The video conversion service, as a newly added proxy service, serves as an intermediary between devices and streaming media. It adopts a Kubernetes service deployment approach, exposing the same port uniformly. The address information (address and port) sent to the device by the service access interface remains consistent. Devices use this address information to connect to the platform service. If the video conversion service malfunctions and becomes unavailable, Kubernetes automatically redirects the connection to the working video conversion service, ensuring business continuity. For load balancing, nginx or Kubernetes load balancing strategies are used to forward traffic to the video conversion service. Deploying forwarding achieves load balancing. For example, the domain name's DNS resolves to the entry point LVS, LVS forwards traffic to nginx via the port (with iphash added for load balancing), and then proxies to the social standard gateway video forwarding service.

[0059] Single domain deployment solution: In scenarios where the system has only one domain address, the platform deploys a video conversion service and configures its address and port information. After deploying multiple instances of the video conversion service, its domain information is uploaded to Redis for backup (or it can be deployed to the device access service environment variables). When retrieving streams, the device access service calls an interface to send the address information to the device to complete the business logic. The device connects to the video conversion service based on the received address and port information and sends audio and video data. The video conversion service supports multi-instance deployment to achieve high availability. The ports exposed by the video conversion service remain consistent, and nginx or Kubernetes load balancing strategies are used to forward connections to the video conversion service, ensuring dynamic load balancing for the service.

[0060] Multi-domain deployment solution: Similar to the single-domain solution, when multiple domains are mixed in the system, the domain environment for the video conversion service needs to be deployed according to the actual resource distribution. After deployment, the video conversion service uploads its domain information to Redis for backup. During stream retrieval, the device access service obtains domain load and same-domain load data from Redis, selects domain data (address information) according to the strategy, and synchronously refreshes the load information to achieve load balancing across multiple domains. The device access service calls the interface to send the address information to the device to complete the business. The device connects to the video conversion service based on the received address and port information and sends audio and video data. Stream reception connections within the same domain follow the same procedures as the single-domain solution.

[0061] like Figure 4 The following is an explanation of the component names: Client: The source that initiates the stream retrieval request.

[0062] UAS (User Access Control): Responsible for verifying permissions for client stream requests.

[0063] STDU_S (Streaming Media Scheduling Service): Selects streaming media and creates channels based on policies.

[0064] STDU (Streaming Media Service): Data reception and forwarding service.

[0065] Device access service: Interacts with front-end devices to provide services such as receiving device registration and stream retrieval.

[0066] Device: Front-end device, which enables interaction with the platform and the transmission of audio and video data.

[0067] Video conversion service: Acts as a relay station, receiving audio and video data from devices and forwarding it to streaming media.

[0068] Redis: a middleware used to store information such as domains and load.

[0069] II. Data Interaction Logic Client to UAS; Data interaction: The client initiates a stream retrieval request.

[0070] Objective: To request the video stream.

[0071] UAS to STDU_S; Data interaction: The UAS forwards the stream retrieval request to the STDU_S.

[0072] Objective: To select streaming media.

[0073] STDU_S to STDU; Data interaction: STDU_S interacts with STDU to select streaming media and create a receiving stream channel.

[0074] Objective: To create a channel.

[0075] STDU_S is connected to the device; Data interaction: STDU_S will send a request to the device access service, carrying streaming media-related information.

[0076] Objective: To provide streaming media information to device access services.

[0077] The device is connected to Redis; Data interaction: The device access service obtains incoming domain information (including address information and load) from Redis.

[0078] Objective: To obtain load information to select the optimal domain. For example, to select the video forwarding service domain with the lowest load.

[0079] Device to device; Data interaction: The device access service calls the SDK stream retrieval interface to send the domain information to the device.

[0080] Objective: To provide platform information so that devices can connect and send audio and video streams.

[0081] Device-to-video conversion service; Data interaction: The device connects to the video conversion service based on the received domain address information and sends audio and video data.

[0082] Objective: To transmit audio and video data to the platform.

[0083] Video conversion service to streaming media; Data interaction: The video conversion service forwards audio and video data to streaming media.

[0084] Objective: To deliver data to streaming media.

[0085] Streaming to the client; Data interaction: The client connects to the streaming media and receives audio and video data sent by the streaming media.

[0086] Objective: To ultimately provide the video stream to the client.

[0087] The entire process, through close collaboration between various modules, enables the complete flow from client-side streaming requests to the final receipt of audio and video data. Redis serves to store and provide load information to aid decision-making, while the video conversion service acts as a crucial intermediary, ensuring stable data transmission.

[0088] Service deployment example: The platform's video conversion service is deployed across multiple instances. The video conversion service refreshes its domain and load information to Redis. The amount of data is consistent with the number of domains (domain data depends on the node or platform's domain data and is unrelated to the video conversion service). After the video conversion service goes live, it periodically refreshes load and domain information to Redis.

[0089] 1. The client sends a streaming request to the User Access Management (UAS) service. After successful permission verification, the request is forwarded to the streaming media scheduling service.

[0090] 2. The streaming media scheduling service selects the streaming media and creates a channel. After successful selection, a request is sent to the device access service.

[0091] 3. The device access service determines that the device meets the receiving conditions, retrieves the receiving network domain information (address information and load) from Redis, selects the optimal network domain based on the load information, and calls the SDK's receiving interface to send the network domain information to the device. Responses are returned sequentially.

[0092] 4. The device connects to the platform based on the received domain address information. After the platform receives the connection, it uses nginx or Kubernetes load balancing strategies to transfer the connection to the video conversion service. After the connection is established, the video conversion service performs the receiving business processing.

[0093] 5. After the video conversion service receives the data, it forwards the data to the streaming media, and the original process remains unchanged.

[0094] 6. The client connects to the streaming media and receives audio and video data sent by the streaming media.

[0095] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A streaming media optimization processing method for a video surveillance platform based on video conversion services, characterized in that: Includes the following steps: S1: The client initiates a streaming request, which is sent to the streaming media module. After the streaming media module creates a channel, it transmits the streaming media information to the device access service. S2: The device access service calls the SDK interface according to the relevant protocol to send the address and port information of the platform receiving the stream to the device; S3: After receiving the information, the device connects to the platform. The platform selects the optimal video conversion service according to the preset strategy. After the connection is normal, the device sends audio and video data to the selected video conversion service. S4: The video conversion service receives the device's connection and audio / video data, and forwards them to the streaming media; S5: The client connects to streaming media and receives audio and video data; In step S3, the video conversion service, as a newly added proxy service, acts as an audio and video data relay between the device and the streaming media. It adopts a K8s containerization and service port configuration deployment method, and exposes the same port to the outside world. The address information sent to the device by the device access service call interface is consistent. The device uses the address information to connect to the platform service. When the video conversion service is abnormal and cannot be used, K8s will forward the device connection to the normal video conversion service to ensure the normal operation of the business. In terms of load balancing, nginx or K8s load balancing strategy is used to forward the connection to the video conversion service. Deployment of receiving and forwarding achieves load balancing. Step S3 includes a single-domain deployment implementation: In scenarios where the system has only one domain address, the platform deploys a video conversion service and configures its address and port information. After the video conversion service is deployed, its domain information is uploaded to Redis for backup or deployed to the device access service environment variables. During subsequent streaming, the device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address and port and sends audio and video data. The video conversion service can be deployed in multiple instances, and the port exposed by the video conversion service is always consistent. The connection is forwarded to the video conversion service using nginx or K8s load balancing strategies to ensure dynamic balance of service business. In scenarios where multiple network domains are mixed in the system, the network environment of the video conversion service is deployed according to the actual resource distribution. After the video conversion service is deployed, its network information is uploaded to Redis for backup. When retrieving streams later, the device access service obtains the network load and the same network load data on Redis, selects the network data according to the policy, and refreshes the network load information synchronously to achieve load balancing of multiple network domains. The device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service according to the received address port and sends audio and video data. The receiving connection business of the same network domain is executed in accordance with the single network domain scheme. The device adopts a ZK-based multi-active mechanism within the group. Under this mechanism, when some video conversion services malfunction, the device can automatically retry connecting to other normal video conversion services to process the business. This ensures that as long as not all video conversion services in the group are offline at the same time, the business will not be affected. The device is non-directional when connecting to the platform's video conversion services and can connect to any video conversion service that is alive. The deployment order and process for multi-instance services are as follows: (1) Multi-instance deployment of video conversion service on the platform: Conduct multi-instance deployment of video conversion service on the platform; (2) Video conversion service interacts with Redis: The video conversion service updates its domain and load information to Redis in real time or periodically, and the amount of data matches the number of domains. This step enables subsequent modules to obtain accurate and real-time updated data from Redis when they need relevant information, providing key basis for the system's load balancing and domain selection operations. (3) Client initiates stream retrieval request: The purpose of the client initiating stream retrieval request is to obtain the video surveillance data that the user wants to view, including the real-time monitoring screen of the area being viewed by the user on the monitoring terminal. (4) The request is forwarded to the user permission management service: The client's request to retrieve the video stream is first sent to the user permission management service. The user permission management service checks whether the client that initiated the request has the permission to obtain the corresponding video stream. Only when the permission check is successful will the request be allowed to continue to flow down. (5) User permission management forwards the request to the streaming media scheduling service: After the permission judgment is passed, the user permission management service forwards the streaming request to the streaming media scheduling service. The streaming media scheduling service includes the steps of selecting streaming media and creating channels, as well as assisting in the selection of streaming media. They cooperate with each other to complete the task of selecting the appropriate streaming media to ensure that the audio and video data transmitted from the device can be received and processed smoothly in the future. (6) Interaction between streaming media scheduling service and device access service: After successfully selecting streaming media and creating a channel, the streaming media scheduling service will send a request to the device access service. The device access service will then determine whether the device meets the receiving conditions. Devices that meet the conditions will enter the subsequent data transmission process. (7) Device access service interacts with Redis to obtain information: It obtains the receiving network domain information from Redis. Based on the obtained receiving network domain information, the device access service selects the optimal network domain according to the load information, so that the device connects to the most suitable platform part. (8) Device access service sends domain information to device: After determining the optimal domain, the device access service calls the SDK stream retrieval interface to send the domain information to the device. After success, the stream identifier and streaming media information are cached in Redis; (9) Device connection platform and video conversion service: The device connects to the platform based on the received domain address information. After the platform receives the device connection request, it uses nginx or K8s load balancing strategy to transfer the connection to the video conversion service to ensure that the device connects to the appropriate video conversion service instance. (10) Video conversion service performs receiving business processing and forwards data: After the device successfully connects to the video conversion service, the video conversion service starts the receiving business processing process, receives the audio and video data sent by the device, and then obtains the streaming media address information according to the stream identifier information, and forwards the audio and video data to the streaming media, thereby achieving the transmission of audio and video data from the front-end device to the streaming media inside the platform, ensuring that the monitoring data can smoothly enter the platform for subsequent processing; (11) Client receives audio and video data: The client connects to the streaming media and receives the audio and video data sent by the streaming media. This completes the entire process from the client initiating the stream request to finally receiving the required video surveillance data, realizing the normal stream retrieval service of the video surveillance platform.

2. The method according to claim 1, characterized in that: The operations of the device access service to retrieve information from Redis include: retrieving domain information when deployed in a single domain, and retrieving domain information and domain load data when deployed in multiple domains. In multi-domain deployment, after successful processing by the device access service, the domain load information in Redis is refreshed synchronously. Load balancing of the multi-domain deployment scheme is achieved based on Redis cached data. Specifically, when the device access service selects domain address information to send to the device, it retrieves the domain data of the video conversion service from Redis, selects the domain address with the least load, refreshes the domain load information after successful processing by the device access service, and the device connects to the platform based on the domain information. The platform uses nginx and Kubernetes strategies to forward the connection to the video conversion service.

3. The method according to claim 2, characterized in that: In a single-domain deployment scheme, the platform uses nginx or Kubernetes load balancing strategies to transfer device connections to the video conversion service, thereby achieving a stable connection and efficient streaming services between the devices and the video conversion service, and ultimately achieving load balancing in the single-domain deployment scheme.

4. The method according to claim 3, characterized in that: The video conversion service is used to achieve bidirectional connection and data forwarding. The connection between the video conversion service and the front-end device and the platform is stateless. The domain address obtained by the device corresponds to a node, so that when the device connects to the video conversion service of the platform, it can connect to any video conversion service that is alive and there is no connection directionality.

Images