Video monitoring platform streaming media optimization processing method based on video conversion service

Abstract

The invention belongs to the technical field of streaming media data processing, and relates to a video conversion service-based streaming media optimization processing method for a video monitoring platform. Comprising the following steps: a client initiates a stream taking request, the request is issued to a streaming media module, and the streaming media module transmits streaming media information to a device access service after creating a channel; the device access service calls an sdk interface according to a related protocol, and sends address port information of platform flow receiving to the device; the equipment is connected with the platform after receiving the information, the platform selects an optimal video conversion service according to a preset strategy, and the equipment sends audio and video data to the selected video conversion service; the video conversion service receives connection and audio and video data of the equipment and forwards the connection and the audio and video data to the streaming media; according to the invention, flexible connection between the device and the streaming media is realized to improve the streaming reliability, multiplexing of the streaming port is achieved to reduce the security risk, load balancing is realized by means of the video conversion service, resource utilization is optimized, the adaptability is improved, and the overall performance of the video monitoring system is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of streaming media data processing technology in a video monitoring platform, and in particular relates to a streaming media optimization processing method for a video monitoring platform based on a video conversion service. Background Art

[0002] With the booming development of the security industry, video surveillance platforms are playing an increasingly important role in maintaining public safety and realizing intelligent management. Such platforms usually have the key capabilities of receiving and forwarding video and audio, and have specific workflows when dealing with common scenarios such as national standard protocols and Uniview active registration signaling and stream separation.

[0003] When the platform needs to obtain audio and video streams from many access devices and transmit them to the streaming module, the optimal streaming media must be carefully selected to create a channel for receiving and forwarding the streams. After receiving information about the streaming media, the device access service will pass it to the device. However, since the device streaming depends on a specific selected streaming media address, the connection mode between the streaming media and the device is limited to a one-to-one stateful connection, and the device cannot freely select other streaming media for streaming operations.

[0004] Based on this connection method, in order to communicate with the device, the streaming media has to expose the port to the outside, and the port used by each streaming media is different. With the continuous expansion of business, the demand for high-definition and large-scale access of video surveillance has prompted the streaming media to expand its capacity to improve processing capabilities, and at the same time, the various protocols accessed by the platform are becoming increasingly rich. This development trend has directly led to a geometric increase in the number of ports that need to be exposed by the streaming media, which has buried serious hidden dangers for network security, just like opening many doors for hacker attacks, making the system face the risk of being invaded at any time.

[0005] In order to enhance network security and reduce the risks brought by port exposure, the industry has tried to introduce stateless video reception and forwarding services. The design concept of this service is that for the same protocol service, only one port needs to be exposed to the outside world. The device can freely connect to any video reception and forwarding service on the platform and send audio and video data through the service. The video reception and forwarding service will then forward the data to a specific streaming media according to the settings, thereby realizing the forwarding function of the platform. At the same time, it has the potential to deploy multiple instances in a multi-domain environment and achieve load balancing.

[0006] In the prior art solutions, Figure 1 As shown, the streaming media service supports multi-instance deployment. The streaming media can be independent of each other. Configure the LAN and public network information according to the streaming media deployment situation (applicable to single-domain and multi-domain deployment) so that it can be used as device connection address information later.

[0007] However, despite these improvements, the prior art still suffers from a number of significant deficiencies.

[0008] 1. In terms of the reliability of the streaming service, the device will send a stream to a specified streaming media each time. Once the streaming media is abnormal, the device will often fail to retry the connection. This causes the streaming service to be temporarily affected, and the client must re-initiate the request and manually select other available streaming media to restore the service. During this period, the streaming service cannot be carried out normally, which seriously interferes with the continuity of video surveillance.

[0009] 2. Regarding the port exposure issue, due to the one-to-one nature of the connection between streaming media and devices, when multiple instances of streaming media are deployed as business demand grows, each instance needs to expose a unique receiving address port, causing the number of exposed ports to increase exponentially with the number of instances, greatly increasing the network security burden.

[0010] 3. Regarding load statistics, streaming media must both receive and forward data from multiple protocols in the system, and the existing load statistics methods cover all streaming services, and cannot accurately achieve effective load balancing for the key link of receiving external data. This makes it impossible to reasonably allocate system resources, just like when all vehicles are mixed in traffic congestion, there are no dedicated lanes for specific types of vehicles, resulting in low overall operating efficiency.

[0011] 4. In multi-domain deployment scenarios, existing technologies need to carefully configure the domain parameter environment variables of streaming media based on the complex domain distribution. This process is like finding an exit in a complex maze, which is full of challenges and uncertainties. It greatly increases the difficulty, complexity and cost of deployment, and severely limits the scalability and flexibility of the system.

[0012] To this end, in view of the above-mentioned deficiencies, the field urgently needs a more innovative and effective technical solution to overcome the limitations of existing technologies, achieve high efficiency, security and stability of streaming media processing on video surveillance platforms, and meet the growing security needs. Summary of the invention

[0013] In view of the shortcomings of the existing technology in terms of streaming media connection with devices, port management, load balancing and multi-domain deployment, the present invention aims to provide a video surveillance platform streaming media optimization processing method based on video conversion service. The method realizes flexible connection between devices and streaming media to improve the reliability of streaming, achieves multiplexing of receiving ports to reduce security risks, realizes load balancing and optimizes resource utilization with the help of video conversion services, simplifies processes and improves adaptability in multi-domain deployment, and ultimately improves the overall performance of the video surveillance system, which is widely applicable to various related application scenarios.

[0014] To achieve the above object, the technical solution adopted by the present invention is: a video monitoring platform streaming media optimization processing method based on video conversion service, comprising the following steps: S1: The client initiates a stream request, which is sent to the streaming module. After the streaming module creates a channel, it passes the streaming information to the device access service. S2: The device access service calls the SDK interface according to the relevant protocol and sends the address and port information of the platform receiving the flow to the device; S3: After receiving the information, the device connects to the platform. The platform selects the best video conversion service according to the preset strategy. After the connection is normal, the device sends audio and video data to the selected video conversion service. S4: The video conversion service receives the device connection and audio and video data and forwards it to the streaming media; S5: The client connects to the streaming media and receives audio and video data.

[0015] Furthermore, in step S3, the video conversion service is used as a new proxy service to transfer audio and video data between the device and the streaming media. It adopts K8s containerization and service port configuration deployment mode, exposes the same port to the outside world, and the address information sent to the device by the device access service call interface remains consistent. The device uses the address information to connect to the platform service. When the video conversion service becomes abnormal and cannot be used, K8s transfers the device connection to the normal video conversion service to ensure normal business operations. In terms of load balancing, nginx or K8s load strategy is used to forward the connection to the video conversion service, and the receiving and forwarding are deployed to achieve load balancing.

[0016] Furthermore, step S3 includes a single domain deployment implementation method: in a scenario where the system has only one domain address, the platform deploys a video conversion service and configures the address port information. After the video conversion service is deployed, its domain information is uploaded to Redis for backup or deployed to the device access service environment variables; when subsequently fetching the stream, the device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address port and sends audio and video data. The video conversion service can be deployed in multiple instances to achieve high availability. The port exposed by the video conversion service is always consistent. With the help of nginx or K8s balancing strategy, the connection is forwarded to the video conversion service to ensure dynamic balance of the service business.

[0017] Furthermore, step S3 includes a multi-domain deployment implementation method: in a scenario where there are multiple mixed domains in the system, the domain environment of the video conversion service is deployed according to the actual resource distribution. After the video conversion service is deployed, its domain information is uploaded to Redis for backup. When subsequently retrieving the stream, the device access service obtains the domain load and the same-domain load data on Redis, selects the domain data according to the strategy, and synchronously refreshes the domain load information to achieve load balancing of multiple domains. The device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service according to the received address port and sends audio and video data. The receiving stream connection service in the same domain is executed in accordance with the single-domain solution.

[0018] Furthermore, the order and process of multi-instance service deployment are as follows: (1) Multi-instance deployment of platform video conversion service: Multi-instance deployment of video conversion service is carried out on the platform. This operation is the basis for realizing efficient and flexible data processing of the system. By deploying multiple instances, it can effectively respond to business needs of different scales and possible service abnormalities, and improve the overall availability and stability of the system; (2) Interaction between the video conversion service and Redis: The video conversion service updates its network domain and load information to Redis in real time or periodically, and the amount of data matches the number of network domains. This step enables subsequent modules to obtain accurate and real-time updated data from Redis when they need relevant information, providing a key basis for system operations such as load balancing and network domain selection; (3) The client initiates a stream acquisition request: The client initiates a stream acquisition request, which marks the beginning of the entire business process. Its purpose is to obtain the video surveillance data that is expected to be viewed. For example, this request is triggered when a user uses a monitoring terminal to view the real-time monitoring screen of a certain area. (4) The request is transferred to the user rights management service: The client's stream acquisition request is first sent to the user rights management service. The user rights management service is mainly responsible for the authority judgment, that is, checking whether the client initiating the request has the authority to obtain the corresponding video stream. Only when the authority judgment is successful, the request is allowed to continue to flow downward, thereby ensuring the security of video surveillance data and the legality of access; (5) User authority management forwards the request to the streaming media scheduling service: When the authority judgment is passed, the user authority management service forwards the streaming request to the streaming media scheduling service. The streaming media scheduling service is responsible for selecting streaming media and creating channels, as well as assisting in the relevant operations of streaming media selection. They work together to complete the task of selecting appropriate streaming media to ensure that the audio and video data transmitted by the device can be successfully received and processed in the future; (6) Interaction between streaming media scheduling service and device access service: After successfully selecting streaming media and creating a channel, the streaming media scheduling service sends a request to the device access service. The device access service then determines whether the device meets the conditions for receiving the stream. The qualified device enters the subsequent data transmission process; (7) The device access service interacts with Redis to obtain information: In order to determine the optimal network domain for the device to connect to, the device access service obtains the incoming traffic domain information from Redis. Based on this information, the device access service can select the optimal network domain according to factors such as load information, so that the device can be connected to the most appropriate part of the platform, ensuring the efficient operation and load balancing of the overall business; (8) The device access service sends network domain information to the device: After the device access service determines the optimal network domain, it calls the SDK stream acquisition interface to send the network domain information to the device. If successful, it caches the stream identifier and streaming media information to Redis. After the device receives this information, it knows how to connect to the platform for subsequent data sending operations. This step ensures that the connection between the device and the platform is based on accurate and effective information. (9) Device connection platform and video conversion service: The device connects to the platform based on the received domain address information. After the platform receives the device connection request, it uses nginx or K8s balancing strategy to transfer the connection to the video conversion service. The balancing strategy here can ensure that the device is connected to the appropriate video conversion service instance, realizing efficient and stable connection and sending and receiving stream services between the device and the video conversion service; (10) The video conversion service performs streaming service processing and forwards data: After the device successfully connects to the video conversion service, the video conversion service starts the streaming service processing flow, receives the audio and video data sent by the device, then obtains the streaming media address information based on the stream identification information, and forwards this data to the streaming media, thereby achieving the transmission of audio and video data from the front-end device to the platform's internal streaming media, ensuring that the monitoring data can smoothly enter the platform for subsequent processing; (11) The client receives audio and video data: The client connects to the streaming media and receives the audio and video data sent by the streaming media. This completes the entire process from the client initiating a streaming request to finally receiving the required video surveillance data, thus realizing the normal streaming service of the video surveillance platform.

[0019] Furthermore, the operation of the device access service obtaining information from Redis includes: obtaining domain information when deployed in a single domain, obtaining domain information and load data in the same domain when deployed in multiple domains, and in multiple domains, the device access service will synchronously refresh the domain load information in Redis after successful processing, and implement load balancing of multi-domain deployment solutions based on Redis cache data. Specifically, when the device access service selects domain address information to send to the device, it obtains the domain data of the video conversion service from Redis and selects the domain address with the least load. After the device access service is successfully processed, the domain load information is refreshed, and the device connects to the platform based on the domain information. The platform uses nginx strategy and K8s strategy to forward to the video conversion service.

[0020] Furthermore, a ZK-based multi-active mechanism within the group is adopted. Under this mechanism, when some video conversion services encounter abnormalities, the device can automatically retry connecting to other normal video conversion services for business processing, thereby ensuring that as long as not all video conversion services in the group are offline at the same time, the business will not be affected; the device is non-directional when connecting to the platform's video conversion service and can connect to any video conversion service on which the node is alive.

[0021] Furthermore, in the single-domain deployment solution, the platform uses nginx or K8s balancing strategy to transfer the device connection to the video conversion service to achieve a stable connection between the device and the video conversion service and efficient sending and receiving stream services, thereby achieving load balancing for the single-domain deployment solution.

[0022] Furthermore, a video conversion service is used to realize two-way connection forwarding data. The connection between the video conversion service and the front-end device and the platform is stateless. The domain address obtained by the device corresponds to a node rather than a pod or service. When the device connects to the platform's video conversion service, it can connect to any video conversion service on the surviving node and there is no connection direction.

[0023] The present invention adopts the above technical solution, which has at least the following beneficial effects: 1. Port management optimization: In the single domain deployment scenario, the multi-instance deployment mode of the video conversion service shows significant advantages. By configuring the domain information in the device access service and passing it to the device when calling the interface, the device can be smoothly forwarded to the video conversion service after connecting to the platform to complete the business process. In this process, multi-instance deployment does not need to repeatedly expose ports, which greatly reduces the number of ports exposed by the system, reduces the network security risks caused by port exposure, and enhances the overall security and stability of the system.

[0024] 2. Excellent load balancing performance: The video conversion service performs well in load balancing of the streaming business. In a complex environment with multiple domains and multiple instances, with the help of the Redis data sharing mechanism, load balancing between domains is successfully achieved, ensuring reasonable and efficient resource allocation between different domains. In the scenario of multiple instance connections in the same domain, nginx or K8s can accurately achieve load balancing based on key factors such as the number of connections, effectively avoiding performance bottlenecks caused by excessive local loads, significantly improving the overall performance and resource utilization of the system, and enabling the video surveillance platform to still operate stably during large-scale data transmission.

[0025] 3. Flexible automatic expansion capability: Based on the innovative stateless video conversion service multi-instance architecture, the system has strong adaptive capabilities. When the business volume shows an upward trend, the architecture can automatically achieve horizontal expansion according to preset conditions, and no new ports need to be exposed during the entire expansion process. This feature effectively overcomes the port exposure problem caused by the traditional manual opening of streaming media and configuring port mapping. It not only simplifies the system management process and reduces operation and maintenance costs, but also ensures that the system can always maintain an efficient and stable operating state when the business volume fluctuates, and can easily cope with video surveillance business needs of various sizes.

[0026] 4. High availability and high reliability guarantee: Thanks to the multi-instance and multi-active operation mode within the group, the reliability of the system has been greatly improved. Even in the extreme case of a crash or offline of an individual service, since other normal services can quickly take over the business, the entire business process can still continue to run stably without any interruption or abnormality. This mechanism effectively solves the problem of interruption of the flow collection business due to abnormal flow collection services, provides a solid guarantee of high availability and high reliability for the video surveillance platform, ensures the continuity and stability of security monitoring work, and meets the needs of the security industry that has extremely high requirements for the stability of video surveillance systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is a schematic diagram of the interaction of the prior art.

[0029] Figure 2 The present invention is a flow chart of the video monitoring platform streaming media optimization processing method.

[0030] Figure 3It is a structural schematic diagram of the streaming media optimization processing system of the video monitoring platform of the present invention.

[0031] Figure 4 It is a schematic diagram of the overall framework interaction of the video monitoring platform streaming media optimization processing of the present invention. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0033] like Figure 1 As shown, the existing streaming media service supports multi-instance deployment. The streaming media can be independent of each other. The LAN and public network information are configured according to the streaming media deployment situation (applicable to single-domain and multi-domain deployment) so as to be used as device connection address information later.

[0034] Business steps: 1 The client requests the platform to obtain the stream, goes to the streaming media scheduling service, the streaming media selects the streaming media as the receiving party, creates a channel, and then sends the streaming request with the streaming media address information to the device access service.

[0035] 2. The device access service calls the interface to send the streaming media information to the device end, and returns the response in sequence. The address here is unique for subsequent device connection and streaming.

[0036] 3. The device connects to the platform streaming media according to the received address, and starts streaming after the connection is successful. If the connection fails, the device handles it according to its processing mechanism (retry or no longer connect if failed).

[0037] 4 The client connects to the streaming media and obtains the stream sent by the streaming device.

[0038] According to the above steps, the connection between the streaming media and the device is stateful, and the device connection will only be one-to-one for the created streaming media.

[0039] For example, the platform has streaming media 1 (IP1, Port1) and streaming media 2 (IP2, Port2). After the streaming media scheduling service selects streaming media 1 to create a channel, it puts IP1 and Port1 in the protocol and carries them to the device access service. The device access service then calls the interface to the front-end device. After that, the device will only use IP1 and Port1 to connect to the streaming media and send the stream. If the service is abnormal and the address is unavailable, the device cannot connect to send data and the streaming service fails.

[0040] It has at least the following deficiencies: 1. Reliability of streaming service: Each time the device sends a stream, it will target the specified streaming media. Once the streaming media is abnormal, the device will fail to retry to connect to the streaming media, and the streaming service will be temporarily affected. The client needs to intervene and re-initiate the request to select other streaming media to resume the service. During this period, the streaming fails. 2. Port exposure: The streaming media and device connection is 1 to 1. The streaming of a single streaming media is limited, and it is necessary to implement multi-instance deployment of streaming media. Multi-instance deployment of streaming media requires that the streaming address port exposed by the streaming media cannot be repeated, and the exposed port is doubled according to the number of instances. 3. Load statistics: Streaming media has the function of receiving and forwarding streams, involving the reception and forwarding of various protocol data (streaming media, as an intermediate service, receives external data of various protocols, and also forwards different protocol streams to meet user needs). At this time, load statistics involve all streaming services, and it is impossible to achieve load balancing only for receiving external data. 4. Multi-domain deployment: According to the distribution of streaming media in the domain deployment, configure the domain parameter environment variables of the streaming media. Increase the complexity of deployment.

[0041] Embodiment 1

[0042] See also Figure 2 This embodiment provides a video monitoring platform streaming media optimization processing method based on video conversion service, including the following steps: S1: The client initiates a stream request, which is sent to the streaming module. After the streaming module creates a channel, it passes the streaming information to the device access service. S2: The device access service calls the SDK interface according to the relevant protocol and sends the address and port information of the platform receiving the flow to the device; S3: After receiving the information, the device connects to the platform. The platform selects the best video conversion service according to the preset strategy. After the connection is normal, the device sends audio and video data to the selected video conversion service. S4: The video conversion service receives the device's connection and audio and video data, and forwards it to the streaming media. S5: The client connects to the streaming media and receives audio and video data.

[0043] As an implementation method, in step S3 of this embodiment, the video conversion service is a newly added proxy service, which plays the role of transferring audio and video data between the device and the streaming media. It adopts K8s containerization and service port configuration deployment method, exposes the same port to the outside world, and the address information sent to the device by the device access service call interface remains consistent. The device uses the address information to connect to the platform service. When the video conversion service becomes abnormal and cannot be used, K8s transfers the device connection to the normal video conversion service to ensure normal business. In terms of load balancing, nginx or K8s load strategy is used to forward the connection to the video conversion service, and the receiving and forwarding are deployed to achieve load balancing.

[0044] As an implementation method, this embodiment includes a single domain deployment implementation method in step S3: in a scenario where the system has only one domain address, the platform deploys a video conversion service and configures the address port information. After the video conversion service is deployed, its domain information will be uploaded to Redis for backup or deployed to the device access service environment variables; when subsequently acquiring the stream, the device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address port and sends audio and video data. The video conversion service can implement multi-instance deployment to achieve high availability. The port exposed by the video conversion service is always consistent. With the help of nginx or K8s balancing strategy, the connection is forwarded to the video conversion service to ensure dynamic balance of the service business.

[0045] As an implementation method, this embodiment includes a multi-domain deployment implementation method in step S3: in a scenario where there are multiple mixed domains in the system, the domain environment of the video conversion service is deployed according to the actual resource distribution. After the video conversion service is deployed, its domain information is uploaded to Redis for backup. When subsequently retrieving the stream, the device access service obtains the domain load and the same domain load data on Redis, selects the domain data according to the strategy, and synchronously refreshes the domain load information to achieve load balancing of multiple domains. The device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service according to the received address port and sends audio and video data. The receiving stream connection service in the same domain is executed in accordance with the single domain solution.

[0046] As an implementation method, the multi-instance service deployment sequence and process of this embodiment are as follows: (1) Multi-instance deployment of platform video conversion service: Multi-instance deployment operation of video conversion service is carried out on the platform. This operation is the basis for realizing efficient and flexible data processing of the system. By deploying multiple instances, it can effectively respond to business needs of different scales and possible service abnormalities, thereby improving the overall availability and stability of the system; (2) Interaction between the video conversion service and Redis: The video conversion service updates its network domain and load information to Redis in real time or periodically, and the amount of data matches the number of network domains. This step enables subsequent modules to obtain accurate and real-time updated data from Redis when they need relevant information, providing a key basis for system operations such as load balancing and network domain selection; (3) The client initiates a stream acquisition request: The client initiates a stream acquisition request, which marks the beginning of the entire business process. Its purpose is to obtain the video surveillance data that is expected to be viewed. For example, this request is triggered when a user uses a monitoring terminal to view the real-time monitoring screen of a certain area. (4) The request is transferred to the user rights management service: The client's stream acquisition request is first sent to the user rights management service. The user rights management service is mainly responsible for the authority judgment, that is, checking whether the client initiating the request has the authority to obtain the corresponding video stream. Only when the authority judgment is successful, the request is allowed to continue to flow downward, thereby ensuring the security of video surveillance data and the legality of access; (5) User authority management forwards the request to the streaming media scheduling service: When the authority judgment is passed, the user authority management service forwards the streaming request to the streaming media scheduling service. The streaming media scheduling service is responsible for selecting streaming media and creating channels, as well as assisting in the relevant operations of streaming media selection. They work together to complete the task of selecting appropriate streaming media to ensure that the audio and video data transmitted by the device can be successfully received and processed in the future; (6) Interaction between streaming media scheduling service and device access service: After successfully selecting streaming media and creating a channel, the streaming media scheduling service sends a request to the device access service. The device access service then determines whether the device meets the conditions for receiving the stream. The qualified device enters the subsequent data transmission process; (7) The device access service interacts with Redis to obtain information: In order to determine the optimal network domain for the device to connect to, the device access service obtains the incoming traffic domain information from Redis. Based on this information, the device access service can select the optimal network domain according to factors such as load information, so that the device can be connected to the most appropriate part of the platform, ensuring the efficient operation and load balancing of the overall business; (8) The device access service sends network domain information to the device: After the device access service determines the optimal network domain, it calls the SDK stream acquisition interface to send the network domain information to the device. If successful, it caches the stream identifier and streaming media information to Redis. After the device receives this information, it knows how to connect to the platform for subsequent data sending operations. This step ensures that the connection between the device and the platform is based on accurate and effective information. (9) Device connection platform and video conversion service: The device connects to the platform based on the received domain address information. After the platform receives the device connection request, it uses nginx or K8s balancing strategy to transfer the connection to the video conversion service. The balancing strategy here can ensure that the device is connected to the appropriate video conversion service instance, realizing efficient and stable connection and sending and receiving stream services between the device and the video conversion service; (10) The video conversion service performs streaming service processing and forwards data: After the device successfully connects to the video conversion service, the video conversion service starts the streaming service processing flow, receives the audio and video data sent by the device, then obtains the streaming media address information based on the stream identification information, and forwards this data to the streaming media, thereby achieving the transmission of audio and video data from the front-end device to the platform's internal streaming media, ensuring that the monitoring data can smoothly enter the platform for subsequent processing; (11) The client receives audio and video data: The client connects to the streaming media and receives the audio and video data sent by the streaming media. This completes the entire process from the client initiating a streaming request to finally receiving the required video surveillance data, thus realizing the normal streaming service of the video surveillance platform.

[0047] As an implementation method, the operation of the device access service in this embodiment obtaining information from Redis includes: obtaining domain information when deployed in a single domain, obtaining domain information and load data in the same domain when deployed in multiple domains, and when deployed in multiple domains, the device access service will synchronously refresh the domain load information in Redis after successful processing, and implement load balancing of multi-domain deployment solutions based on Redis cache data. Specifically, when the device access service selects domain address information to send to the device, the domain data of the video conversion service is obtained from Redis, and the domain address with the least load is selected. After the device access service is successfully processed, the domain load information is refreshed, and the device connects to the platform based on the domain information. The platform uses nginx strategy and K8s strategy to forward to the video conversion service.

[0048] As an implementation method, this embodiment adopts a ZK-based multi-active mechanism within the group. Under this mechanism, when some video conversion services are abnormal, the device can automatically retry connecting to other normal video conversion services for business processing, thereby ensuring that as long as not all video conversion services in the group are offline at the same time, the business will not be affected; when the device connects to the platform's video conversion service, it is non-directional and can connect to any video conversion service on which the node is alive.

[0049] As an implementation method, in this embodiment, in a single domain deployment solution, the platform uses nginx or K8s balancing strategy to transfer the device connection to the video conversion service to achieve a stable connection between the device and the video conversion service and efficient sending and receiving stream services, thereby achieving load balancing for the single domain deployment solution.

[0050] As an implementation method, this embodiment uses a video conversion service to implement bidirectional connection forwarding data. The video conversion service is connected to the front-end device and to the platform in a stateless manner. The domain address obtained by the device corresponds to a node rather than a pod or service, so that when the device connects to the platform's video conversion service, it can connect to any video conversion service on the surviving node and there is no connection direction.

[0051] Embodiment 2

[0052] like Figure 3 and Figure 4As shown, this embodiment provides a method for optimizing streaming media processing of a video surveillance platform based on a video conversion service. The core idea of ​​this solution is to introduce a proxy service (video conversion service) as a transfer station, one end of which is connected to the front-end device to receive audio and video data; the other end is connected to the streaming media to send the audio and video data to the streaming media. The connection of the front-end device is non-directional and stateless, thereby realizing the streaming media service of the system.

[0053] like Figure 3 As shown, the process is described as follows: The client initiates a stream request and sends a request to the streaming media module. After creating a channel, it carries the streaming media information to the device access service.

[0054] The device access service calls the SDK interface according to the protocol (currently supported signaling diversion protocols, such as Uniview active registration, national standard, etc.), and sends the address port information of the platform receiving traffic to the device.

[0055] After receiving the information, the device connects to the platform. The platform selects the best video conversion service for transfer according to the strategy. After the connection is normal, the device starts to send audio and video data.

[0056] The video conversion service receives the connection from the device, receives the audio and video data, and forwards it to the streaming media.

[0057] The client connects to the streaming media and receives audio and video data.

[0058] Video conversion service business description: As a newly added proxy service, the video conversion service is used for transfer between devices and streaming media. The K8s service deployment method is adopted to uniformly expose the same port to the outside. The address information (address and port) sent to the device by the device access service call interface is consistent. The device uses the address information to connect to the platform service. If the video conversion service fails due to an abnormality, K8s will automatically transfer the connection to the normal video conversion service to ensure business continuity. In terms of load balancing, use nginx or K8s load strategy to forward to the video conversion service, and deploy forwarding to achieve load balancing. For example, the domain name DNS is resolved to the entry LVS, and the LVS is forwarded to nginx through the port (iphash is added to achieve load balancing), and then proxied to the social standard gateway video forwarding service.

[0059] Single domain deployment solution: In the scenario where there is only one domain address in the system, the platform deploys the video conversion service and configures the address and port information. After multiple instances of the video conversion service are deployed, their domain information will be uploaded to Redis for backup (it can also be deployed to the device access service environment variables). When taking the stream, the device access service calls the interface to send the address information to the device to complete the service. The device connects to the video conversion service based on the received address and port information and sends audio and video data. The video conversion service supports multi-instance deployment to achieve high availability. The ports exposed by the video conversion service are consistent, and nginx or K8s balancing strategies are used to forward the connection to the video conversion service to ensure dynamic balance of the service business.

[0060] Multi-domain deployment solution: Similar to the single domain solution, when there are multiple domains mixed in the system, it is necessary to deploy the domain environment of the video conversion service according to the actual resource distribution. After the video conversion service is deployed, its domain information will be uploaded to Redis for backup. When fetching the stream, the device access service obtains the domain load and the same domain load data from Redis, selects the domain data (address information) according to the strategy, and refreshes the load information synchronously to achieve load balancing of multiple domains. The device access service calls the interface to send the address information to the device to complete the service. The device connects to the video conversion service based on the received address port information and sends audio and video data. The same domain's incoming stream connection service refers to the single domain solution.

[0061] like Figure 4 As shown, explanation of component names: Client: The source that initiates the stream fetching request.

[0062] UAS (User Authority Management): responsible for verifying the authority of the client's streaming request.

[0063] STDU_S (Streaming Media Scheduling Service): selects streaming media and creates channels based on policies.

[0064] STDU (Streaming Service): Data reception and forwarding service.

[0065] Device access service: interacts with front-end devices to implement services such as receiving device registration and data flow.

[0066] Device: Front-end device that interacts with the platform and sends audio and video data.

[0067] Video conversion service: acts as a transit station, receiving audio and video data from the device and forwarding it to the streaming media.

[0068] Redis: middleware used to store domain, load and other information.

[0069] 2. Data Interaction Logic Client to UAS; Data interaction: The client initiates a stream acquisition request.

[0070] Purpose: Request to obtain the video stream.

[0071] UAS to STDU_S; Data interaction: UAS forwards the stream request to STDU_S.

[0072] Purpose: To make streaming media selection.

[0073] STDU_S to STDU; Data interaction: STDU_S interacts with STDU, and streaming media is selected to create a receiving stream channel.

[0074] Purpose: Create a channel.

[0075] STDU_S to device access; Data interaction: STDU_S sends a request to the device access service, carrying streaming media related information.

[0076] Purpose: To provide streaming media information to device access services.

[0077] The device is connected to Redis; Data interaction: The device access service obtains the incoming traffic domain information (including address information and load) from Redis.

[0078] Purpose: Obtain load information to select the optimal domain. For example, select the domain name of the video forwarding service with the smallest load.

[0079] Device access to device; Data interaction: The device access service calls the SDK stream acquisition interface to send the network domain information to the device.

[0080] Purpose: Provide platform information to allow devices to connect and send audio and video streams.

[0081] Device to video conversion services; Data interaction: The device connects to the video conversion service based on the received domain address information and sends audio and video data.

[0082] Purpose: To transmit audio and video data to the platform.

[0083] Video conversion services to streaming media; Data exchange: The video conversion service forwards audio and video data to streaming media.

[0084] Purpose: To pass data to the streaming media.

[0085] Streaming media to clients; Data interaction: The client connects to the streaming media and receives the audio and video data sent by the streaming media.

[0086] Purpose: Finally provide the video stream to the client.

[0087] The entire process is realized through close collaboration between various modules, from client streaming request to final reception of audio and video data. Among them, Redis plays a role in storing and providing load information to assist decision-making, and the video conversion service plays a key role as a transit station, ensuring stable data transmission.

[0088] Service deployment example: The platform video conversion service is deployed in multiple instances. The video conversion service will refresh its domain and load information to Redis. The number of data is consistent with the number of domains (domain data depends on the node or platform domain data and has nothing to do with the video conversion service). After the video conversion service goes online, the load information and domain information will be refreshed to Redis regularly.

[0089] 1. The client sends a streaming request to the User Authority Management (UAS) service. After the authority verification is successful, the request is forwarded to the streaming media scheduling service.

[0090] 2. The streaming media scheduling service selects the streaming media and creates a channel. If successful, the request is sent to the device access service.

[0091] 3. The device access service determines whether the device meets the conditions for receiving traffic, obtains the receiving domain information (address information and load) from Redis, selects the optimal domain based on the load information, and calls the SDK flow acquisition interface to send the domain information to the device. The responses are returned in sequence.

[0092] 4. The device connects to the platform based on the received domain address information. After receiving the connection, the platform uses nginx or K8s balancing strategy to transfer to the video conversion service. After the connection, the video conversion service performs the streaming service processing.

[0093] 5. After the video conversion service receives the stream, it forwards the data to the streaming media, and the original process remains unchanged.

[0094] 6. The client connects to the streaming media and receives the audio and video data sent by the streaming media.

[0095] Although the embodiments of the present invention have been shown and described above, it will be appreciated that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, substitute and vary the above embodiments within the scope of the present invention.

Claims

1. A video monitoring platform streaming media optimization processing method based on video conversion service, characterized in that: The steps include: S1: The client initiates a stream request, which is sent to the streaming module. After the streaming module creates a channel, it passes the streaming information to the device access service. S2: The device access service calls the SDK interface according to the relevant protocol and sends the address and port information of the platform receiving the flow to the device; S3: After receiving the information, the device connects to the platform. The platform selects the best video conversion service according to the preset strategy. After the connection is normal, the device sends audio and video data to the selected video conversion service. S4: The video conversion service receives the device connection and audio and video data and forwards it to the streaming media; S5: The client connects to the streaming media and receives audio and video data.

2. The method according to claim 1, characterized in that: In step S3, the video conversion service is a newly added proxy service that transfers audio and video data between the device and the streaming media. It adopts K8s containerization and service port configuration deployment mode, uniformly exposes the same port to the outside, and the address information sent to the device by the device access service call interface remains consistent. The device uses the address information to connect to the platform service. When the video conversion service becomes abnormal and cannot be used, K8s transfers the device connection to the normal video conversion service to ensure normal business operations. In terms of load balancing, nginx or K8s load strategy is used to forward the connection to the video conversion service, and the receiving and forwarding are deployed to achieve load balancing.

3. The method according to claim 2, characterized in that: Step S3 includes a single domain deployment implementation method: in a scenario where the system has only one domain address, the platform deploys a video conversion service and configures the address port information. After the video conversion service is deployed, its domain information is uploaded to Redis for backup or deployed to the device access service environment variables; when subsequently acquiring the stream, the device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service based on the received address port and sends audio and video data; the video conversion service can be deployed in multiple instances to achieve high availability. The port exposed by the video conversion service is always consistent. With the help of nginx or K8s balancing strategy, the connection is forwarded to the video conversion service to ensure dynamic balance of the service business.

4. The method according to claim 3, characterized in that: Step S3 includes a multi-domain deployment implementation method: in a scenario where there are multiple domains mixed in the system, the domain environment of the video conversion service is deployed according to the actual resource distribution. After the video conversion service is deployed, its domain information is uploaded to Redis for backup. When the stream is subsequently retrieved, the device access service obtains the domain load and the same-domain load data on Redis, selects the domain data according to the strategy, and synchronously refreshes the domain load information to achieve load balancing of multiple domains. The device access service calls the interface to send the address information to the device to complete the business operation. The device connects to the video conversion service according to the received address port and sends audio and video data. The receiving stream connection service in the same domain is executed in accordance with the single domain solution.

5. The method according to claim 4, characterized in that: The order and process of multi-instance service deployment are as follows: (1) Multi-instance deployment of platform video conversion service: Multi-instance deployment of video conversion service is carried out on the platform. This operation is the basis for realizing efficient and flexible data processing of the system. By deploying multiple instances, it can effectively respond to business needs of different scales and possible service abnormalities, and improve the overall availability and stability of the system; (2) Interaction between the video conversion service and Redis: The video conversion service updates its network domain and load information to Redis in real time or periodically, and the amount of data matches the number of network domains. This step enables subsequent modules to obtain accurate and real-time updated data from Redis when they need relevant information, providing a key basis for system operations such as load balancing and network domain selection; (3) The client initiates a stream acquisition request: The client initiates a stream acquisition request, which marks the beginning of the entire business process. Its purpose is to obtain the video surveillance data that is expected to be viewed. For example, this request is triggered when a user uses a monitoring terminal to view the real-time monitoring screen of a certain area. (4) The request is transferred to the user rights management service: The client's stream acquisition request is first sent to the user rights management service. The user rights management service is mainly responsible for the authority judgment, that is, checking whether the client initiating the request has the authority to obtain the corresponding video stream. Only when the authority judgment is successful, the request is allowed to continue to flow downward, thereby ensuring the security of video surveillance data and the legality of access; (5) User authority management forwards the request to the streaming media scheduling service: When the authority judgment is passed, the user authority management service forwards the streaming request to the streaming media scheduling service. The streaming media scheduling service is responsible for selecting streaming media and creating channels, as well as assisting in the relevant operations of streaming media selection. They work together to complete the task of selecting appropriate streaming media to ensure that the audio and video data transmitted by the device can be successfully received and processed in the future; (6) Interaction between streaming media scheduling service and device access service: After successfully selecting streaming media and creating a channel, the streaming media scheduling service sends a request to the device access service. The device access service then determines whether the device meets the conditions for receiving the stream. The qualified device enters the subsequent data transmission process; (7) The device access service interacts with Redis to obtain information: In order to determine the optimal network domain for the device to connect to, the device access service obtains the incoming traffic domain information from Redis. Based on this information, the device access service can select the optimal network domain according to factors such as load information, so that the device can be connected to the most appropriate part of the platform, ensuring the efficient operation and load balancing of the overall business; (8) The device access service sends network domain information to the device: After the device access service determines the optimal network domain, it calls the SDK stream acquisition interface to send the network domain information to the device. If successful, it caches the stream identifier and streaming media information to Redis. After the device receives this information, it knows how to connect to the platform for subsequent data sending operations. This step ensures that the connection between the device and the platform is based on accurate and effective information. (9) Device connection platform and video conversion service: The device connects to the platform based on the received domain address information. After the platform receives the device connection request, it uses nginx or K8s balancing strategy to transfer the connection to the video conversion service. The balancing strategy here can ensure that the device is connected to the appropriate video conversion service instance, realizing efficient and stable connection and sending and receiving stream services between the device and the video conversion service; (10) The video conversion service performs streaming service processing and forwards data: After the device successfully connects to the video conversion service, the video conversion service starts the streaming service processing flow, receives the audio and video data sent by the device, then obtains the streaming media address information based on the stream identification information, and forwards this data to the streaming media, thereby achieving the transmission of audio and video data from the front-end device to the platform's internal streaming media, ensuring that the monitoring data can smoothly enter the platform for subsequent processing; (11) The client receives audio and video data: The client connects to the streaming media and receives the audio and video data sent by the streaming media. This completes the entire process from the client initiating a streaming request to finally receiving the required video surveillance data, thus realizing the normal streaming service of the video surveillance platform.

6. The method according to claim 4 or 5, characterized in that: The operations of the device access service obtaining information from Redis include: obtaining domain information in single-domain deployment, obtaining domain information and domain load data in multi-domain deployment, and in multi-domain deployment, the device access service will synchronously refresh the domain load information in Redis after successful processing, and implement load balancing of multi-domain deployment solutions based on Redis cache data. Specifically, when the device access service selects the domain address information to send to the device, it obtains the domain data of the video conversion service from Redis and selects the domain address with the least load. After the device access service is successfully processed, the domain load information is refreshed, and the device connects to the platform based on the domain information. The platform uses nginx strategy and K8s strategy to forward to the video conversion service.

7. The method according to claim 5, characterized in that: A ZK-based multi-active mechanism within the group is adopted. Under this mechanism, when some video conversion services are abnormal, the device can automatically retry to connect to other normal video conversion services for business processing, thereby ensuring that as long as not all video conversion services in the group are offline at the same time, the business will not be affected; the device is non-directional when connecting to the platform's video conversion service and can connect to any video conversion service on the surviving node.

8. The method according to claim 5, characterized in that: In the single-domain deployment solution, the platform uses nginx or K8s balancing strategies to transfer device connections to the video conversion service to achieve a stable connection between the device and the video conversion service and efficient streaming services, thereby achieving load balancing for the single-domain deployment solution.

9. The method according to claim 5, characterized in that: The video conversion service is used to realize two-way connection and forwarding data. The connection between the video conversion service and the front-end device and the platform is stateless. The domain address obtained by the device corresponds to a node rather than a pod or service. When the device connects to the platform's video conversion service, it can connect to any video conversion service on the surviving node without connection directionality.

Images