Video information transmission control method and device, electronic equipment and storage medium

Abstract

The application provides a video information transmission control method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring video service model description information transmitted by a video acquisition terminal; acquiring a bearer identifier of a communication network; configuring an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information; and when the optimized scheduling strategy matches the video service model description information, controlling the transmission of video information collected by the video acquisition terminal based on the optimized scheduling strategy. The application can realize the optimized scheduling strategy, control the transmission of video information collected by the video acquisition terminal, reduce the time delay of video transmission, ensure the timely and accurate transmission of video, and improve the user experience.

Description

Technical Field

[0001] This invention relates to video information transmission control and processing technology, and more particularly to video information transmission control methods, devices, systems, equipment, and storage media. Background Technology

[0002] In related technologies, the end-to-end latency from video capture, encoding, and network transmission to video decoding, rendering, and playback needs to be controlled within 100ms. However, in 4G / 5G mobile communication systems, the uplink air interface network transmission latency for video streams alone can be around 10-20ms, putting significant pressure on the overall latency performance and hindering the transmission of real-time video capture, thus limiting its applicability. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a video information transmission control method, apparatus, electronic device, and storage medium, which can realize optimized scheduling strategies, control the transmission of video information collected by video acquisition terminals, reduce video transmission latency, ensure timely and accurate video transmission, and improve user experience.

[0004] The technical solution of this invention is implemented as follows:

[0005] This invention provides a video information transmission control method including:

[0006] Obtain the video service model description information transmitted by the video acquisition terminal;

[0007] Obtain the bearer identifier of the communication network;

[0008] Based on the bearer identifier of the communication network and the video service model description information, an optimized scheduling strategy is configured for the video acquisition terminal;

[0009] When it is determined that the optimized scheduling strategy matches the video service model description information, the video information collected by the video acquisition terminal is controlled to be transmitted based on the optimized scheduling strategy.

[0010] This invention also provides a video information transmission control device, comprising:

[0011] The information transmission module is used to acquire video service model description information transmitted by the video acquisition terminal;

[0012] The information transmission module is used to obtain the bearer identifier of the communication network;

[0013] The information processing module is used to configure an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information;

[0014] The information processing module is used to determine that when the optimized scheduling strategy matches the video service model description information, it controls the transmission of the video information collected by the video acquisition terminal based on the optimized scheduling strategy.

[0015] In the above scheme,

[0016] The information processing module is used to determine a sampling period that matches the video service model description information based on the usage environment of the video acquisition terminal.

[0017] The information processing module is used to determine at least one data pair based on the sampling period, the transmission time parameters corresponding to different scheduling requests, and the data volume parameters of the uplink video data.

[0018] The information processing module is used to respond to the at least one data pair and, based on the usage environment of the video acquisition terminal, trigger a matching algorithm corresponding to the usage environment of the video acquisition terminal.

[0019] The information processing module is used to match the optimized scheduling strategy with the video service model description information based on a matching algorithm corresponding to the usage environment of the video acquisition terminal.

[0020] This invention also provides an electronic device, the electronic device comprising:

[0021] Memory, used to store executable instructions;

[0022] The processor, when executing the executable instructions stored in the memory, implements the aforementioned video information transmission control method.

[0023] This invention also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the aforementioned video information transmission control method.

[0024] The embodiments of the present invention have the following beneficial effects:

[0025] This invention acquires video service model description information transmitted by a video acquisition terminal; acquires the bearer identifier of the communication network; configures an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information; and when the optimized scheduling strategy matches the video service model description information, controls the transmission of video information acquired by the video acquisition terminal based on the optimized scheduling strategy. This enables the optimized scheduling strategy to control the transmission of video information acquired by the video acquisition terminal, reducing video transmission latency, ensuring timely and accurate video transmission, and improving the user experience. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the usage environment of the video information transmission control method provided in the embodiments of the present invention;

[0027] Figure 2 This is a schematic diagram of the composition structure of the video information transmission control device provided in an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram illustrating the encoding of video transmission using related technologies.

[0029] Figure 4 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of an optional process for video information transmission control in an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of an optional process for video information transmission control in an embodiment of the present invention;

[0032] Figure 7 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention;

[0033] Figure 8 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention;

[0034] Figure 9 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention;

[0035] Figure 10 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0038] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0039] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0040] 1) Video encoding (Video Transcoding) refers to converting an already compressed and encoded video stream into another video stream to adapt to different network bandwidths, different terminal processing capabilities, and different user needs.

[0041] 2) Terminals, including but not limited to: ordinary terminals and dedicated terminals, wherein the ordinary terminals maintain a long connection and / or a short connection with the transmission channel, and the dedicated terminals maintain a long connection with the transmission channel.

[0042] 3) Client: A carrier in a terminal that implements specific functions. For example, a mobile client (APP) is a carrier of specific functions in a mobile terminal, such as performing online live streaming or playing online videos.

[0043] 4) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0044] 5) P-frame: Inter-frame prediction frame, which can use intra-frame prediction and inter-frame prediction, and can use forward reference prediction video coding method.

[0045] 6) B-frame: Inter-frame prediction frame, which can use intra-frame prediction and inter-frame prediction, and can use forward, backward and bi-directional reference prediction.

[0046] 7) I-frame: Intra-predictive frame, which uses intra-frame information for prediction.

[0047] Figure 1 This is a schematic diagram illustrating a usage scenario of the video information transmission control method provided in an embodiment of the present invention. See also... Figure 1The terminals (including terminals 10-1 and 10-2) are equipped with corresponding clients capable of performing different functions. These clients, on the other hand, allow terminals 10-1 and 10-2 to access and browse different video information from their respective servers 200 via network 300 using different business processes. The terminals connect to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both, using a wireless link for data transmission. The types of video accessed by the terminals 10-1 and 10-2 from server 200 via network 300 are not the same. For example, terminals 10-1 and 10-2 can access video (i.e., video containing video information or corresponding video links) or real-time video (such as camera monitoring video from industrial equipment or road video captured by cameras in autonomous driving systems) via network 300. Server 200 can store different types of video. In some embodiments of the present invention, the processes for storing different types of videos in the server 200 can be written in software code of different programming languages, and the code objects can be different types of code entities. For example, in C language software code, a code object can be a function. In JAVA language software code, a code object can be a class, and in iOS Objective-C, it can be a piece of object code. In C++ language software code, a code object can be a class or a function. This application does not distinguish between the compilation environments of different types of videos.

[0048] During the process of server 200 sending or receiving different types of video to terminals (terminal 10-1 and / or terminal 10-2) via network 300, due to the high requirements for latency, it is necessary to optimize the transmission control of video information. Specifically, this can be achieved by obtaining the video service model description information transmitted by the video acquisition terminal; obtaining the bearer identifier of the communication network; configuring an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information; and when it is determined that the optimized scheduling strategy matches the video service model description information, controlling the transmission of the video information acquired by the video acquisition terminal based on the optimized scheduling strategy, thereby reducing video transmission latency, ensuring timely and accurate video transmission, and improving the user experience.

[0049] In this invention, embodiments can be implemented using cloud technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It can also be understood as a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on cloud computing business models. The backend services of network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites; therefore, cloud technology needs cloud computing as its support.

[0050] It's important to note that cloud computing is a computing model that distributes computing tasks across a resource pool comprised of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" are infinitely scalable, readily available, and can be used on demand, expanded at any time, and paid for based on usage. As the foundational providers of cloud computing capabilities, they establish cloud resource pool platforms, often referred to as cloud platforms or Infrastructure as a Service (IaaS). These platforms deploy various types of virtual resources within the resource pool for external customers to choose from. The cloud resource pool primarily includes: computing devices (which can be virtualized machines containing operating systems), storage devices, and network devices.

[0051] In conjunction with the embodiments Figure 1 As shown, the target object determination method provided in this embodiment of the invention can be implemented through corresponding cloud devices. For example, terminals (including terminals 10-1 and 10-2) connect to a server 200 located in the cloud via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. It is worth noting that the server 200 can be a physical device or a virtualized device.

[0052] Specifically, in conjunction with the preceding embodiments Figure 1 As shown, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0053] The structure of the video information transmission control device according to an embodiment of the present invention will be described in detail below. The video information transmission control device can be implemented in various forms, such as a dedicated terminal with video information transmission control device processing function, or a server or server group equipped with video information transmission control device processing function, such as a target system deployed in a target system, such as the preceding sequence. Figure 1 Server 200 in the middle. Figure 2 This is a schematic diagram of the composition of the video information transmission control device provided in an embodiment of the present invention. It can be understood that... Figure 2 The diagram shows only an exemplary structure of the video information transmission control device, not the entire structure; implementation is possible as needed. Figure 2 The structure shown may be part or all of the structure.

[0054] The video information transmission control device provided in this embodiment of the invention includes: at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The various components in the video information transmission control device are coupled together through a bus system 205. It can be understood that the bus system 205 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 205.

[0055] The user interface 203 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0056] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 202 is capable of storing data to support the operation of a terminal (such as 10-1). Examples of this data include any computer programs used to operate on the terminal (such as 10-1), such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0057] In some embodiments, the video information transmission control device provided in this invention can be implemented using a combination of hardware and software. For example, the video information transmission control device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the video information transmission control method provided in this invention. For instance, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0058] As an example of the video information transmission control device provided in this embodiment of the invention, which is implemented using a combination of hardware and software, the video information transmission control device provided in this embodiment of the invention can be directly embodied as a combination of software modules executed by processor 201. The software modules can be located in a storage medium, which is located in memory 202. Processor 201 reads the executable instructions included in the software modules in memory 202 and combines them with necessary hardware (e.g., including processor 201 and other components connected to bus 205) to complete the video information transmission control method provided in this embodiment of the invention.

[0059] As an example, processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0060] As an example of the hardware implementation of the video information transmission control device provided in the embodiments of the present invention, the device provided in the embodiments of the present invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, it can be executed by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the video information transmission control method provided in the embodiments of the present invention.

[0061] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the video information transmission control device. Examples of such data include: any executable instructions for operation on the video information transmission control device, such as executable instructions, whereby a program implementing the video information transmission control method of this embodiment of the invention may be included in the executable instructions.

[0062] In other embodiments, the video information transmission control device provided in this invention can be implemented in software. Figure 2 A video information transmission control device stored in memory 202 is shown. This device can be software in the form of programs and plugins, and includes a series of modules. As an example of a program stored in memory 202, it may include the video information transmission control device. The video information transmission control device includes the following software modules: an information transmission module 2081 and an information processing module 2082. When the software modules in the video information transmission control device are read into RAM and executed by processor 201, the video information transmission control method provided in this embodiment of the invention will be implemented. The functions of each software module in the video information transmission control device include:

[0063] Information transmission module 2081 is used to acquire video service model description information transmitted by the video acquisition terminal;

[0064] The information transmission module 2081 is used to obtain the bearer identifier of the communication network;

[0065] The information processing module 2082 is used to configure an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information;

[0066] The information processing module 2082 is used to determine that when the optimized scheduling strategy matches the video service model description information, it controls the transmission of the video information collected by the video acquisition terminal based on the optimized scheduling strategy.

[0067] Before introducing the video information transmission control method provided in this application, the shortcomings of related technologies are first described, with reference to... Figure 3 , Figure 3 This diagram illustrates the encoding process for video transmission using related technologies. During real-time video transmission, the end-to-end latency, from video acquisition, encoding, and network transmission to video decoding, rendering, and playback, needs to be controlled within 100ms. However, in 4G / 5G mobile communication systems, the uplink air interface network transmission latency for the video stream alone can reach approximately 10-20ms, placing significant pressure on the overall latency performance across the entire link.

[0068] The reasons for significant air interface latency include: Mobile communication systems employ a base station-based scheduling transmission mechanism. When a video capture terminal needs to send uplink data, it cannot do so immediately. It must first request uplink transmission resources from the base station. This request signaling, known as a scheduling request (SR), can only be sent on a specific uplink frame. After receiving the scheduling request from the terminal, the base station sends an uplink scheduling authorization signaling to the terminal in a downlink frame. This signaling contains information about the uplink time-frequency resources that the terminal can send. Upon receiving this scheduling authorization signaling, the terminal then performs uplink data transmission on the specified uplink time-frequency resources (the OFDM symbols specified on the specified uplink frame). Figure 3 As shown, D represents a downlink frame, S represents a special frame, and U represents an uplink frame. Each frame is fixed at 1ms, with one uplink frame every 5ms. The video capture terminal can only send scheduling requests on the assigned specific uplink frames. Figure 3 This indicates that the scheduling request period is 10ms. Of course, if the terminal has no uplink data to send, it will not send a scheduling request. At some point, if uplink data arrives at the terminal, a scheduling request will be triggered, and the terminal will send the scheduling request on the next available uplink frame. After receiving the terminal's scheduling request, the base station will perform scheduling processing and send a scheduling grant to the terminal on a downlink subframe. After receiving the scheduling grant, the terminal will send uplink data on the specified OFDM symbol in its designated uplink frame.

[0069] Therefore, there is a 3ms delay from when the data reaches the terminal to when the terminal sends the SR signaling to the base station; there is another 10ms delay from when the base station receives and decodes the SR, then sends the scheduling authorization to the terminal, and finally when the terminal actually sends the uplink data on the uplink frame. Thus, the entire process introduces a 13ms delay (not considering the base station's decoding time for uplink data). Furthermore, if the base station fails to decode the SR request, the terminal can only retransmit the SR request on the uplink frame of the next SR cycle, thus introducing a 10ms delay.

[0070] To address the aforementioned shortcomings, combined with Figure 1 The server 200 shown illustrates the video information transmission control method provided in this embodiment of the invention. See also: Figure 4 , Figure 4 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention. It can be understood that... Figure 4 The steps shown can be performed by various servers running the video information transmission control device, such as dedicated terminals, servers, or server clusters with video information transmission control functions. The following section addresses... Figure 4 The steps shown are explained.

[0071] Step 401: The communication base station obtains the video service model description information transmitted by the video acquisition terminal.

[0072] In some embodiments of the present invention, the video acquisition terminal is exemplified by a mobile phone. The mobile phone includes an application processor (AP) and a baseband processor (BP). The application processor determines the service model based on the current service characteristics. For example, the application layer can determine the frame rate per second, bit rate, etc., based on the relevant parameters set by the H.264 encoder, while the baseband processor performs video transmission. (See reference...) Figure 5 , Figure 5This is a schematic diagram of an optional process for video information transmission control in an embodiment of the present invention. In the video encoding sequence, GOP (Group of Pictures) refers to the distance between two I-frames, and Reference refers to the distance between two P-frames. An I-frame occupies more bytes than a P-frame, and a P-frame occupies more bytes than a B-frame. With a constant bitrate, a larger GOP value results in more P and B frames, more image detail, and thus easier acquisition of better image quality. Similarly, a larger Reference value results in more B frames, also making it easier to achieve better image quality. However, there are limits to improving image quality by increasing the GOP value. When scene transitions occur, the H.264 encoder automatically inserts an I-frame, shortening the actual GOP value. Furthermore, within a GOP, P and B frames are predicted from I-frames. If the image quality of an I-frame is poor, it will affect the image quality of subsequent P and B frames within that GOP, which may not be recovered until the next GOP begins. Therefore, the GOP value should not be set too high. Furthermore, since P and B frames are more complex than I frames, an excessive number of P and B frames can negatively impact coding efficiency, leading to a decrease in coding efficiency. Additionally, an excessively long GOP (Group of Pictures) can affect the response speed of Seek operations (finding I frames). Because P and B frames are predicted from preceding I or P frames, Seek operations require direct location. When decoding a P or B frame, it is necessary to first decode the I frames within the current GOP and the preceding N predicted frames. The longer the GOP, the more predicted frames need to be decoded, and the longer the seek response time. Therefore, by using the video coding decision-making method provided in this embodiment to determine the coding method to be used and processing the video to be encoded, the efficiency of video coding can be improved, and the video coding time can be reduced.

[0073] refer to Figure 6 In VoLTE services, voice packets are sent with a period of 20ms.

[0074] like Figure 6As shown, in real-time video streaming services, to reduce latency, a group of Groups of Pictures (GOPs) typically contains only I-frames and P-frames, without B-frames. I-frames are intra-coded frames; the first frame of a GOP sequence is always an I-frame. P-frames are forward-predictive coded frames, representing the difference between this frame and a previous I-frame or P-frame. Because I-frames compress and encode the entire frame's image information, they occupy a much larger amount of data than P-frames. Therefore, for this video streaming service scenario, the service model description (SMD) can be represented in the following general form: SMD = [F1, T1; F2, T2; ...; FN, TN]. Here, N is the number of frames in a GOP, F1 represents the size of the first frame in a GOP, T1 represents the latency between the first and second frames, accurate to milliseconds; similarly, FN represents the size of the last frame in a GOP, and TN represents the latency between the last frame and the first frame of the next GOP. Of course, it can also be described in other forms, such as [I, P, T], where I represents the I-frame size, P represents the P-frame size, and T represents the frame rate. In some embodiments of the present invention, considering that control latency is more important than control bandwidth in remote control scenarios, especially in 5G industry private networks, on the one hand, there are only users of the industry private network in the network, and on the other hand, taking the 5G network as an example, the frame size of SMD is increased proportionally according to the 5G network bandwidth to achieve the margin protection effect of base station scheduling. Specifically, the bandwidth capacity of the 5G network is very large. Therefore, when estimating the frame size in SMD, it can be appropriately relaxed, for example, increased by 20%, to leave a certain margin protection for subsequent base station scheduling.

[0075] Step 402: The communication base station obtains the bearer identifier of the communication network.

[0076] In some embodiments of this invention, during video transmission, the terminal sends SMD information to a service server, such as the backend server of a remote control application, via a 5G network (hereinafter, 5G networks are used as an example; the 4G network is similar). This is communication between application-layer peer entities. Simultaneously, the terminal also transmits its IP address (generally a private network IP) in the 5G network to the service server. Upon receiving the SMD and IP address information, the service server transmits the SMD and IP address information to the 5G core network through an EF (Network Exposure Function) in the 5G network. The 5G core network first uses the IP address information to look up the 5G base station currently serving the user and the corresponding bearer identifier. Then, the 5G core network transmits the bearer identifier and SMD information to the corresponding 5G base station. The reason for transmitting the bearer identifier is that 4G or 5G base stations do not recognize the terminal's IP address. On the air interface, the base station identifies the user based on its assigned Radio Network Temporary Identifier (RNTI), while the base station and the 5G core network identify the user through the bearer ID.

[0077] Step 403: The communication base station configures an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information.

[0078] In some embodiments of the present invention, configuring an optimized scheduling strategy for the video acquisition terminal can be achieved in the following ways:

[0079] Based on the bearer identifier of the communication network, a video acquisition terminal matching the bearer identifier of the communication network is determined; a scheduling request period parameter is configured for the video acquisition terminal; in response to the scheduling request period parameter, a first transmission time parameter for a first scheduling request is determined; based on the video service model description information, the uplink channel of the video acquisition terminal is determined; using the uplink subframe corresponding to the uplink channel, uplink video data sent by the video acquisition terminal is received; a first data volume of the uplink video data sent by the video acquisition terminal is determined; when the first data volume of uplink video data is transmitted, a second scheduling request is triggered; in response to the second scheduling request, the second data volume of uplink video data is controlled to be transmitted from the video acquisition terminal to the communication base station.

[0080] Taking 5G networks as an example, refer to Figure 7 , Figure 7 An optional flowchart of the video information transmission control method provided in this embodiment of the invention specifically includes the following steps:

[0081] Step 701: Configure the scheduling request period parameters for the video acquisition terminal.

[0082] In this invention, the base station configures the secondary terminal's SR period to the minimum value, which is the uplink subframe period. For example, in a typical 5G network, the uplink subframe period is 5ms. To save SR resources (i.e., physical uplink control channel resources) and serve more users, a common SR period configuration parameter is 20ms. In this invention, the secondary user's SR period is first modified to the minimum value of 5ms. Of course, in subsequent processes, once the base station has completed processing and started stably scheduling the terminal for uplink transmission, the SR period can be modified back to 20ms to save physical uplink control channel resources. Due to equipment model limitations in actual use, since the base station cannot identify the user's mobile phone number or IMSI number, the network management system cannot set different SR values ​​for different users. This preferred configuration can only set a uniform value, such as 5ms or 20ms, for all users in the cell.

[0083] Step 702: Encode the video information acquired by the video acquisition terminal and determine the corresponding initial transmission time parameters.

[0084] When a terminal receives new uplink data arriving in its buffer, it needs to immediately initiate the uplink transmission process. The video capture terminal can trigger a video encoding process matching the usage environment. The video capture terminal will then send an SR request to the base station in the next available SR resource. The base station records this SR transmission time, designated as the first transmission time TS1. (The initial transmission time TS0 can be timed according to the base station's own frame number, accurate to 1ms.)

[0085] Step 703: The base station sends an uplink scheduling request authorization information to this terminal in the downlink subframe.

[0086] In this process, the scheduling terminal transmits uplink data on the uplink subframe closest to and available to TS0, and the required uplink resources are determined in the previous step. Simultaneously, the base station configures the terminal to report BSR (Buffer Status Report) status information along with the uplink data in the MAC signaling.

[0087] Step 704: The terminal sends uplink data on the scheduled uplink subframe.

[0088] The uplink channel of the video acquisition terminal can be determined based on the maximum frame of the video group in the video service model description information and the first scheduling request information; or, when the number of data channels corresponding to the communication base station is lower than the data channel number threshold, the uplink channel of the video acquisition terminal can be determined based on the minimum frame of the video group in the video service model description information and the first scheduling request information. Whether to select the maximum or minimum frame of the video group can be flexibly adjusted according to the usage environment of the video acquisition terminal. For example, the minimum frame of the video group can be selected for video transmission of medical equipment, while the maximum frame of the video group can be selected in the remote operation environment of a gantry crane. Of course, in some embodiments of the present invention, in order to reduce the impact of human intervention on the video information transmission control method provided in this application, the maximum frame of the video group can also be fixed.

[0089] Step 705: The base station receives the uplink data sent by the terminal and calculates the amount of data transmitted for video.

[0090] Specifically, when all the uplink video data sent by the video acquisition terminal is sent, the amount of uplink video data sent is determined as the first data amount; when the uplink video data sent by the video acquisition terminal is not sent, the first data amount of uplink video data sent by the video acquisition terminal is determined based on the status report of the video acquisition terminal and the amount of video data sent in the current uplink frame.

[0091] The base station can identify the uplink data volume of the video data actually sent by the terminal based on the BSR Buffer Status Report or whether the terminal has invalid data padding. If the terminal has not sent all the data in the buffer, the base station calculates P1 as the uplink data volume of the video data based on the sum of the BSR Buffer Status Report reported by the terminal along with the current uplink frame and the data volume sent in the current uplink frame.

[0092] Step 706: Determine the second transmission time information.

[0093] As new data arrives at the terminal, the terminal sends the next SR. The terminal and base station repeat steps 701 to 705. The new data is denoted as P2. The transmission time of this SR is also recorded and denoted as TS2.

[0094] After completing step 706, it can be further determined whether the optimized scheduling strategy matches the video service model description information. Specifically, based on the usage environment of the video acquisition terminal, a sampling period matching the video service model description information can be determined; at least one data pair can be determined according to the sampling period, the transmission time parameters corresponding to different scheduling requests, and the data volume parameters of uplink video data; in response to the at least one data pair, a matching algorithm corresponding to the usage environment of the video acquisition terminal can be triggered; and the optimized scheduling strategy and the video service model description information can be matched based on the matching algorithm corresponding to the usage environment of the video acquisition terminal. Continuing with the 5G network as an example, we can use N data points as one SMD cycle, and the video acquisition terminal and base station repeat at least N1 SMD cycles. (j) The base station matches these K data pairs with SMD = [F1, T1; F2, T2; ...; FN, TN], resulting in a total of K = N*N1 data pairs, denoted as [P1, TS1; P2, TS2; ...; PK, TSK]. Figure 5 As shown, in H.264 encoded video streams, the size of I-frames is much larger than that of P-frames, which is a characteristic of H.264 encoding. Therefore, matching can be performed using feature matching algorithms based on this characteristic.

[0095] Step 404: When the communication base station determines that the optimized scheduling strategy matches the video service model description information, it controls the transmission of the video information collected by the video acquisition terminal based on the optimized scheduling strategy.

[0096] In some embodiments of the present invention, when it is determined that the optimized scheduling strategy does not match the video service model description information, the communication base station sends optimization failure information to the corresponding service server;

[0097] In response to the optimization failure information, the original scheduling strategy that matches the communication base station is executed. This ensures that the video acquisition terminal still transmits video to the communication base station and avoids video transmission failure due to mismatch between the optimized scheduling strategy and the video service model description information.

[0098] In some embodiments of the present invention, when it is determined that the optimized scheduling strategy matches the video service model description information, the base station can determine the data pairs corresponding to [F1, T1; F2, T2; ...; PK, TSK] and [FN, TN]. For example, assume that [P6, TS6] corresponds to [F1, T1], and [P7, TS7] corresponds to [F2, T2]. It should be noted that in the use of 5G networks, in addition to data packets, the terminal may need to send some additional uplink control packets (such as heartbeat packets). The connection order of P6 and P7 is not necessarily sequential. The time interval between TS6 and TS7 and the size of P6 and P7 must be considered for matching. Furthermore, the base station can combine the information in [P1, TS1; P2, TS2; ...; PK, TSK] and [F1, T1; F2, T2; ...; FN, TN] to determine the terminal's scheduling policy [SF1, ST1; SF2, ST2; ...; SFM, STM] and service period SP.

[0099] In some embodiments of the present invention, a corresponding proportion of redundancy can also be added during the determination of SF1 to ensure that the latency meets the user's usage requirements.

[0100] Continuing with 5G networks as an example, refer to Figure 8 , Figure 8 This is an optional flowchart illustrating the video information transmission control method provided in this embodiment of the invention. When it is determined that the optimized scheduling strategy matches the video service model description information, the video information collected by the video acquisition terminal is controlled to be transmitted based on the optimized scheduling strategy. Specifically, the method includes the following steps:

[0101] Step 801: The communication base station determines the target uplink subframe.

[0102] The base station determines the next target uplink subframe, which can be the target uplink subframe that is closest to the current time, ST1+L*SP, where L is a positive integer.

[0103] Step 802: Control the video acquisition terminal to send an uplink sniffing reference signal, and in response to the uplink sniffing reference signal, determine the uplink subframe that is in an available state.

[0104] The base station can determine the uplink subframe where the most recent uplink sniffing reference signal (SRS) is available before ST1+L*SP. The availability of SRS means that the base station has time to demodulate SRS and send an uplink scheduling authorization request targeting the uplink frame ST1+L*SP. The communication base station schedules the terminal to send the uplink SRS signal, which can help the communication base station to more accurately predict uplink scheduling resources, ensure that the data in the terminal buffer can be transmitted in one uplink frame, and reduce the latency introduced by multiple frame transmissions.

[0105] Step 803: Based on the detection results of the uplink sniffing reference signal and the optimized scheduling strategy, predict the required uplink resources for video transmission.

[0106] The communication base station estimates the required uplink resources based on the received SRS signal detection results and the SF1 value. To reduce retransmission latency, the target block error rate (BLER) of the video acquisition terminal can be set to 1% or 0.1%. Specifically, for I-frame data, the target BLER can be set to 0.1% to fully ensure its reliability; while for other service data of terminals not in the SMD, or data from other ordinary terminals, the target BLER can still be set to 10%.

[0107] In some embodiments of the present invention, when implementing the scheduling process using the video information transmission control method provided in this application, this user has the highest scheduling priority when competing for resources with other users because it is necessary to reduce the latency of this user. In particular, if all users are video stream users (for example, in a 5G local industry private network that only serves monitoring services, all users are camera terminals), since the transmission time of the I-frame can be determined according to SMD (e.g., the first frame is definitely an I-frame), the terminal that sends the I-frame has the highest priority, which further reduces the video transmission latency.

[0108] Step 804: Based on the prediction results of uplink resources for video transmission and the uplink subframes of available status, schedule the uplink subframes for sending uplink video data from the video acquisition terminal.

[0109] In this invention, the communication base station can actively schedule the uplink subframe at time ST1+L*SP to send uplink data without requiring the terminal to send an SR request. In some embodiments of the invention, the terminal's traditional SR request function still exists, and the SR period can be reallocated back to 20ms to save SR resources. If the terminal needs to send other uplink data at other times, such as uplink heartbeat packets, it can still trigger the base station to schedule uplink resources through the SR procedure.

[0110] To better illustrate the video information transmission control method provided in this application, the following example uses the real-time video transmission of a gantry crane in an automated terminal as a case study. Each gantry crane is typically operated by three drivers on a rotating basis, and a terminal usually requires hundreds of gantry crane drivers, resulting in high demand for manpower. After remote control modification, cameras are installed on the gantry cranes, and drivers operate them from the central control room, viewing multiple real-time video feeds. This allows for the precise movement of the crane's spreader and the lifting of containers. One remote control operator can control 3-6 gantry cranes, improving the working environment, reducing driver requirements, and significantly lowering labor costs. (Reference) Figure 9 , Figure 9 This is an optional flowchart illustrating the video information transmission control method provided in an embodiment of the present invention. The communication environment is a 5G network, and the video acquisition terminal is an internet camera. Specifically, it includes the following steps:

[0111] Step 901: Obtain the video service model description information of the Internet camera.

[0112] Step 902: The internet camera sends the video service model description information to the service server via the 5G network.

[0113] Step 903: After receiving the video service model description information and IP address information, the service server transmits the video service model description information and IP address information to the 5G core network through the network capability open network elements in the 5G network.

[0114] Step 904: The 5G core network uses the IP address information to look up the 5G base station currently serving this user and the corresponding bearer identifier. The 5G core network then transmits the bearer identifier and video service model description information to the corresponding 5G base station.

[0115] Step 905: The 5G base station configures an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information.

[0116] Step 906: The 5G base station determines whether the optimized scheduling strategy matches the video service model description information based on the matching algorithm corresponding to the usage environment of the video acquisition terminal. If it matches, proceed to step 907; otherwise, proceed to step 909.

[0117] Step 907: When it is determined that the optimized scheduling strategy matches the video service model description information, the video information collected by the video acquisition terminal is controlled to be transmitted based on the optimized scheduling strategy.

[0118] Step 909: 5G base stations adjust scheduling strategies in real time.

[0119] Step 909: The 5G base station returns a failure message to the service server via NEF, ending the process.

[0120] Steps 901-909 avoid the latency introduced by the SR process; they also avoid the latency introduced by multi-frame transmission; the latency introduced by the retransmission process; and the latency introduced by competing for resources with other users when the base station's uplink load is heavy. Simultaneous monitoring of multiple cranes allows for remote manual intervention in case of emergencies, significantly improving operational safety.

[0121] See Figure 10 , Figure 10 The present invention provides an optional flowchart of a video information transmission control method. It can be understood that... Figure 10 The video information transmission control method shown can be applied to the field of medical video processing, enabling the detection of lesions in patients by a robotic arm equipped with video acquisition capabilities, such as ultrasound images of renal bleeding and gallbladder tumors. Figure 10 The steps shown can be performed by various electronic devices of the video information transmission control device, such as medical robots with video transmission capabilities or medical robotic arms with human body examination capabilities. The following section addresses... Figure 10 The steps shown are explained.

[0122] Step 1001: Obtain the video service model description information and the bearer identifier of the communication network for the robotic arm.

[0123] Step 1002: The 5G base station triggers the configuration process of the visual robotic arm configuration optimization scheduling strategy based on the bearer identifier of the communication network and the video service model description information.

[0124] Step 1003: The 5G base station configures the period of the uplink subframe to the minimum value of 5ms.

[0125] Step 1004: The 5G base station selects the largest frame in the SMD, estimates the uplink channel of the robotic arm, calculates the uplink resources required by the terminal, and determines the scheduling strategy.

[0126] Step 1005: When it is determined that the scheduling strategy matches the video service model description information, the video information collected by the video acquisition terminal is controlled to be transmitted based on the optimized scheduling strategy.

[0127] The medical videos obtained by the robotic arm position determination method provided in this application include a collection of medical videos to be segmented for the same target region of the target object. These videos can be at least one CT image or endoscopic image of the lesion site of the same patient. Through robotic arm position adjustment, the medical videos can be flipped, rotated, zoomed, and contrast-enhanced. The type of input medical video can be a two-dimensional planar image or a three-dimensional image; for example, images generated by medical instruments such as computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, X-ray, electrocardiogram, electroencephalogram, and optical photography.

[0128] Step 1006: When the coordinate difference between the position of the robotic arm and the lesion detection position exceeds a threshold based on the transmitted video, the position of the robotic arm is adjusted.

[0129] Beneficial technical effects:

[0130] This invention acquires video service model description information transmitted by a video acquisition terminal; acquires the bearer identifier of the communication network; configures an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information; and when the optimized scheduling strategy matches the video service model description information, controls the transmission of video information acquired by the video acquisition terminal based on the optimized scheduling strategy. This enables the optimized scheduling strategy to control the transmission of video information acquired by the video acquisition terminal, reducing video transmission latency, ensuring timely and accurate video transmission, and improving the user experience.

[0131] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A video information transmission control method, the method being applied in a communication base station, characterized in that, The method includes: Obtain the video service model description information transmitted by the video acquisition terminal; Obtain the bearer identifier of the communication network, which is used to identify the user between the communication base station and the core network; Based on the bearer identifier of the communication network and the video service model description information, an optimized scheduling strategy is configured for the video acquisition terminal; When it is determined that the optimized scheduling strategy matches the video service model description information, the video information collected by the video acquisition terminal is controlled to be transmitted based on the optimized scheduling strategy. The step of configuring an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information includes: Based on the bearer identifier of the communication network, determine the video acquisition terminal that matches the bearer identifier of the communication network; Configure a scheduling request period parameter for the video acquisition terminal, and determine the transmission time parameter corresponding to the first scheduling request based on the scheduling request period parameter; Based on the usage environment of the video acquisition terminal, select the largest or smallest frame of the video service model description information; determine the uplink channel of the video acquisition terminal based on the largest frame of the video service model description information and the first scheduling request; or, when the number of data channels corresponding to the communication base station is lower than the data channel number threshold, determine the uplink channel of the video acquisition terminal based on the smallest frame of the video service model description information and the first scheduling request. When the first amount of uplink video data is transmitted through the uplink channel, a second scheduling request is triggered to control the transmission of the second amount of uplink video data from the video acquisition terminal to the communication base station.

2. The method according to claim 1, characterized in that, The method further includes: In response to the scheduling request period parameter, the video acquisition terminal is controlled to clear the video data in the uplink buffer; The video acquisition terminal triggers a video encoding process that matches the usage environment based on different usage environments; The video encoding process encodes the video information acquired by the video acquisition terminal and determines the corresponding initial transmission time parameters.

3. The method according to claim 1, characterized in that, The method further includes: When all the uplink video data sent by the video acquisition terminal has been sent, the amount of uplink video data sent is determined to be the first data amount; When the uplink video data sent by the video acquisition terminal is not fully transmitted, the first data volume of the uplink video data sent by the video acquisition terminal is determined based on the status report of the video acquisition terminal and the data volume of the video data sent in the current uplink frame.

4. The method according to claim 1, characterized in that, The method further includes: Based on the usage environment of the video acquisition terminal, a sampling period that matches the video service model description information is determined; Based on the sampling period, the transmission time parameters corresponding to different scheduling requests, and the data volume parameters of the uplink video data, at least one data pair is determined; In response to the at least one data pair, a matching algorithm corresponding to the usage environment of the video acquisition terminal is triggered based on the usage environment of the video acquisition terminal. Based on a matching algorithm corresponding to the usage environment of the video acquisition terminal, the optimized scheduling strategy is matched with the video service model description information.

5. The method according to claim 4, characterized in that, The method further includes: When it is determined that the optimized scheduling strategy does not match the video service model description information, the communication base station sends optimization failure information to the corresponding service server. In response to the optimization failure information, the original scheduling strategy matching the communication base station is executed.

6. The method according to claim 1, characterized in that, When it is determined that the optimized scheduling strategy matches the video service model description information, the video information collected by the video acquisition terminal is controlled to be transmitted based on the optimized scheduling strategy, including: The communication base station determines the target uplink subframe; The video acquisition terminal is controlled to send an uplink sniffing reference signal, and in response to the uplink sniffing reference signal, an uplink subframe in an available state is determined. Based on the detection results of the uplink sniffing reference signal and the optimized scheduling strategy, the required uplink resources for video transmission are predicted. Based on the prediction results of uplink resources for video transmission and the uplink subframes of available status, the uplink subframes for sending uplink video data by the video acquisition terminal are scheduled.

7. A video information transmission control device, characterized in that, The device includes: The information transmission module is used to acquire video service model description information transmitted by the video acquisition terminal; The information transmission module is used to obtain the bearer identifier of the communication network, which is used to identify the user between the communication base station and the core network. The information processing module is used to configure an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information; The information processing module is further configured to, when determining that the optimized scheduling strategy matches the video service model description information, control the transmission of the video information collected by the video acquisition terminal based on the optimized scheduling strategy; The step of configuring an optimized scheduling strategy for the video acquisition terminal based on the bearer identifier of the communication network and the video service model description information includes: Based on the bearer identifier of the communication network, determine the video acquisition terminal that matches the bearer identifier of the communication network; Configure a scheduling request period parameter for the video acquisition terminal, and determine the transmission time parameter corresponding to the first scheduling request based on the scheduling request period parameter; Based on the usage environment of the video acquisition terminal, select the largest or smallest frame of the video service model description information; determine the uplink channel of the video acquisition terminal based on the largest frame of the video service model description information and the first scheduling request; or, when the number of data channels corresponding to the communication base station is lower than the data channel number threshold, determine the uplink channel of the video acquisition terminal based on the smallest frame of the video service model description information and the first scheduling request. When the first amount of uplink video data is transmitted through the uplink channel, a second scheduling request is triggered to control the transmission of the second amount of uplink video data from the video acquisition terminal to the communication base station.

8. The apparatus according to claim 7, characterized in that, The information processing module is used to control the video acquisition terminal to clear the video data in the uplink buffer in response to the scheduling request period parameter; The information processing module is used by the video acquisition terminal to trigger a video encoding process that matches the different usage environments. The information processing module is used to encode the video information acquired by the video acquisition terminal through the video encoding process, and determine the corresponding initial transmission time parameters.

9. The apparatus according to claim 7, characterized in that, The information processing module is used to determine the amount of uplink video data sent as the first data amount when all the uplink video data sent by the video acquisition terminal has been sent. The information processing module is used to determine the first data volume of the uplink video data sent by the video acquisition terminal based on the status report of the video acquisition terminal and the data volume of the video data sent in the current uplink frame when the uplink video data sent by the video acquisition terminal has not been fully sent.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the video information transmission control method according to any one of claims 1 to 6.

11. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the video information transmission control method according to any one of claims 1 to 6.

Images