A video transmission method and device, computer equipment and storage medium

Abstract

The present application relates to the technical field of image processing, and in particular to a video transmission method and device, computer equipment and storage medium, wherein the video transmission method comprises: acquiring a to-be-transmitted image, dividing the to-be-transmitted image into a plurality of data blocks according to a preset block rule; selecting a current to-be-transmitted data block from the plurality of data blocks, and transmitting the current to-be-transmitted data block to the memory of the second core of a heterogeneous framework; determining whether the current to-be-transmitted data block is a region of interest; and when the current to-be-transmitted data block is the region of interest, ending the transmission of the to-be-transmitted image. When transmitting the image, all data blocks do not need to be transmitted to the memory of the second core of the heterogeneous framework, but when the current to-be-transmitted data block is the region of interest, it is considered that the transmission of the to-be-transmitted image is successful, thereby the frequency of transmitting data blocks can be reduced, and the cost of starting transmission and resource consumption are reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a video transmission method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Currently, with the development of autonomous driving systems, the demand for and analysis of massive amounts of data has become a research focus. The analysis and processing of real-time image data has become crucial, especially since the transmission speed and efficiency of large real-time image data have a significant impact on the performance of many-core applications in heterogeneous architectures. In heterogeneous architectures, data is typically transferred between one on-chip memory space (also known as main memory) and another memory space (also known as local memory) via Direct Memory Access (DMA). Specifically, in heterogeneous architectures, a common method for transmitting images is to first divide the data into blocks and then transfer the data block by block via DMA. This requires numerous DMA operations, resulting in excessive data transfer startup overhead and resource consumption. Summary of the Invention

[0003] In view of this, the present invention provides a video transmission method, apparatus, computer device and storage medium to solve the problem of excessive data transmission startup overhead and resource consumption caused by the current method of transmitting data via DMA.

[0004] In a first aspect, embodiments of the present invention provide a video transmission method, the method comprising the following steps: acquiring an image to be transmitted, dividing the image to be transmitted into multiple data blocks according to a preset segmentation rule; selecting the current data block to be transmitted from the multiple data blocks, and transmitting the current data block to be transmitted to the memory of the second core of the heterogeneous framework; determining whether the current data block to be transmitted is a region of interest; and when the current data block to be transmitted is a region of interest, ending the transmission of the image to be transmitted.

[0005] The video transmission method provided in this embodiment first divides the image to be transmitted into multiple data blocks according to a preset block division rule. When transmitting the image, it is not necessary to transmit all the data blocks to the memory of the second core of the heterogeneous framework. Instead, when the current data block to be transmitted is the region of interest, the image to be transmitted is considered to have been successfully transmitted. This can reduce the frequency of transmitting data blocks and reduce the overhead and resource consumption of starting transmission.

[0006] In one alternative implementation, before selecting the current data block to be transmitted from multiple data blocks, the method further includes: when the image to be transmitted is the first frame image, inserting the numbers of the multiple data blocks into the splay tree.

[0007] This is because video stream transmission has the following two characteristics: (1) a data block that has just been transmitted is very likely to be accessed again after the next frame; (2) the next data block to be accessed is very likely to be located near a data block that was accessed not long ago. Using a splay tree can achieve the above objectives.

[0008] In one optional implementation, the video transmission method further includes the following steps: when the current data block to be transmitted is a region of interest, obtaining the number of the current data block to be transmitted in the splay tree; and climbing the number of the current data block to be transmitted in the splay tree to the root of the splay tree.

[0009] This is because, in a splay tree, when selecting a data block to be transmitted from multiple data blocks according to the order in the splay tree, the selection starts from the root. Raising the number of the current data block to be transmitted in the splay tree to the root ensures that the next image to be transmitted can quickly find a data block that meets the threshold during transmission, further reducing the frequency of data block transmission, reducing the overhead and resource consumption of initiating transmission, and improving transmission speed.

[0010] In one alternative implementation, selecting the current data block to be transmitted from a plurality of data blocks includes: selecting the current data block to be transmitted from a plurality of data blocks according to the order in the splay tree.

[0011] In other words, by utilizing splay trees, the frequency of transmitting data blocks can be further reduced, the overhead and resource consumption of initiating transmission can be reduced, and the transmission speed can be improved.

[0012] In one optional implementation, determining whether the current data block to be transmitted is a region of interest includes: obtaining the feature value of the current data block to be transmitted; determining whether the feature value reaches a preset threshold; and when the feature value reaches the threshold, determining that the current data block to be transmitted is a region of interest.

[0013] This provides a method for determining whether a data block is a region of interest, applicable to different application scenarios.

[0014] In one optional implementation, the video transmission method further includes the following steps: when the current data block to be transmitted is not a region of interest, obtaining the next data block to be transmitted, using the next data block as the current data block to be transmitted, and returning to the step of transmitting the current data block to be transmitted to the memory of the second core of the heterogeneous framework.

[0015] This allows the transmission of the image to be sent.

[0016] Thirdly, embodiments of the present invention also provide a video transmission method applied to the second core of a heterogeneous frame. The method includes the following steps: obtaining the current data block to be transmitted from the memory of the second core; calculating the feature value of the current data block to be transmitted; and sending the feature value to the first core of the heterogeneous frame, so that the first core of the heterogeneous frame can determine whether the current data block to be transmitted is a region of interest based on the feature value.

[0017] The video transmission method provided in this embodiment can determine whether the current data block to be transmitted is a region of interest by calculating the current data block to be transmitted. Therefore, it is not necessary to transmit all data blocks to the memory of the second core of the heterogeneous framework. Instead, when the current data block to be transmitted is a region of interest, the image to be transmitted is considered to be successfully transmitted. This can reduce the frequency of transmitting data blocks and reduce the overhead and resource consumption of starting transmission.

[0018] Thirdly, embodiments of the present invention also provide a video transmission device, the device including an acquisition module, a data block division module, a data block transmission module, a judgment module, and a termination transmission module; the acquisition module is used to acquire an image to be transmitted; the data block division module is used to divide the image to be transmitted into multiple data blocks according to a preset block division rule; the data block transmission module is used to select the current data block to be transmitted from the multiple data blocks and transmit the current data block to be transmitted to the memory of the second core of the heterogeneous framework; the judgment module is used to determine whether the current data block to be transmitted is a region of interest; when the current data block to be transmitted is a region of interest, the termination transmission module is used to terminate the transmission of the image to be transmitted.

[0019] Fourthly, embodiments of the present invention also provide a computer device, including a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the video transmission method of the first aspect or any corresponding embodiment thereof, or the video transmission method of the second aspect or any corresponding embodiment thereof.

[0020] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the video transmission method of the first aspect or any corresponding embodiment thereof, or the video transmission method of the second aspect or any corresponding embodiment thereof. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the DMA data transfer method;

[0023] Figure 2 This is a flowchart of a video transmission method according to an embodiment of the present invention;

[0024] Figure 3 This is a flowchart of another video transmission method according to an embodiment of the present invention.

[0025] Figure 4 This is a flowchart of yet another video transmission method according to an embodiment of the present invention;

[0026] Figure 5 This is a flowchart of another video transmission method according to an embodiment of the present invention;

[0027] Figure 6 This is a flowchart illustrating an example of a video transmission method according to an embodiment of the present invention;

[0028] Figure 7 This is a structural block diagram of a video transmission device according to an embodiment of the present invention;

[0029] Figure 8 This is a structural block diagram of another video transmission device according to an embodiment of the present invention;

[0030] Figure 9 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] DMA (Direct Memory Access) refers to copying data blocks from one memory space (also known as main memory) to another memory space (also known as local memory), providing high-speed data transfer between main memory and local memory. It opens a direct data transfer channel between RAM and I / O devices through hardware, which greatly improves the efficiency of the CPU.

[0033] Specifically, in a heterogeneous framework, main memory can be understood as the memory on the CPU side, and local memory can be understood as the memory of other cores besides the CPU, such as the content on the GPU side.

[0034] like Figure 1 As shown, the DMA data transfer method includes three stages: a preprocessing stage, a formal transfer stage, and a post-processing stage. The preprocessing stage involves writing the data to be transferred into the target memory space (also known as the destination address). The formal transfer stage involves transferring data block by block, specifically by modifying the address pointer to obtain the block data and continuously checking whether the data transfer is complete. Currently, in image data transfer, the image data transfer is considered complete when all data blocks contained in the image data have been transferred. The post-processing stage involves responding to DMA interrupt requests, stopping the execution of the main program, and completing the DMA completion processing. Therefore, transferring data block by block via DMA requires numerous DMA operations, resulting in excessive data transfer startup overhead and resource consumption.

[0035] According to an embodiment of the present invention, a video transmission method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] This embodiment provides a video transmission method that can be used on the first core of a heterogeneous architecture. The first core can be a CPU (Central Processing Unit). Specifically, the video transmission method can be applied to the DMA controller on the first core. For example, a corresponding interface can be encapsulated on the DMA controller, and the application only needs to call the interface.

[0037] Figure 2 This is a flowchart of a video transmission method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0038] Step S201: Obtain the image to be transmitted.

[0039] Specifically, the image to be transmitted can be a single image from the video stream.

[0040] Step S202: Divide the image to be transmitted into multiple data blocks according to the preset block division rules.

[0041] Specifically, the segmentation rules can be determined based on the needs of the upper-layer application. For example, the image to be transmitted can be divided into multiple data blocks in a top-to-bottom order.

[0042] It should be noted that a video stream contains multiple images, which can be divided into multiple data blocks according to the same segmentation rule.

[0043] Step S203: Select the current data block to be transmitted from multiple data blocks.

[0044] Specifically, the current data block to be transmitted can be selected from multiple data blocks in a preset order.

[0045] Step S204: Transfer the current data block to be transferred to the memory of the second core of the heterogeneous framework.

[0046] Specifically, DMA can be used to transfer the current data block to be transferred to the memory of the second core of the heterogeneous framework.

[0047] Step S205: Determine whether the current data block to be transmitted is a region of interest; if the current data block to be transmitted is a region of interest, proceed to step S206; otherwise, proceed to step S207.

[0048] Step S206: When the current data block to be transmitted is a region of interest, the image to be transmitted is determined to be successfully transmitted, and the transmission of the image to be transmitted ends.

[0049] Step S207: When the current data block to be transmitted is not the region of interest, select the next data block to be transmitted from multiple data blocks, and use the next data block to be transmitted as the current data block to be transmitted, and return to step S204.

[0050] This is because video streams are transmitted very quickly, and the differences between images are minimal. Therefore, when transmitting a video stream, it is unnecessary to transmit every single image from one storage area to another; only the regions of the image that need to be transmitted need to be transmitted from one storage area to another. In this embodiment of the invention, the region of interest can be understood as the region of the image that needs to be transmitted.

[0051] The video transmission method provided in this embodiment first divides the image to be transmitted into multiple data blocks according to a preset block division rule. When transmitting the image, it is not necessary to transmit all the data blocks to the memory of the second core of the heterogeneous framework. Instead, when the current data block to be transmitted is the region of interest, the image to be transmitted is considered to have been successfully transmitted. This can reduce the frequency of transmitting data blocks and reduce the overhead and resource consumption of starting transmission.

[0052] This embodiment provides a video transmission method that can be used on the second core of a heterogeneous framework. The second core can be a GPU (graphics processing unit). Specifically, the video transmission method can be applied to the DMA controller on the second core. For example, a corresponding interface can be encapsulated on the DMA controller, and the application only needs to call the interface.

[0053] Figure 3 This is a flowchart of another video transmission method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0054] Step S301: Obtain the current data block to be transferred from the memory of the second core.

[0055] Step S302: Calculate the feature value of the current data block to be transmitted.

[0056] In this embodiment of the invention, different feature value calculation methods can be used for different application scenarios.

[0057] Step S303: Send the feature value to the first core of the heterogeneous framework so that the first core of the heterogeneous framework can determine whether the data block to be transmitted is a region of interest based on the feature value.

[0058] The video transmission method provided in this embodiment can determine whether the current data block to be transmitted is a region of interest by calculating the current data block to be transmitted. Therefore, it is not necessary to transmit all data blocks to the memory of the second core of the heterogeneous framework. Instead, when the current data block to be transmitted is a region of interest, the image to be transmitted is considered to be successfully transmitted. This can reduce the frequency of transmitting data blocks and reduce the overhead and resource consumption of starting transmission.

[0059] This embodiment provides a video transmission method that can be used on the first core of a heterogeneous framework. In this example, the first core can be a CPU. Figure 4 This is a flowchart of another video transmission method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0060] Step S401: Obtain the image to be transmitted.

[0061] Step S402: Divide the image to be transmitted into multiple data blocks according to the preset block division rules.

[0062] Step S403: Select the current data block to be transmitted from multiple data blocks.

[0063] Step S404: Transfer the current data block to be transferred to the memory of the second core of the heterogeneous framework.

[0064] Specifically, DMA can be used to transfer the current data block to the memory of the second core in a heterogeneous architecture. For example, the second core could be a GPU.

[0065] Step S405: Obtain the feature value of the current data block to be transmitted sent by the second core of the heterogeneous framework.

[0066] In this embodiment of the invention, the feature value of the current data block to be transmitted can be calculated by the acceleration area of ​​the second core of the heterogeneous framework. Specifically, the calculation process of the acceleration area of ​​the second core of the heterogeneous framework is as follows: the current data block to be transmitted is obtained from the memory of the second core; the feature value of the current data block to be transmitted is calculated; the feature value is sent to the first core of the heterogeneous framework, so that the first core of the heterogeneous framework can determine whether the current data block to be transmitted is a region of interest based on the feature value.

[0067] Step S406: Determine whether the feature value has reached the preset threshold; when the feature value reaches the preset threshold, proceed to step S407; when the feature value has not reached the preset threshold, proceed to step S408.

[0068] It should be noted that, in the embodiments of the present invention, different thresholds can be set for different feature value calculation methods. This makes the embodiments of the present invention more adaptable to different scenarios.

[0069] Step S407: When the feature value reaches the threshold, it is determined that the image to be transmitted has been successfully transmitted, and the transmission of the image to be transmitted ends;

[0070] Step S408: When the feature value does not reach the threshold, select the next data block to be transmitted from multiple data blocks, and use the next data block to be transmitted as the current data block to be transmitted, and return to step S404.

[0071] The video transmission method provided in this embodiment can not only reduce the frequency of transmitting data blocks, thus reducing the overhead and resource consumption of starting transmission; but also provide a method for determining whether a data block is a region of interest that can be applied to different application scenarios.

[0072] This embodiment provides a video transmission method that can be used in computer devices. Figure 4 This is a flowchart of another video transmission method according to an embodiment of the present invention, such as... Figure 4As shown, the process includes the following steps:

[0073] Step S501: Obtain the image to be transmitted.

[0074] Step S502: Divide the image to be transmitted into multiple data blocks according to the preset block division rules.

[0075] Specifically, the segmentation rules can be determined based on the needs of the upper-layer application. For example, the image to be transmitted can be divided into multiple data blocks in a top-to-bottom order.

[0076] Step S503: When the image to be transmitted is the first frame image, insert the numbers of multiple data blocks into the splay tree.

[0077] Specifically, a splay tree, also called a split tree, is a binary search tree that can perform insertion, search, and deletion operations in O(log n).

[0078] Step S504: Select the current data block to be transmitted from multiple data blocks according to the order in the splay tree.

[0079] Step S505: Transfer the current data block to be transferred to the memory of the second core of the heterogeneous framework.

[0080] Specifically, DMA can be used to transfer the current data block to be transferred to the memory of the second core of the heterogeneous framework. For example, transferring the current data block to be transferred to the memory of the second core of the heterogeneous framework includes: obtaining the number of the current data block to be transferred from the splay tree; obtaining the storage address of the current data block to be transferred based on the number; and transferring the current data block to be transferred to the memory of the second core of the heterogeneous framework based on the storage address. This is because the splay tree stores the data block numbers, so it is necessary to determine the storage address of the current data block to be transferred based on its number before proceeding with the transfer.

[0081] Step S506: Obtain the feature value of the currently transmitted data block sent by the acceleration region of the second core of the heterogeneous framework.

[0082] Step S507: Determine whether the feature value has reached the preset threshold. If the feature value reaches the threshold, proceed to step S508; otherwise, proceed to step S509.

[0083] Step S508: When the feature value reaches the threshold, it is determined that the image to be transmitted has been successfully transmitted, the transmission of the image to be transmitted ends, the number of the current data block to be transmitted in the splay tree is obtained, and the number of the current data block to be transmitted in the splay tree is promoted to the root of the splay tree.

[0084] This is because video stream transmission has the following two characteristics: (1) a data block that has just been transmitted is very likely to be accessed again after the next frame; (2) the next data block to be accessed is very likely to be located near a data block that was accessed not long ago. In the splay tree, when selecting a data block to be transmitted from multiple data blocks according to the order in the splay tree, the selection starts from the root of the tree. Raising the number of the current data block to be transmitted in the splay tree to the root of the splay tree can ensure that the next image to be transmitted can quickly find a data block that reaches the threshold during the transmission process, further reducing the frequency of data block transmission, reducing the overhead and resource consumption of starting transmission, and improving the transmission speed.

[0085] Specifically, a splay tree is a self-adjusting form of binary search tree that moves a node (i.e., the number of the data block to be transferred in the splay tree) to the root of the tree through a series of rotations along the path from a certain node to the root.

[0086] Step S509: When the feature value does not reach the threshold, select the next data block to be transmitted from multiple data blocks according to the order in the splay tree, and take the next data block to be transmitted as the current data block to be transmitted, and return to step S505.

[0087] The video transmission method provided in this invention not only avoids transmitting all data blocks to the memory of the second core of the heterogeneous framework, but also considers the image to be transmitted successfully when the current data block to be transmitted is a region of interest. Furthermore, by inserting the numbers of multiple data blocks into a splay tree and using a splay binary tree to store the numbers of the transmitted blocks, it can ensure that the next image to be transmitted can quickly find the data block that reaches the threshold during the transmission process, further reducing the frequency of transmitting data blocks, reducing the overhead and resource consumption of starting transmission, and improving the transmission speed.

[0088] To provide a detailed explanation of the video transmission method according to an embodiment of the present invention, a specific example is given. Figure 6 This is a flowchart illustrating an example of a video transmission method according to an embodiment of the present invention, such as... Figure 6 As shown, the video transmission method includes the following steps:

[0089] Step S1: The first frame image is divided into blocks according to the needs of the upper layer application, for example, the data blocks are marked in order from top to bottom, and the data block numbers are inserted into the extended binary tree;

[0090] Step S2: Obtain the current data block to be transmitted in the order of the extended binary tree;

[0091] Step S3: Transfer the current data block from the CPU's memory to the GPU's memory via DMA, i.e., send the main memory address where the data block is located according to the binary tree.

[0092] Step S4: Calculate the feature value of the current data block to be transmitted using the GPU's acceleration area, and then return the calculated feature value to the CPU.

[0093] Step S5: If the feature value reaches the threshold, stop the data transfer from CPU to GPU, record the number of the current data block to be transferred, and proceed to step S6; if the feature value does not reach the threshold, obtain the next data block to be transferred in the order of the extended binary tree, use the next data block to be transferred as the current data block to be transferred, and return to step S3.

[0094] Step S6: Obtain the number of the current data block to be transmitted on the extended binary tree, and then the number will climb to the root of the tree;

[0095] Step S7: When the next frame of image arrives, the data is divided into blocks according to the needs of the upper layer application. For example, the data blocks are marked in order from top to bottom, and the current data blocks to be transmitted are obtained according to the order of the updated extended binary tree.

[0096] Step S8: Transfer the currently transmitted data block from the CPU's memory to the GPU's memory via DMA;

[0097] Step S9: Calculate the feature value of the current data block to be transmitted using the GPU's acceleration area, and then return the calculated feature value to the CPU.

[0098] Step S10: If the feature value reaches the threshold, stop the data transmission from CPU to GPU, record the number of the current data block to be transmitted, and proceed to step S11; if the feature value does not reach the threshold, obtain the next data block to be transmitted in the order of the extended binary tree, use the next data block to be transmitted as the current data block to be transmitted, and return to step S8.

[0099] Step S11: Obtain the number of the current data block to be transmitted on the extended binary tree, and then the number will climb to the root of the tree;

[0100] Step S12: Process each subsequent frame of image data according to steps S7 to S11.

[0101] In summary, the embodiments of the present invention propose a method for automatically transferring regions of interest in an image via DMA among heterogeneous frameworks and many cores. Furthermore, by storing the data block numbers containing feature values ​​through a splay tree, the operation of initiating DMA can be greatly reduced, significantly lowering the data transmission startup overhead and resource consumption.

[0102] In other words, the embodiments of the present invention can achieve the following beneficial effects:

[0103] (1) Automatic transfer of regions of interest in an image via DMA between heterogeneous framework multi-cores reduces the frequency of data block transfer, which will reduce the overhead and resource consumption of initiating transfer.

[0104] (2) Use a stretched binary tree to store the number of the transmission block. The data block that has just been transmitted is very likely to be accessed again in the next frame of the image. This will greatly shorten the time when querying the same data or adjacent data.

[0105] (3) An extensible threshold calculation method makes stopping DMA transfer more applicable to different scenarios for different implementation algorithms.

[0106] This embodiment also provides a video transmission device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0107] This embodiment provides a video transmission device applied to the first core of a heterogeneous framework, such as... Figure 7 As shown, it includes:

[0108] The first acquisition module 701 acquires the image to be transmitted;

[0109] The data block partitioning module 702 is used to divide the image to be transmitted into multiple data blocks according to a preset partitioning rule;

[0110] The data block transmission module 703 is used to select the current data block to be transmitted from multiple data blocks and transmit the current data block to be transmitted to the memory of the second core of the heterogeneous framework;

[0111] The judgment module 704 is used to determine whether the current data block to be transmitted is a region of interest;

[0112] The transmission termination module 705 is used to terminate the transmission of the image when the current data block to be transmitted is a region of interest.

[0113] As an optional implementation, the video transmission device also includes an insertion module. Before selecting the current data block to be transmitted from multiple data blocks, when the image to be transmitted is the first frame, the insertion module is used to insert the numbers of multiple data blocks into the stretch tree.

[0114] As an optional implementation, the video transmission device also includes a numbering order adjustment module. When the current data block to be transmitted is a region of interest, the numbering order adjustment module is used to obtain the number of the current data block to be transmitted in the splay tree; and to move the number of the current data block to be transmitted in the splay tree up to the root of the splay tree.

[0115] As an optional implementation, the judgment module 704 is specifically used to: obtain the feature value of the current data block to be transmitted; determine whether the feature value reaches a preset threshold; and when the feature value reaches the threshold, determine that the current data block to be transmitted is a region of interest.

[0116] As an optional implementation, the video transmission device further includes an insertion module and a first loop module. When the current data block to be transmitted is not a region of interest, the first loop module is used to: select the next data block to be transmitted from multiple data blocks, use the next data block as the current data block to be transmitted, and return to the step of transmitting the current data block to be transmitted to the memory of the second core of the heterogeneous framework.

[0117] As an optional implementation, the data block transmission module 703 is specifically used to: determine the number of the current data block to be transmitted; obtain the storage address of the current data block to be transmitted according to the number; and transmit the current data block to be transmitted to the memory of the second core of the heterogeneous framework according to the storage address.

[0118] This embodiment also provides a video transmission device applied to the second core of a heterogeneous framework, such as... Figure 8 As shown, it includes:

[0119] The second acquisition module 801 is used to acquire the current data block to be transmitted from the memory of the second core;

[0120] Feature value calculation module 802 is used to calculate the feature value of the current data block to be transmitted;

[0121] The feature value sending module 803 is used to send feature values ​​to the first core of the heterogeneous framework, so that the first core of the heterogeneous framework can determine whether the data block to be transmitted is a region of interest based on the feature values.

[0122] In this embodiment, the video transmission device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0123] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0124] This invention also provides a computer device having the above-described features. Figure 7 or Figure 8The video transmission device shown.

[0125] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 9 Take a processor 10 as an example.

[0126] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0127] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0128] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0130] The computer device also includes an input device 40 and an output device 50. The processor 10, memory 20, input device 40, and output device 20 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0131] Input device 40 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 50 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0132] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0133] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A video transmission method, characterized in that, The method, applied to the first core of a heterogeneous framework, includes: Acquire the image to be transmitted; The image to be transmitted is divided into multiple data blocks according to a preset block division rule; Select the current data block to be transmitted from the plurality of data blocks, and transmit the current data block to be transmitted to the memory of the second core of the heterogeneous framework; Determine whether the currently transmitted data block is a region of interest; When the current data block to be transmitted is a region of interest, the transmission of the image to be transmitted is terminated.

2. The method according to claim 1, characterized in that, Before selecting the current data block to be transmitted from the plurality of data blocks, the method further includes: When the image to be transmitted is the first frame image, the numbers of the multiple data blocks are inserted into the splay tree.

3. The method according to claim 2, characterized in that, Also includes: When the current data block to be transmitted is a region of interest, obtain the number of the current data block to be transmitted in the splay tree; The number of the currently transmitted data block in the splay tree is promoted to the root of the splay tree.

4. The method according to claim 2 or 3, characterized in that, Selecting the current data block to be transmitted from the plurality of data blocks includes: The current data block to be transmitted is selected from the plurality of data blocks according to the order in the stretch tree.

5. The method according to claim 1, characterized in that, The step of determining whether the currently transmitted data block is a region of interest includes: Obtain the feature value of the currently transmitted data block sent by the second core of the heterogeneous framework; Determine whether the feature value reaches a preset threshold; When the feature value reaches the threshold, the current data block to be transmitted is determined to be a region of interest.

6. The method according to claim 1, characterized in that, Also includes: When the current data block to be transmitted is not the region of interest, the next data block to be transmitted is selected from the plurality of data blocks, and the next data block to be transmitted is used as the current data block to be transmitted. Then, the process of transmitting the current data block to be transmitted to the memory of the second core of the heterogeneous framework is returned.

7. A video transmission method, characterized in that, The method, applied to a second core in a heterogeneous framework, includes: Obtain the current data block to be transferred from the memory of the second core; Calculate the feature value of the current data block to be transmitted; The feature value is sent to the first core of the heterogeneous framework so that the first core of the heterogeneous framework can determine whether the current data block to be transmitted is a region of interest based on the feature value; when the current data block to be transmitted is a region of interest, the transmission of the image to be transmitted is terminated.

8. A video transmission device, characterized in that, The device includes: The first acquisition module acquires the image to be transmitted; The data block partitioning module is used to divide the image to be transmitted into multiple data blocks according to a preset partitioning rule; A data block transmission module is used to select the current data block to be transmitted from the plurality of data blocks and transmit the current data block to be transmitted to the memory of the second core of the heterogeneous framework; The judgment module is used to determine whether the current data block to be transmitted is a region of interest; The transmission termination module is used to terminate the transmission of the image when the current data block to be transmitted is a region of interest. Alternatively, the device may include: The second acquisition module is used to acquire the current data block to be transmitted from the memory of the second core; The feature value calculation module is used to calculate the feature value of the current data block to be transmitted; The feature value sending module is used to send the feature value to the first core of the heterogeneous frame, so that the first core of the heterogeneous frame can determine whether the current data block to be transmitted is a region of interest based on the feature value; when the current data block to be transmitted is a region of interest, the transmission of the image to be transmitted is terminated.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the video transmission method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the video transmission method according to any one of claims 1 to 7.

Images