Image Transmission Method, Apparatus, Device, and Medium

Abstract

The present disclosure relates to an image transmission method, apparatus, device, and medium. The image transmission method includes: determining the transmission time required for the sending end to transmit a frame of an image to be transmitted in each channel according to the data volume of a frame of the image to be transmitted in each channel of the sending end and the transmission bandwidth between the sending end and the receiving end; segmenting the theoretical time corresponding to transmitting a frame of the image to be transmitted according to the transmission time required for the sending end to transmit a frame of the image to be transmitted in each channel, to obtain a time period corresponding to the transmission time; and controlling the sending end to send the image to be transmitted corresponding to the transmission time to the receiving end during the time period corresponding to the transmission time.

Description

Technical Field

[0001] This application relates to the field of image transmission, for example, to an image transmission method, device, equipment, and medium. Background Art

[0002] With the rapid development and iteration of technology, users' requirements for image display clarity are gradually increasing, and they are gradually pursuing ultra-high-definition image display quality. Ultra-high-definition image display means improving the image resolution while ensuring the smoothness of image display, that is, the amount of data transmitted per unit time will increase significantly.

[0003] In the fields of image acquisition, processing, and display technology, Field Programmable Gate Array (FPGA) has been gradually applied in this technical field due to its advantages such as powerful functions, flexibility, and cost economy. In the distributed application scenario of FPGA + Peripheral Component Interconnect Express (PCIE), FPGA acts as an Endpoint (EP) to send or receive data, and the amount of data that can be received or sent per unit time is often limited by the PCIE bandwidth. During the process of image acquisition and transmission with multiple capture cards corresponding to one display card or multiple capture cards corresponding to multiple display cards, due to the increase in the amount of data transmitted by the FPGA at the capture card end per unit time, it often causes instantaneous congestion of PCIE data packets at the display card end, thereby causing image playback jitter.

[0004] To solve the above-mentioned image jitter caused by instantaneous congestion of data packets, there are mainly three technical means commonly adopted: expanding the bandwidth from the hardware, compressing and transmitting the image, or stopping the transmission after congestion occurs. The solution of expanding the bandwidth from the hardware, although it solves the problem of image jitter, increases its hardware cost and reduces the bandwidth utilization rate of PCIE. The solution of compressing and transmitting the image will result in a decrease in image quality. The solution of stopping image transmission after congestion reduces the system resource utilization rate. Summary of the Invention

[0005] This application provides an image transmission method, device, equipment, and medium to achieve the orderly, fast transmission and display of high-resolution images.

[0006] An image transmission method is provided, including:

[0007] Determine the transmission time required for the sending end to transmit one frame of the to-be-transmitted image in each channel according to the amount of data of one frame of the to-be-transmitted image in each channel of the sending end and the transmission bandwidth between the sending end and the receiving end;

[0008] Segment the theoretical time corresponding to transmitting a frame of to-be-transmitted image according to the transmission time required for the transmitting end to transmit a frame of to-be-transmitted image in each channel, to obtain the time period corresponding to the transmission time;

[0009] Control the transmitting end to send the to-be-transmitted image corresponding to the transmission time to the receiving end during the time period corresponding to the transmission time.

[0010] There is also provided an image transmission device, including:

[0011] A transmission time determination module, configured to determine the transmission time required for the transmitting end to transmit a frame of to-be-transmitted image in each channel according to the data volume of a frame of to-be-transmitted image in each channel of the transmitting end and the transmission bandwidth between the transmitting end and the receiving end;

[0012] A segmentation module, configured to segment the theoretical time corresponding to transmitting a frame of to-be-transmitted image according to the transmission time required for the transmitting end to transmit a frame of to-be-transmitted image in each channel, to obtain the time period corresponding to the transmission time;

[0013] A control module, configured to control the transmitting end to send the to-be-transmitted image corresponding to the transmission time to the receiving end during the time period corresponding to the transmission time.

[0014] There is also disclosed an electronic device, including:

[0015] At least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the image transmission method as described above.

[0017] There is also disclosed a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to cause the computer to execute the image transmission method as described above. Description of the Drawings

[0018] Figure 1 is a schematic flowchart of an image transmission method provided by an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of an FPGA+PCIE distributed application system provided by an embodiment of the present application;

[0020] Figure 3 is a schematic flowchart of another image transmission method provided by an embodiment of the present application;

[0021] Figure 4It is a schematic diagram of the time slice misalignment between the sender and the receiver provided by the embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the synchronous pulse signal transmission provided by the embodiment of the present application;

[0023] Figure 6 It is a flowchart of the implementation of an image transmission method provided by the embodiment of the present application;

[0024] Figure 7 It is a schematic diagram of the theoretical data flow of images in a distributed application system of FPGA+PCIE provided by the embodiment of the present application;

[0025] Figure 8 It is a schematic diagram of the allocation of image data A and A';

[0026] Figure 9 It is a schematic diagram of the transmission preemption that occurs when allocating image data A and A';

[0027] Figure 10 It is a schematic diagram of the allocation of image data B and B';

[0028] Figure 11 It is a schematic diagram of the transmission preemption that occurs when allocating image data B and B';

[0029] Figure 12 It is a schematic diagram of the preemption result of the allocated image data;

[0030] Figure 13 It is a schematic diagram of the instantaneous bandwidth overload of the display card provided by the embodiment of the present application;

[0031] Figure 14 It is a schematic diagram of the planned data flow of images in a distributed application system of FPGA+PCIE provided by the embodiment of the present application;

[0032] Figure 15 It is a schematic diagram of the time slice allocation for image data A and A';

[0033] Figure 16 It is a schematic diagram of the time slice allocation for image data B and B';

[0034] Figure 17 It is a schematic diagram of the time slice allocation for image data C and C';

[0035] Figure 18 It is a schematic diagram of the time slice allocation for image data D and D';

[0036] Figure 19 It is a comparison schematic diagram of the instantaneous bandwidth in image transmission provided by an embodiment of the present application;

[0037] Figure 20 It is a schematic structural diagram of an image transmission device provided by an embodiment of the present application;

[0038] Figure 21 It is a block diagram of an electronic device for implementing an image transmission method provided by an embodiment of the present application. Specific embodiments

[0039] The following describes exemplary embodiments of the present application with reference to the accompanying drawings. Among them, various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary.

[0040] In an embodiment of the present application, the sending end can be a node on the side where image data flows out, and the receiving end can be a node on the side where image data flows in. The devices serving as the sending end and the receiving end are not unique. For example, the sending end can be an acquisition card, a server, a single-chip microcomputer, etc., and the receiving end can be a display card, a memory, etc. The data transmission protocol or intermediate device between the sending end and the receiving end depends on the devices serving as the sending end and the receiving end. For example, if the sending end is an acquisition card and the receiving end is a display card, the PCIE bus can be used as the data transmission protocol.

[0041] Figure 1 It is a schematic flowchart of an image transmission method provided by an embodiment of the present application. This embodiment is applicable to the situation where the sending end transmits an image to the receiving end. For example, this embodiment can be applicable to the situation where multiple sending ends transmit images to one receiving end, or multiple sending ends transmit images to multiple receiving ends. The image transmission method disclosed in this embodiment can be executed by an image transmission device, and this device can be implemented in a software and / or hardware manner and can be integrated into an electronic device. Refer to Figure 1 As shown in the figure, the image transmission method provided in this embodiment includes:

[0042] S110. Determine the transmission time required for the sending end to transmit a frame of the image to be transmitted in each channel according to the data volume of a frame of the image to be transmitted in each channel of the sending end and the transmission bandwidth between the sending end and the receiving end.

[0043] The image to be transmitted is an image collected by the sending end and to be sent to the receiving end. Figure 2 It is a schematic diagram of an FPGA+PCIE distributed application system provided by an embodiment of the present application. As shown in Figure 2As shown in the figure, the system may include a main control board, multiple acquisition cards, and multiple display cards. Among them, the structure and form of the main control board are not limited, and it can be a device capable of managing and controlling the acquisition cards and display cards. Multiple acquisition cards can respectively send multiple channels of images to any display card, that is, the display card can receive multiple channels of images sent by multiple acquisition cards. Figure 2 Only 4 channels of images are shown in the figure, which is not a limitation on the number of transmitted images. In fact, any number of channels of images can be transmitted. In this system, when multiple acquisition cards send images to a display card simultaneously, the instantaneous bandwidth of the display card may be overloaded, and the images cannot be received normally. The backpressure is applied to the acquisition cards, and then the transmission is alternated, resulting in a decrease in the transmission frame rate and a stuttering phenomenon when the display card is playing the display.

[0044] In the embodiment of the present application, the main control board manages and controls the image sending process of the acquisition cards, so that multiple acquisition cards send images in an orderly manner when sending images to the same display card, improving the efficiency of image sending. In the embodiment of the present application, in order to clarify the time used by each acquisition card to transmit images and manage and control each acquisition card, all acquisition cards that need to send images to a display card are determined in advance, and the data volume of the images to be transmitted that each acquisition card needs to send to the display card is determined. According to the data volume of the images to be transmitted and the transmission bandwidth, the transmission time required to transmit one frame of the images to be transmitted in each channel can be calculated. The transmission bandwidth is the PCIE transmission bandwidth between the EP ends. In one embodiment, the ratio of the data volume of one frame of the images to be transmitted to the transmission bandwidth is the transmission time required to transmit one frame of the images to be transmitted in each channel.

[0045] The above solution determines in advance the transmission time required to transmit one frame of the images to be transmitted in each channel, and can comprehensively and overall control the transmission time required for multiple acquisition cards to transmit the images to be transmitted, so as to effectively manage and control the image sending process of the acquisition cards according to the transmission time of the images to be transmitted.

[0046] S120. Segment the theoretical time corresponding to transmitting one frame of the images to be transmitted according to the transmission time required for each channel of the images to be transmitted at the sending end, and obtain the time period corresponding to the transmission time.

[0047] For each frame of the images to be transmitted of each acquisition card, there is a corresponding transmission time, which is the time required to transmit this frame of the images to be transmitted. In order to solve the problem of congestion on the display card when multiple acquisition cards send images to the same display card simultaneously, in the embodiment of the present application, for a display card, all acquisition cards that need to send images to the display card are determined, and then all the images to be transmitted that need to be sent to the display card are determined. The transmission times corresponding to all the images to be transmitted are determined, and then the theoretical time corresponding to transmitting one frame of the images to be transmitted is segmented according to the transmission time.

[0048] Exemplarily, the images to be transmitted that the capture card 1 needs to send to the display card 1 are Image 1, Image 2, and Image 3. The transmission time required for transmitting Image 1 is t1, the transmission time required for transmitting Image 2 is t2, and the transmission time required for transmitting Image 3 is t3. The images to be transmitted that the capture card 2 needs to send to the display card 1 are Image 1', Image 2', and Image 3'. The transmission time required for transmitting Image 1' is t4, the transmission time required for transmitting Image 2' is t5, and the transmission time required for transmitting Image 3' is t6. Then, the time corresponding to transmitting one frame of the image to be transmitted can be divided into 6 segments, corresponding to t1, t2, t3, t4, t5, and t6 respectively. That is, the theoretical time corresponding to transmitting one frame of the image to be transmitted is divided into 6 segments: 0 - t1, t1 - (t1 + t2), (t1 + t2) - (t1 + t2 + t3), (t1 + t2 + t3) - (t1 + t2 + t3 + t4), (t1 + t2 + t3 + t4) - (t1 + t2 + t3 + t4 + t5), (t1 + t2 + t3 + t4 + t5) - (t1 + t2 + t3 + t4 + t5 + t6).

[0049] In the embodiments of the present application, there is no limitation on the order of the time periods obtained by segmenting the theoretical time corresponding to transmitting one frame of the image to be transmitted. For example, in the above example, the 6 time periods respectively correspond to t1, t2, t3, t4, t5, and t6. It is also possible to make the 6 time periods obtained by segmentation respectively correspond to t3, t5, t2, t1, t6, and t4, or correspond to the transmission times in other orders, which is not limited herein.

[0050] The above solution segments the theoretical time corresponding to transmitting one frame of the image to be transmitted according to the time required for transmitting one frame of the image to be transmitted in each channel, so as to stagger multiple frames of the image to be transmitted in terms of time periods for orderly management and control.

[0051] S130. Control the sending end to send the image to be transmitted corresponding to the transmission time to the receiving end in the time period corresponding to the transmission time.

[0052] In the embodiments of the present application, when approaching the time period for each capture card to send the image to be transmitted, a control instruction can be sent to the capture card to control the capture card to send the data to be transmitted in the corresponding time period. It is also possible to send control instructions to all capture cards in advance to control the capture cards to send the transmission data when reaching the corresponding time periods. The control instruction can also be actively obtained by the capture card from the main control board.

[0053] Exemplarily, it is illustrated by the example in S120. If the theoretical time corresponding to transmitting one frame of the image to be transmitted is divided into 6 segments, corresponding to t1, t2, t3, t4, t5, and t6 respectively, then control the acquisition card 1 to send Image 1 to the display card 1 in the time period from 0 to t1, control the acquisition card 1 to send Image 2 to the display card 1 in the time period from t1 to (t1 + t2), control the acquisition card 1 to send Image 3 to the display card 1 in the time period from (t1 + t2) to (t1 + t2 + t3), control the acquisition card 2 to send Image 1' to the display card 1 in the time period from (t1 + t2 + t3) to (t1 + t2 + t3 + t4), control the acquisition card 2 to send Image 2' to the display card 1 in the time period from (t1 + t2 + t3 + t4) to (t1 + t2 + t3 + t4 + t5), and control the acquisition card 2 to send Image 3' to the display card 1 in the time period from (t1 + t2 + t3 + t4 + t5) to (t1 + t2 + t3 + t4 + t5 + t6), thereby achieving the orderly and fast transmission of images from multiple acquisition cards to one display card.

[0054] If the theoretical time corresponding to transmitting one frame of the image to be transmitted is divided into 6 segments, corresponding to t3, t5, t2, t1, t6, and t4 respectively, control the acquisition card 1 to send Image 3 to the display card 1 in the time period from 0 to t3, control the acquisition card 2 to send Image 2' to the display card 1 in the time period from t3 to (t3 + t5), control the acquisition card 1 to send Image 2 to the display card 1 in the time period from (t3 + t5) to (t3 + t5 + t2), control the acquisition card 1 to send Image 1 to the display card 1 in the time period from (t3 + t5 + t2) to (t3 + t5 + t2 + t1), control the acquisition card 2 to send Image 3' to the display card 1 in the time period from (t3 + t5 + t2 + t1) to (t3 + t5 + t2 + t1 + t6), and control the acquisition card 2 to send Image 1' to the display card 1 in the time period from (t3 + t5 + t2 + t1 + t6) to (t3 + t5 + t2 + t1 + t6 + t4). The above is only an example, and there can be other situations. The order of segmentation and the order of image transmission can be determined according to the actual situation.

[0055] The above solution enables multiple acquisition cards to send the images to be transmitted to the same display card in a staggered manner, improving the orderliness and transmission speed of sending the images to be transmitted to the display card, and achieving the fast transmission and smooth playback of images on the premise of ensuring image quality.

[0056] The technical solution of the embodiment of the present application segments the theoretical time corresponding to transmitting one frame of the image to be transmitted according to the transmission time required for transmitting one frame of the image to be transmitted in each channel, and controls multiple frames of the images to be transmitted in multiple channels to be transmitted orderly in different time periods, achieving the orderly and fast transmission of high-resolution images on the premise of ensuring image quality.

[0057] Figure 3It is a schematic flowchart of another image transmission method provided by an embodiment of the present application. The embodiment of the present application is an alternative solution proposed on the basis of the above embodiment. Refer to Figure 3 , the image transmission method provided by the embodiment of the present application includes:

[0058] S210. Determine the transmission time required for the sending end to transmit one frame of the image to be transmitted in each channel according to the data volume of one frame of the image to be transmitted in each channel of the sending end and the transmission bandwidth between the sending end and the receiving end.

[0059] S220. Segment the theoretical time corresponding to transmitting one frame of the image to be transmitted according to the transmission time required for the sending end to transmit one frame of the image to be transmitted in each channel, so as to obtain the time period corresponding to the transmission time.

[0060] In the embodiment of the present application, segmenting the theoretical time corresponding to transmitting one frame of the image to be transmitted according to the transmission time required for the sending end to transmit one frame of the image to be transmitted in each channel to obtain the time period corresponding to the transmission time includes: if the transmission times required to respectively transmit one frame of the image to be transmitted in multiple channels are the same, then determine a preset number according to the number of channels of the sending end and / or the ratio of the theoretical time corresponding to transmitting one frame of the image to be transmitted to the transmission time; perform an average segmentation on the theoretical time corresponding to transmitting one frame of the image to be transmitted according to the preset number to obtain a preset number of time periods. Determining the preset number according to the number of channels of the sending end and / or the ratio of the theoretical time corresponding to transmitting one frame of the image to be transmitted to the transmission time includes: setting the preset number to be equal to the number of channels of the sending end; or setting the preset number to be equal to the ratio of the theoretical time corresponding to transmitting one frame of the image to be transmitted to the transmission time; or setting the preset number to any value between the number of channels of the sending end and the ratio.

[0061] Exemplarily, assume that capture card 1 and capture card n respectively capture and send 4 YUV422-format images to be transmitted with a resolution of 1920*1200 to display card 1 and display card n. Each capture card has 4 outputs of images to be transmitted, and each display card has 4 inputs of images to be transmitted. Therefore, as long as it is ensured that within the theoretical time corresponding to transmitting one frame of the image to be transmitted, the transmission of 4 frames of images to be transmitted corresponding to 4 channels respectively can be completed, the theoretical time corresponding to transmitting one frame of the image to be transmitted can be evenly divided into 4 segments. The PCIE bandwidth between the capture card and the display card is X2. According to the PCIE bandwidth utilization rate, its actual effective bandwidth is 6.6 Gbps, that is, the bandwidth threshold between the sending end and the receiving end is 6.6 Gbps. The image data is transmitted between the capture card and the display card at a rate of 30 frames per second. It can be calculated that the bandwidth occupied by one image to be transmitted is 1920*1200*16*30 / 1000 / 1000 / 1000 = 1.1 Gbps. For the capture card and the display card, within the theoretical time (T = 1 / 30 s) corresponding to transmitting one frame of the image to be transmitted, 6 frames of images to be transmitted corresponding to 6 channels can be sent and received. The theoretical time corresponding to transmitting one frame of the image to be transmitted can be evenly divided into 6 segments to meet the transmission time requirements of each image to be transmitted. It is also possible to make the preset quantity between 4 and 6. For example, the preset quantity is 5, and the theoretical time corresponding to transmitting one frame of the image to be transmitted is evenly divided into 5 segments.

[0062] S230. Determine the images to be transmitted transmitted within the transmission time corresponding to each time period.

[0063] Exemplarily, segment the theoretical time corresponding to transmitting one frame of the image to be transmitted according to the transmission time. Each time segment is actually corresponding to the transmission time. The transmission time is the time required to transmit one frame of the image to be transmitted. Each transmission time is also corresponding to the image to be transmitted. Therefore, the images to be transmitted corresponding to each time period can be determined.

[0064] If the theoretical time for transmitting a frame of the image to be transmitted is divided into six segments, namely 0 - t1, t1 - (t1 + t2), (t1 + t2) - (t1 + t2 + t3), (t1 + t2 + t3) - (t1 + t2 + t3 + t4), (t1 + t2 + t3 + t4) - (t1 + t2 + t3 + t4 + t5), (t1 + t2 + t3 + t4 + t5) - (t1 + t2 + t3 + t4 + t5 + t6), corresponding to t1, t2, t3, t4, t5, and t6 respectively. t1 is the time required to transmit Image 1 and corresponds to Image 1. t2 is the time required to transmit Image 2 and corresponds to Image 2. t3 is the time required to transmit Image 3 and corresponds to Image 3. t4 is the time required to transmit Image 1' and corresponds to Image 1'. t5 is the time required to transmit Image 2' and corresponds to Image 2'. t6 is the time required to transmit Image 3' and corresponds to Image 3'. Thus, with the transmission time as the link, the image to be transmitted corresponding to each time period is determined.

[0065] S240. Establish the corresponding relationship among the time period, the image to be transmitted, the sending end and the receiving end of the image to be transmitted.

[0066] In the embodiment of the present application, the corresponding relationship is established so that the sending end sends the image to be transmitted according to the corresponding relationship. The process of establishing the corresponding relationship can be executed before all images start to be transmitted, or during the image transmission process, when it is detected that there is a capture card that needs to send a new image to be transmitted to the display card, the step of establishing the corresponding relationship is executed again.

[0067] The above example is illustrated. Establish the corresponding relationship of time period: 0 - t1, image to be transmitted: Image 1, sending end: Capture Card 1, receiving end: Display Card 1, so that Capture Card 1 sends Image 1 to Display Card 1 within the time period 0 - t1 according to the corresponding relationship. Establish the corresponding relationship of time period: t1 - (t1 + t2), image to be transmitted: Image 2, sending end: Capture Card 1, receiving end: Display Card 1, so that Capture Card 1 sends Image 2 to Display Card 1 within the time period t1 - (t1 + t2) according to the corresponding relationship, and so on. Establish the corresponding relationship among each time period, the corresponding image to be transmitted, the sending end and the receiving end of the image to be transmitted to instruct the capture card to send the images to be transmitted in an orderly manner.

[0068] The above solution improves the image transmission efficiency by pre - establishing the corresponding relationship, so that the capture card performs orderly image transmission, while ensuring the image quality and without changing the hardware structure.

[0069] In an embodiment of the present application, if there are at least two sending ends, establishing the corresponding relationship among the time period, the image to be transmitted, the sending end of the image to be transmitted, and the receiving end includes: traversing multiple time periods starting from the starting moment to determine whether the receiving end is assigned to establish the corresponding relationship in the current time period; if the receiving end is not assigned to establish the corresponding relationship in the current time period, searching for the sending end that is not assigned to establish the corresponding relationship in the current time period, and establishing the corresponding relationship among the current time period, the sending end, the image to be transmitted to be assigned by the sending end, and the receiving end.

[0070] Exemplarily, the transmission times corresponding to multiple time periods may be different or the same. If the transmission times required for at least two images to be transmitted are the same during transmission, there may be at least two images to be transmitted corresponding to the same time period. In an embodiment of the present application, the corresponding relationship can also be established according to the allocation situation of the sending end and the receiving end in each time period. Exemplarily, in the time period from 0 to T1, it corresponds to "image to be transmitted 1" and "image to be transmitted 2". "Image to be transmitted 1" is to be sent by capture card 1, and "image to be transmitted 2" is to be sent by capture card 2. In the time period from 0 to T1, display card 1 is not assigned to establish the corresponding relationship, and capture card 1 and capture card 2 are also not assigned to establish the corresponding relationship. Capture card 1 can be selected to establish the corresponding relationship among the time period from 0 to T1, "image to be transmitted 1", capture card 1, and display card 1. The time period from 0 to T1 of display card 1 has been assigned to establish the corresponding relationship. In the time period from T1 to 2T1 of display card 1, it is not assigned to establish the corresponding relationship. Therefore, the corresponding relationship among the time period from T1 to 2T1, "image to be transmitted 2", capture card 2, and display card 1 is established.

[0071] Similarly for display card 2, display card 2 is not assigned to establish the corresponding relationship in the time period from 0 to T1, capture card 1 has been assigned to establish the corresponding relationship in the time period from 0 to T1, and capture card 2 is not assigned to establish the corresponding relationship in the time period from 0 to T1. Therefore, the corresponding relationship among the time period from 0 to T1, capture card 2, "image to be transmitted 3", and display card 2 is established. In the time period from T1 to 2T1, display card 2 is not assigned to establish the corresponding relationship, capture card 2 has been assigned to establish the corresponding relationship in the time period from T1 to 2T1, and capture card 1 is not assigned to establish the corresponding relationship in the time period from T1 to 2T1. Therefore, the corresponding relationship among the time period from T1 to 2T1, capture card 1, "image to be transmitted 4", and display card 2 is established. And so on, until the corresponding relationships among all the images to be transmitted, the capture cards sending the images to be transmitted, and the corresponding display cards are established, or the theoretical time corresponding to transmitting one image to be transmitted is all assigned to establish the corresponding relationships with the capture card and the display card.

[0072] The above solution can accurately and efficiently establish the corresponding relationship among the time period, the image to be transmitted, the sending end, and the receiving end, so that the capture card can send images orderly according to the corresponding relationship.

[0073] S250 sends a synchronization pulse signal to the sending end and the receiving end at a period of the theoretical time corresponding to transmitting one frame of the image to be transmitted.

[0074] Exemplarily, by sending a synchronization pulse signal to the sending end and the receiving end, it is used to instruct the sending end to start sending the image to be transmitted, and to instruct the receiving end to synchronously start receiving the image to be transmitted.

[0075] Since the start time points of data sending or receiving between different sending ends and receiving ends are not necessarily unified, there may be a situation where the time slices between multiple devices do not correspond. As Figure 4 shown, the time slice t0 of the sending end n corresponds to the time slice t1 of the sending end 1, and the corresponding time slice of the receiving end n is t2, resulting in the disorder of image data transmission and the randomness of congestion.

[0076] As Figure 5 shown, in order to achieve the synchronization of the time slices of the acquisition card and the display card, the main control board sends a synchronization pulse signal to the sending end and the receiving end at a period of the theoretical time corresponding to transmitting one frame of the image to be transmitted. After all the FPGAs in the current system receive this pulse signal, they synchronously start sending and receiving data, ensuring that the time slices are absolutely synchronized when the acquisition card and the display card send and receive each frame of image data.

[0077] S260 controls the sending end to send the image to be transmitted corresponding to the transmission time to the receiving end in the time period corresponding to the transmission time.

[0078] In the embodiment of the present application, by pre - establishing a corresponding relationship, the acquisition card performs orderly image transmission according to the corresponding relationship, improving the image transmission efficiency while ensuring the image quality and without changing the hardware structure.

[0079] The image transmission method of the present application is implemented in a distributed application system of FPGA + PCIE. The FPGA acts as the EP end to send or receive data, and the amount of data that can be received or sent per unit time is limited by the PCIE bandwidth. In this embodiment, the acquisition card is used as the sending end to send data on the FPGA, and the display card is used as the receiving end to receive data on the FPGA. The acquisition card can simultaneously acquire multiple channels of image data or simultaneously send multiple channels of image data.

[0080] The present application cuts the theoretical time (T) corresponding to transmitting one frame of an image into multiple time equal - parts according to the size of one frame of the transmitted image in each channel, and uniformly manages and allocates all time slices. For the acquisition card, a time slice needs to be allocated to it before sending image data so that it can send image data in the time period corresponding to this time slice.

[0081] In the distributed application system of FPGA + PCIE, the display card receives data passively, and the acquisition card sends data actively. Therefore, on the data sending side, that is, the acquisition card side, only by controlling the sending time slice sequence of the image data of the acquisition card within one frame can the problem of peak bandwidth overload in the display card receiving data be solved. Taking the process of allocating time slices for a frame of image data to be sent in one channel of an acquisition card as an example to illustrate the process of allocating time slices.

[0082] As Figure 6 shown, first, the flag bit x (0 ≤ x ≤ m) of the time slice is initialized to be the same as the initial number. Since the numbers of the time slices in this embodiment are t0, t1, t2, …, t m-2 、t m-1 、t m , so x is initialized to 0.

[0083] Judge whether the display card is congested within the time slice numbered t x . This display card is the one that receives the image data to be sent by the acquisition card. The method for judging whether it is congested is: obtain the bandwidth value of the image data normally transmitted on the time slice numbered t x of the display card (i.e., the existing bandwidth value), and add the bandwidth value of the image data that the acquisition card is about to send. If the sum of the two is greater than the actual effective bandwidth of the PCIE (i.e., the bandwidth threshold between the display card and the acquisition card), it is regarded as congested; if the sum of the two is not greater than the actual effective bandwidth of the PCIE, it is regarded as not congested. If it is congested, continue to traverse the next time slice of the display card.

[0084] If all time slices have been traversed and no non-congested time slice is found, it means that the current display card has reached the actual effective bandwidth of the PCIE, and this allocation operation is stopped.

[0085] If a time slice t x is found to meet the non-congested condition and this time slice on the acquisition card is not occupied by other services, then use this time slice. And use the sum of the existing bandwidth value of this time slice and the bandwidth value of the image data that the current acquisition card is about to send as the new existing bandwidth value.

[0086] If a time slice t x is found to meet the non-congested condition but this time slice on the acquisition card is occupied by other services, then continue to traverse the next time slice of the acquisition card. Through the above steps, the time slice for the acquisition card to send the current image data to be sent can be obtained.

[0087] This embodiment proposes a method of time slicing, and a method of controlling the transmission time of the image data sender within the theoretical time corresponding to transmitting a frame of image data according to the congestion situation of each time slice, effectively alleviating the PCIE bandwidth congestion situation at the data receiver end caused by concurrency. And this solution greatly improves the bandwidth utilization rate of PCIE while maintaining the image quality.

[0088] Based on the FPGA+PCIE distributed application system, this embodiment of the present application makes a comparative description of the image transmission method and the image transmission method proposed in this application.

[0089] As Figure 7 shown, in the FPGA+PCIE distributed application system, it is assumed that acquisition card 1 and acquisition card n respectively acquire and send 4-channel YUV422 format and 1920*1200 resolution image data to display card 1 and display card n. Each acquisition card has 4-channel image data output, and each display card has 4-channel image data input. The PCIE bandwidth between the acquisition card and the display card is X2. According to the PCIE bandwidth utilization rate, its actual effective bandwidth is 6.6 Gbps, that is, the bandwidth threshold between the sender and the receiver is 6.6 Gbps.

[0090] The image data is transmitted between the acquisition card and the display card at a rate of 30 frames per second. From this, the bandwidth occupied by one channel of image data can be calculated as 1920*1200*16*30 / 1000 / 1000 / 1000 = 1.1 Gbps. For the acquisition card and the display card, within the theoretical time (T = 1 / 30 s) corresponding to transmitting a frame of image data, 6 frames of image data corresponding to 6 channels can be sent and received.

[0091] The theoretical time (T) corresponding to transmitting a frame of image data is evenly divided into 6 equal parts for image data transmission. The image data sent by acquisition card 1 is A, B, C, D. The images sent by acquisition card n are A', B', C', D'. Among them, the image data A, B, A', B' are sent to display card n, and the image data C, D, C', D' are sent to display card 1.

[0092] In this distributed application scenario, there is a situation where multiple acquisition cards send image data to one display card at the same time. Acquisition card 1 and acquisition card n may send image data to the same display card at the same time, resulting in instantaneous PCIE bandwidth overload at the display card, and the data is back-pressured to the acquisition card, and then the acquisition card alternately sends image data to the display card. The image data that the acquisition card can theoretically send in time t actually needs 2t time to complete the transmission, and the transmission frame rate decreases, which in turn causes playback jitter.

[0093] For Figure 7 the FPGA+PCIE distributed application scenario set in, a transmission method is as follows:

[0094] As Figure 8 shown, when the A image data of capture card 1 and the A' image data of capture card n are sent to display card n simultaneously, it causes the image data A and the image data A' to preempt at the 0 to (1 / T) moment on display card n.

[0095] As Figure 9 shown, the image data A and A' are sent simultaneously, resulting in the image data A and the image data A' being back - pressured to the capture card, and the data packets of the image data A and the image data A' are sent alternately, occupying a total time of (2 / T).

[0096] As Figure 10 shown, starting from the 2 / T moment, the image data B and the image data B' are sent. At this time, these two image data will also preempt at the (2 / T) to (3 / T) moment on display card n.

[0097] As Figure 11 shown, the result of the preemption of the image data B and the image data B' causes the image data B and the image data B' to be back - pressured to the capture card, and the data packets of the image data B and the image data B' are sent alternately, occupying a total time of (2 / T) to (4 / T).

[0098] As Figure 12 shown, similar to the image data C and the image data C', the image data D and the image data D' will all have preemption, and the final preemption result is that the transmission time is prolonged.

[0099] In this embodiment, the image data D and the image data D' should have been sent within the time T, but because the peak bandwidth of the display card is overloaded, the transmission cannot be completed. At this time, due to the PCIE packet - sending mechanism, it will take a single frame of time to send the image data D and the image data D' completely. For the transmission process of the image data A, B, C, D and the image data A', B', C', D', it should have been completed within one - frame time T, but actually 2T time is used, and the total frame rate is 30 / 2 = 15 frames. The reduction of the frame rate causes image stuttering.

[0100] As Figure 13 shown, the instantaneous bandwidth of the display card is overloaded, that is, within the same time period, the total data volume of the image data sent by multiple capture cards on the display card exceeds the actual effective bandwidth of PCIE, resulting in the instantaneous bandwidth overload of the display card.

[0101] Theoretically, for the display card, the PCIE bandwidth for 4 - channel image data inflow is 4 * 1920 * 1200 * 16 * 30 / 1000 / 1000 / 1000 = 4.4 Gbps, and this value is significantly less than 6.6 Gbps. However, due to the instantaneous bandwidth overload of the display card, it causes the capture card to be back - pressured, thereby causing playback stuttering.

[0102] Assume that the current actual effective bandwidth of PCIE is X (Gbps) (where (Gbps) is the unit of PCIE transmission bandwidth). The data volume corresponding to one frame of image data is Q. Then, within the theoretical time (time T) for transmitting one frame of image data, the received data bandwidth is:

[0103]

[0104] In formula (1), when Q remains unchanged, the smaller T is, the larger the value of Q / T. In Figure 13 , t < T. Therefore, since Q / t > X (Gbps), it causes peak bandwidth overload. The peak bandwidth overload causes the data sender to preempt bandwidth, resulting in backpressure. After backpressure, the data sending frame rate decreases, which in turn causes playback stuttering.

[0105] Therefore, in the embodiments of the present application, one frame of time (T) is cut into multiple time equal parts according to the size of one frame of transmitted image in each channel, and the time slices are uniformly managed and allocated.

[0106] As shown in formula (1), the actual time size occupied by the sliced time slice t x during the transmission of one frame of image is the ratio of the total data volume of the image data to be transmitted corresponding to the current time slice to the actual effective bandwidth X (Gbps) of PCIE. If the total data volume corresponding to time slice t0 is Q0, then the actual time used for time slice t0 is Q0 / X.

[0107] When the number of slices of the theoretical time for transmitting one frame of image data is p and image data is allocated in each time slice, then T = Q0 / X + Q1 / X +... + Q p / X.

[0108] According to the setting of the data transmission specification in the embodiments of the present application, the sizes of all transmitted images are 1920*1200, and the PCIE bandwidth is x2. According to the PCIE bandwidth utilization rate, the actual effective bandwidth X is 6.6 Gbps. Transmitting at a rate of 30 frames per second, one frame of time T is 1 / 30 (s) = 33 ms. The time slice t x corresponding to transmitting one frame of image in one channel x = 1920*1200*16 / 1000 / 1000 / 1000 = 0.037 Gb. From the above, t x = 0.037 / 6.6 = 5.5 (ms). Then, the number of effective slices after slicing is 33 / 5.5 = 6.

[0109] According to the slicing method of the embodiments of the present application, it can be known that the actual time of each time slice will change according to the size of the transmitted image. The default number of time slices is allocated before image transmission, but the effective number of slices is less than or equal to the default number of time slices during actual image transmission. In the application scenario of this embodiment, 32 time slices are allocated for it, but actually only 6 time slices are effective.

[0110] The effective time slices are also calculated in real time according to data allocation, that is, in actual applications, the effective number of slices cannot be determined before data allocation starts. If the amount of data sent by the acquisition card becomes larger or smaller, the effective number of slices will also change accordingly. In this embodiment, the data transmission specification is limited in advance, so the effective slice data is calculated to help understanding.

[0111] For Figure 7 the distributed application scenario of FPGA+PCIE set in

[0112] By default, the theoretical time T corresponding to transmitting one frame of image data is cut into multiple time slices. For example, it can be 32, but due to the same size of the transmitted image, the effective number of time slices cut within the theoretical time T corresponding to transmitting one frame of image data is 6. As Figure 14 shown, the images sent by acquisition card 1 are A, B, C, D, and the images sent by acquisition card n are A', B', C', D'. Among them, images A, B, A', B' are sent to display card n, and images C, D, C', D' are sent to display card 1.

[0113] As Figure 15 shown, when the image A' of acquisition card n sends data to display card n, the time slices are allocated first according to the congestion situation of the time slices of display card n. When image A' is sent, the time slice t0 of display card n is not congested and the time slice t0 of acquisition card n is not occupied, so the sending time slice t0 is allocated for it; when the image A of acquisition card 1 is sent to display card n, the time slice t0 of display card n is congested, the time slice t1 of display card n is not congested and the time slice t1 of acquisition card 1 is not occupied, so the time slice t1 is allocated for it.

[0114] As Figure 16 shown, when the image B' of acquisition card n is sent, the time slices t0 and t1 of display card n are both congested, the time slice t2 of display card n is not congested and the time slice t2 of acquisition card n is not occupied, so the sending time slice t2 is allocated for it; when the image B of acquisition card 1 is sent to display card n, the time slices t0, t1 and t2 of display card n are all congested, the time slice t3 of display card n is not congested and the time slice t3 of acquisition card 1 is not occupied, so the time slice t3 is allocated for it.

[0115] As Figure 17As shown in the figure, when the image C' of the capture card n sends data to the display card 1, the time slice is allocated according to the congestion situation of the time slice of the display card 1. When the image C' is sent, the time slice t0 of the display card 1 is not congested, but the time slice t0 of the capture card n has been occupied. At this time, the time slice t0 of the capture card 1 is not occupied. Then, for the image C on the capture card 1, the time slice t0 of the display card 1 and the time slice t0 of the capture card 1 are allocated to it; when continuing to allocate the image C' of the capture card n, the time slice t1 of the display card 1 is not congested and the time slice t1 of the capture card n is not occupied, and the transmission time slice t1 is allocated to it.

[0116] As Figure 18 shown in the figure, when the image D' of the capture card n sends data to the display card 1, the time slice is allocated according to the congestion situation of the time slice of the display card 1. When the image D' is sent, the time slice t2 of the display card 1 is not congested, but the time slice t2 of the capture card n has been occupied. At this time, the time slice t2 of the capture card 1 is not occupied. Then, the time slice t2 of the display card 1 and the time slice t2 of the capture card 1 are first allocated to the image D to be allocated on the capture card 1; when continuing to allocate the image D' of the capture card n, the time slice t3 of the display card 1 is not congested and the time slice t3 of the capture card n is not occupied, and the transmission time slice t3 is allocated to it.

[0117] After the allocation is completed, the image data transmitted corresponding to the time slice is as Figure 18 shown in the figure. At this time, the actual number of slices of the capture card and the display card is 4, and the actual time of each time slice is calculated according to the total data volume of the image data in the time slice. For example, the actual time of the time slice t0 of the capture card 1 is: the data volume of the image C divided by the actual effective bandwidth of the current PCIE.

[0118] As Figure 19 shown in the figure, Figure 19 The left shows the schematic diagram of the instantaneous bandwidth of data transmission in the related art, Figure 19 The right shows the schematic diagram of the instantaneous bandwidth of data transmission in the present application. It can be seen from the figure that the image transmission method of the present application effectively alleviates the image stuttering problem caused by the overload of the PCIE instantaneous bandwidth in the left figure Figure 19 under the condition that the total data volume of a frame of image data remains unchanged, and effectively improves the utilization rate of the PCIE bandwidth.

[0119] Figure 20 This is the schematic structural diagram of an image transmission device provided by an embodiment of the present application. Refer to Figure 20 , an embodiment of the present application discloses an image transmission device 300, and the device 300 includes: a transmission time determination module 301, a segmentation module 302, and a control module 303.

[0120] The transmission time determination module 301 is configured to determine the transmission time required for the sending end to transmit a frame of the to-be-transmitted image in each channel according to the data volume of a frame of the to-be-transmitted image in each channel of the sending end and the transmission bandwidth between the sending end and the receiving end; the segmentation module 302 is configured to segment the theoretical time corresponding to transmitting a frame of the to-be-transmitted image according to the transmission time required for the sending end to transmit a frame of the to-be-transmitted image in each channel, so as to obtain the time period corresponding to the transmission time; the control module 303 is configured to control the sending end to send the to-be-transmitted image corresponding to the transmission time to the receiving end during the time period corresponding to the transmission time.

[0121] In an embodiment of the present application, the segmentation module 302 includes:

[0122] The preset quantity determination unit is configured to, if the transmission times required for respectively transmitting a frame of the to-be-transmitted image in multiple channels are the same, determine the preset quantity according to the number of channels of the sending end and / or the ratio of the theoretical time corresponding to transmitting a frame of the to-be-transmitted image to the transmission time; the time period segmentation unit is configured to evenly segment the theoretical time corresponding to transmitting a frame of the to-be-transmitted image according to the preset quantity, so as to obtain a preset number of time periods.

[0123] In an embodiment of the present application, the preset quantity determination unit is configured to:

[0124] Set the preset quantity to be equal to the number of channels of the sending end; or, set the preset quantity to be equal to the ratio of the theoretical time corresponding to transmitting a frame of the to-be-transmitted image to the transmission time; or, set the preset quantity to be any value between the number of channels of the sending end and the ratio.

[0125] In an embodiment of the present application, if there are at least two sending ends, the segmentation module 302 includes:

[0126] The alternating segmentation unit is configured to alternately segment the theoretical time corresponding to transmitting a frame of the to-be-transmitted image according to the transmission time required for at least two sending ends to transmit a frame of the to-be-transmitted image, so as to obtain the time period corresponding to the transmission time.

[0127] In an embodiment of the present application, the device further includes:

[0128] The to-be-transmitted image determination module is configured to determine the to-be-transmitted image transmitted during the transmission time corresponding to each time period; the correspondence relationship establishment module is configured to establish the correspondence relationship among the time period, the to-be-transmitted image, the sending end and the receiving end of the to-be-transmitted image, so that the sending end sends the to-be-transmitted image according to the correspondence relationship.

[0129] In the embodiments of the present application, if there are at least two sending ends, the corresponding relationship establishing module includes:

[0130] A traversing unit, configured to traverse multiple time periods starting from the starting moment to determine whether the receiving end is assigned to establish a corresponding relationship in the current time period; a relationship establishing unit, configured to, if the receiving end is not assigned to establish a corresponding relationship in the current time period, find the sending end that is not assigned to establish a corresponding relationship in the current time period, and establish the corresponding relationship among the current time period, the sending end, the image to be transmitted to be assigned by the sending end, and the receiving end.

[0131] In the embodiments of the present application, the control module 303 is configured to:

[0132] Taking the theoretical time corresponding to transmitting one frame of the image to be transmitted as a period, send a synchronization pulse signal to the sending end and the receiving end to instruct the sending end to start sending the image to be transmitted, and instruct the receiving end to synchronously start receiving the image to be transmitted.

[0133] The image transmission device provided in the embodiments of the present application can execute the image transmission method provided in any embodiment of the present application, and has the corresponding function modules and effects for executing the method.

[0134] According to the embodiments of the present application, the present application also provides an electronic device and a readable storage medium.

[0135] As Figure 21 shown, Figure 21 is a block diagram of an electronic device for implementing the image transmission method provided in the embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0136] As Figure 21As shown, the electronic device includes: one or more processors 401, a memory 402, and an interface configured to connect multiple components, including a high-speed interface and a low-speed interface. The multiple components are interconnected using different buses and can be mounted on a common motherboard or otherwise mounted as needed. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of a graphical user interface (GUI) on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories if needed. Similarly, multiple electronic devices can be connected, and the multiple devices provide some necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 21 In this case, one processor 401 is taken as an example.

[0137] The memory 402 is the non-transitory computer-readable storage medium provided in this application. Among them, the memory stores instructions executable by at least one processor, so that the at least one processor executes the image transmission method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions, and the computer instructions are used to cause the computer to execute the image transmission method provided in this application.

[0138] The memory 402, as a non-transitory computer-readable storage medium, can be configured to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the image transmission method in the embodiments of this application (for example, Figure 20 the transmission time determination module 301, the segmentation module 302, and the control module 303 shown). The processor 401 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 402, that is, implements the image transmission method in the above method embodiments.

[0139] The memory 402 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created according to the use of the electronic device for image transmission, etc. In addition, the memory 402 may include high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 402 may optionally include a memory remotely disposed relative to the processor 401, and these remote memories may be connected to the image transmission electronic device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0140] The electronic device of the image transmission method may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403, and the output device 404 may be connected through a bus or other means. Figure 21 Taking the connection through the bus as an example.

[0141] The input device 403 may receive input digital or character information and generate key signal inputs related to the user settings and function controls of the electronic device for image transmission, such as input devices like a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 404 may include a display device, an auxiliary lighting device (e.g., a Light Emitting Diode (LED)), and a haptic feedback device (e.g., a vibration motor), etc. The display device may include but is not limited to a Liquid Crystal Display (LCD), an LED display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0142] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuit systems, integrated circuit systems, Application Specific Integrated Circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] These computing procedures (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can implement these computing procedures using high-level procedures and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a Programmable Logic Device (PLD)) configured to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal configured to provide machine instructions and / or data to a programmable processor.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or an LCD monitor) configured to display information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be configured to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), a blockchain network, and the Internet.

[0146] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other.

[0147] The various forms of processes shown above can be used to reorder, add, or delete steps. For example, the multiple steps described in this application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in this application can be achieved, and no limitations are imposed herein.

Claims

1. An image transmission method, comprising: Determining the transmission time required for the sending end to transmit a frame of an image to be transmitted in each channel according to the data volume of a frame of the image to be transmitted in each channel of the sending end and the transmission bandwidth between the sending end and the receiving end; Segmenting the theoretical time corresponding to transmitting a frame of the image to be transmitted according to the transmission time required for the sending end to transmit a frame of the image to be transmitted in each channel, to obtain time periods corresponding to the transmission time, including: In the case where the transmission times required for transmitting a frame of the image to be transmitted in multiple channels are the same, determining a preset number according to at least one of the number of channels of the sending end and the ratio of the theoretical time corresponding to transmitting a frame of the image to be transmitted to the transmission time; Averagely segmenting the theoretical time corresponding to transmitting a frame of the image to be transmitted according to the preset number to obtain the preset number of time periods; The determining the preset number according to at least one of the number of channels of the sending end and the ratio of the theoretical time corresponding to transmitting a frame of the image to be transmitted to the transmission time includes one of the following: setting the preset number to be equal to the number of channels of the sending end; setting the preset number to be equal to the ratio of the theoretical time corresponding to transmitting a frame of the image to be transmitted to the transmission time; setting the preset number to be a value between the number of channels of the sending end and the ratio; Controlling the sending end to send the image to be transmitted corresponding to the transmission time to the receiving end during the time period corresponding to the transmission time, including: Sending a synchronization pulse signal to the sending end and the receiving end with the theoretical time corresponding to transmitting a frame of the image to be transmitted as a period, to instruct the sending end to start sending the image to be transmitted and to instruct the receiving end to synchronously start receiving the image to be transmitted.

2. The method according to claim 1, wherein, In the case where there are at least two sending ends, the segmenting the theoretical time corresponding to transmitting a frame of the image to be transmitted according to the transmission time required for the sending end to transmit a frame of the image to be transmitted in each channel, to obtain time periods corresponding to the transmission time, includes: Alternately segmenting the theoretical time corresponding to transmitting a frame of the image to be transmitted according to the transmission times required for at least two sending ends to transmit a frame of the image to be transmitted, to obtain time periods corresponding to the transmission time.

3. The method according to claim 1, before the controlling the sending end to send the image to be transmitted corresponding to the transmission time to the receiving end during the time period corresponding to the transmission time, further comprising: Determining the image to be transmitted transmitted during the transmission time corresponding to each time period; Establishing a correspondence relationship among the time period, the image to be transmitted, the sending end of the image to be transmitted, and the receiving end of the image to be transmitted, so that the sending end sends the image to be transmitted according to the correspondence relationship.

4. The method according to claim 3, wherein, In the case where there are at least two sending ends, the establishing the correspondence relationship among the time period, the image to be transmitted, the sending end of the image to be transmitted, and the receiving end of the image to be transmitted, includes: Traverse multiple time periods starting from the starting moment to determine whether the receiver is assigned to establish a corresponding relationship in the current time period; In the case that the receiver is not assigned to establish a corresponding relationship in the current time period, find a sender that is not assigned to establish a corresponding relationship in the current time period, and establish the corresponding relationship among the current time period, the sender, the image to be transmitted that the sender is to be assigned, and the receiver.

5. An image transmission device, comprising: A transmission time determination module, configured to determine the transmission time required for the sender to transmit an image to be transmitted in each channel according to the data volume of an image to be transmitted in each channel of the sender and the transmission bandwidth between the sender and the receiver; A segmentation module, configured to segment the theoretical time corresponding to transmitting an image to be transmitted according to the transmission time required for the sender to transmit an image to be transmitted in each channel, to obtain time periods corresponding to the transmission time; A control module, configured to control the sender to send the image to be transmitted corresponding to the transmission time to the receiver in the time period corresponding to the transmission time; The segmentation module includes: A preset quantity determination unit, configured to determine a preset quantity according to at least one of the number of channels of the sender and the ratio of the theoretical time corresponding to transmitting an image to be transmitted to the transmission time when the transmission times required for transmitting an image to be transmitted in multiple channels are the same; A time period splitting unit, configured to evenly segment the theoretical time corresponding to transmitting an image to be transmitted according to the preset quantity to obtain a preset number of time periods; The preset quantity determination unit is specifically configured to: set the preset quantity to be equal to the number of channels of the sender; or set the preset quantity to be equal to the ratio of the theoretical time corresponding to transmitting an image to be transmitted to the transmission time; or set the preset quantity to be a value between the number of channels of the sender and the ratio; The control module is specifically configured to: Taking the theoretical time corresponding to transmitting an image to be transmitted as a period, send a synchronization pulse signal to the sender and the receiver to instruct the sender to start sending the image to be transmitted and instruct the receiver to synchronously start receiving the image to be transmitted.

6. An electronic device, comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the image transmission method according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause the computer to execute the image transmission method according to any one of claims 1-4.

Images