An image streaming method, device, electronic equipment and storage medium

Abstract

The application discloses an image streaming method and device, electronic equipment and storage medium, and belongs to the technical field of image processing, which comprises the following steps: receiving original image data output by a to-be-tested chip; storing the original image data as to-be-transmitted data according to a storage format supported by an ARM chip; the to-be-transmitted data contains redundant data generated in the storage process; determining a plurality of data packets to be transmitted in the to-be-transmitted data according to a data packet division rule; creating two parallel tasks for the transmission of each data packet in the to-be-transmitted data, one of which is used to delete the redundant data of each data packet in the to-be-transmitted data, and the other is used to send each data packet after the deletion of the redundant data to an upper computer, so as to realize the streaming transmission of each data packet in the to-be-transmitted data. The application solves the problem that the transmission thread is long-time vacant in the process of deleting redundant data in the prior art, improves the transmission efficiency and the utilization rate of the transmission bandwidth, and reduces the transmission delay.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically, relates to an image streaming method, apparatus, electronic device, and storage medium. Background Technology

[0002] For image acquisition devices configured with ARM chips, due to the configuration requirements of the ARM chip itself, when receiving image data from the image sensor, on the one hand, the received image data needs to be stored frame by frame before it can be sent to the PC. On the other hand, frame-by-frame storage specifically means aligning the data according to pixel size; that is, if a pixel is less than 1 byte, it is stored as 1 byte, and if it is greater than 1 byte but less than 2 bytes, it is stored as 2 bytes. For example, if a pixel includes 10 bits of RAW10 format image data, the ARM chip will process it as two bytes (16 bits) per pixel. This results in redundant data in the image data stored on the ARM chip. Redundant data increases the size of the image data, thereby reducing the utilization of the transmission bandwidth between the image acquisition device and the PC, and increasing the transmission latency of the image data.

[0003] In summary, existing technologies still suffer from problems such as low efficiency and high latency in transmitting image data in image acquisition devices equipped with ARM chips. Summary of the Invention

[0004] To address the shortcomings of related technologies, this invention provides an image streaming method, aiming to solve the problems of low efficiency and high latency in image data transmission by image acquisition devices configured with ARM chips.

[0005] The technical solution is as follows:

[0006] According to one aspect of the present invention, an image streaming method is applied to an image acquisition device configured with an ARM chip. The method includes: receiving raw image data output by a chip under test; storing the raw image data as data to be transmitted according to a storage format supported by the ARM chip; the data to be transmitted includes redundant data generated during storage; determining multiple data packets waiting to be transmitted in the data to be transmitted according to a data packet partitioning rule; creating two parallel tasks for transmitting each data packet in the data to be transmitted, one task being used to delete redundant data in each data packet in the data to be transmitted, and the other task being used to send each data packet after deleting redundant data to a host computer, so as to realize the streaming transmission of each data packet in the data to be transmitted.

[0007] According to one aspect of the present invention, an image streaming device is provided, the device comprising: a data receiving module for receiving raw image data output by a chip under test; a data storage module for storing the raw image data as data to be transmitted according to a storage format supported by the ARM chip; the data to be transmitted includes redundant data generated during storage; a data packet determination module for determining multiple data packets waiting to be transmitted in the data to be transmitted according to data packet partitioning rules; and a data transmission module for creating two parallel tasks for transmitting each data packet in the data to be transmitted, one task for deleting redundant data in each data packet in the data to be transmitted, and the other task for sending each data packet after deleting redundant data to a host computer, so as to realize the streaming transmission of each data packet in the data to be transmitted.

[0008] According to one aspect of the present invention, an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein a computer program is stored in the memory, and the processor reads the computer program from the memory via the communication bus; when the computer program is executed by the processor, it implements the image streaming method described above.

[0009] According to one aspect of the present invention, a storage medium having a computer program stored thereon, which, when executed by a processor, implements the image streaming method as described above.

[0010] According to one aspect of the present invention, a computer program product includes a computer program stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program such that the computer device implements the image streaming method as described above when executed.

[0011] The technical solution provided by this invention has the following beneficial effects:

[0012] In the above technical solution, redundant data is deleted before the data to be transmitted, avoiding the consumption of extra bandwidth and increased transmission latency by redundant data during transmission, thereby improving the utilization rate of transmission bandwidth. Simultaneously, redundant data deletion and data packet transmission are created as two parallel tasks. While deleting redundant data from the current data packet, the transmission task of the previous data packet whose redundant data has been deleted is performed concurrently, thus realizing the streaming transmission of data packets in the data to be transmitted. This solves the problem of long-term idle transmission threads during the redundancy deletion process in existing technologies, improving transmission efficiency and reducing transmission latency. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below.

[0014] Figure 1 This is a schematic diagram of the implementation environment of an image streaming method provided in an embodiment of the present invention;

[0015] Figure 2 This is a flowchart of an image streaming method provided by an embodiment of the present invention;

[0016] Figure 3 yes Figure 2 One implementation of step 220 in the corresponding embodiment;

[0017] Figure 4 This is a diagram illustrating the format of a RAW10 image and the corresponding data to be transmitted.

[0018] Figure 5 yes Figure 3 One implementation of step 320 in the corresponding embodiment;

[0019] Figure 6 yes Figure 2 One implementation of step 260 in the corresponding embodiment is "one of the tasks is used to delete redundant data in each data packet of the data to be transmitted";

[0020] Figure 7 This is a schematic diagram illustrating the splicing of actual data in an exemplary embodiment;

[0021] Figure 8 yes Figure 6 This is one implementation of step 620 in the corresponding embodiment, which involves "removing redundant data and then splicing the actual data to obtain the data payload of each data packet".

[0022] Figure 9 yes Figure 2 This corresponds to one implementation of step 260 in the embodiment where "another task is used to send each data packet after deleting redundant data to the host computer";

[0023] Figure 10 This is a schematic diagram illustrating the parallel execution of two tasks in an exemplary embodiment;

[0024] Figure 11 This is a structural block diagram of an image streaming device according to an exemplary embodiment;

[0025] Figure 12 This is a hardware structure diagram of an image acquisition device according to an exemplary embodiment;

[0026] Figure 13This is a structural block diagram of an electronic device according to an exemplary embodiment.

[0027] Specific implementation method

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various implementations of this invention described below can be combined with each other as long as they do not conflict with each other.

[0029] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0030] Before describing the various embodiments shown in this invention, several concepts involved in this invention will first be introduced.

[0031] RAW refers to the raw data format in which a CMOS or CCD image sensor converts the captured light source signal into a digital signal. RAW formats include, but are not limited to, RAW6, RAW8, RAW10, RAW12, and RAW14. For example, RAW6 refers to a format where one pixel contains 6 bits of data, and RAW10 refers to a format where one pixel contains 10 bits of data.

[0032] ARM is a 32-bit RISC (Reduced Instruction Set Computing) processor architecture, and ARM processors / chips are microprocessors based on this architecture. Data types in the ARM architecture include: 1. Word: In the ARM architecture, a word is 32 bits long. 2. Half-Word: In the ARM architecture, a half-word is 16 bits long. 3. Byte: In the ARM architecture, a byte is 8 bits long. This means that data in ARM chips is stored and processed according to byte alignment. However, this characteristic of ARM also leads to redundant data for non-integer byte sizes. For example, when storing RAW10 format image data, ARM chips store it with 2 bytes aligned to 1 pixel.

[0033] MIPI refers to an open standard and specification for mobile application processors.

[0034] The Maximum Transmission Unit (MTU) is used to inform the other party of the maximum size of the data service unit that can be accepted, indicating the payload size that the sender can accept.

[0035] As mentioned earlier, due to the configuration requirements of the ARM chip itself, when an image acquisition device equipped with an ARM chip receives raw image data from an image sensor, it stores the raw image data as data to be transmitted according to the storage format supported by the ARM chip (i.e., byte-aligned). In this process, redundant data is generated. That is, the image data stored on the ARM chip contains redundant data. The redundant data will make the image data larger, thereby reducing the utilization rate of the transmission bandwidth between the image acquisition device and the PC and increasing the transmission latency of the image data.

[0036] Therefore, redundant data in the image data needs to be removed before the image acquisition device transmits the image data to the PC.

[0037] Typically, an ARM chip first removes redundant data from a stored frame of image data, and then transmits the image data after removing redundant data to the PC. Before the redundant data removal of a frame of image data is completed, no image data is transmitted between the image acquisition device and the PC. That is, the transmission thread is idle during the redundancy removal process. Only after the redundancy removal of a frame of image data is completed can the image data after redundancy removal be transmitted. This leads to transmission delay. Moreover, as the image size increases, the amount of image data also increases, and the amount of data to be transmitted also increases accordingly. The transmission delay will increase more and more significantly, ultimately affecting the real-time processing of the PC.

[0038] As can be seen from the above, the related technologies still have limitations such as low efficiency and high latency in transmitting image data for image acquisition devices configured with ARM chips.

[0039] Therefore, the image streaming method provided by the present invention can effectively improve transmission efficiency and reduce transmission latency. Accordingly, the image streaming method is applicable to image streaming devices, which can be deployed in electronic devices, such as image acquisition devices configured with ARM chips.

[0040] Please see Figure 1 This diagram illustrates an implementation environment for an image streaming method according to an embodiment of the present invention. The implementation environment includes an image acquisition device 101 configured with an ARM chip, a host computer 102, and a chip under test 103.

[0041] The image acquisition device 101 configured with an ARM chip can be an image acquisition card, used to acquire image data from the chip under test and transmit the acquired image data to the host computer 102 for image processing.

[0042] The chip under test 103 can be an image sensor, such as a CMOS image sensor, a CCD image sensor, etc., and is used to output raw image data. The raw image data output by the chip under test 103 is acquired by an image acquisition device 101 configured with an ARM chip.

[0043] The chip under test 103 and the image acquisition device 101 configured with an ARM chip communicate via the MIPI protocol. For example, the chip under test 103 transmits image data to the image acquisition device 101 configured with an ARM chip through the MIPI interface.

[0044] The image acquisition device 101, configured with an ARM chip, stores the original image data as data to be transmitted according to the storage format supported by the ARM chip. During this storage process, redundant data is generated in the data to be transmitted. In order to remove the redundant data in the data to be transmitted, the image acquisition device 101, configured with an ARM chip, divides the data to be transmitted into multiple data packets waiting to be transmitted according to the data packet partitioning rules, and creates two parallel tasks to perform redundant data removal and data packet transmission in parallel, so as to realize the streaming transmission of data packets in the data to be transmitted.

[0045] The host computer 102 can be a desktop computer, laptop computer, server, etc. A communication connection is pre-established between the image acquisition device 101 and the host computer 102 via wired or wireless means to realize data transmission between the image acquisition device 101 and the host computer 102. For example, the transmitted data can be data packets sent by the image acquisition device 101.

[0046] Specifically, for the host computer 102, after receiving the data packet sent by the image acquisition device 101, the received data packet is processed according to byte alignment.

[0047] In the above process, the compression and transmission of the data to be transmitted in the image acquisition device 101 and the receiving and decompression process in the host computer 102 are like an assembly line operation from a microscopic perspective. This effectively parallelizes the time for algorithm processing and network I / O transmission. The overall time is theoretically close to half that of traditional processing methods, which greatly improves transmission efficiency and reduces transmission latency.

[0048] Please see Figure 2 The diagram illustrates a flowchart of an image streaming method provided by an embodiment of the present invention. This method is applicable to image acquisition devices configured with ARM chips, such as image acquisition cards.

[0049] like Figure 2 As shown, the method may include the following steps:

[0050] Step 200: Receive the raw image data output by the chip under test.

[0051] The chip under test can be an image sensor, such as a CMOS image sensor, a CCD image sensor, etc. The chip under test is used to output raw image data.

[0052] Raw image data refers to the original data generated by a CMOS image sensor or CCD image sensor converting the captured light source signals into digital signals. For example, raw image data can be RAW images, i.e., image data conforming to the RAW format, which includes, but is not limited to, RAW6, RAW8, RAW10, RAW12, RAW14, etc.

[0053] In one possible implementation, an image acquisition device configured with an ARM chip receives raw image data output by the chip under test via a MIPI interface.

[0054] Step 220: Store the original image data as data to be transmitted according to the storage format supported by the ARM chip.

[0055] The data to be transmitted includes redundant data generated during the storage process.

[0056] The storage format supported by ARM chips refers to the format in which each pixel data is processed using an integer number of bytes.

[0057] In other words, after receiving a RAW image, the ARM MIPI decoder typically aligns and stores the data according to the length of each pixel. If a pixel is less than 1 byte, it is stored as 1 byte; if it is greater than 1 byte but less than 2 bytes, it is stored as 2 bytes. For example, for commonly used RAW10, RAW12, and RAW14 storage formats, ARM chips store image data of each pixel that is greater than 1 byte but less than 2 bytes in alignment as 2 bytes (16 bits). Therefore, for RAW10, RAW12, and RAW14 format image data, each pixel contains 10, 12, or 14 bits of raw image data. The final data to be transmitted stored in the ARM chip contains 16 bits of data per pixel. In this case, the data corresponding to each pixel will have 6, 4, or 2 bits of redundant data.

[0058] Step 240: According to the data packet segmentation rules, determine the multiple data packets waiting to be transmitted in the data to be transmitted.

[0059] Among them, the packet segmentation rule can refer to the maximum transmission unit (MTU) limit rule.

[0060] For example, if the maximum transmission unit (MTU) is 512 bits, then when the data to be transmitted is 512 * 10 bits, it can be determined that there are 10 data packets to be transmitted.

[0061] Step 260: Create two parallel tasks for transmitting each data packet in the data to be transmitted. One task is used to remove redundant data from each data packet in the data to be transmitted, and the other task is used to send each data packet after removing redundant data to the host computer, so as to realize the streaming transmission of each data packet in the data to be transmitted.

[0062] As mentioned earlier, redundant data consumes storage space and transmission bandwidth, wasting resources and increasing transmission latency. Therefore, redundant data generally needs to be removed before transmission. The traditional approach is to remove redundant data from a complete frame of image data before transmitting the entire frame to the PC. Before the redundant data removal is complete, the transmission thread remains idle and cannot perform any transmission tasks, resulting in low efficiency and transmission latency. If the image data becomes larger, the transmission latency increases even faster. In this embodiment, two parallel tasks are created for the multiple data packets divided into the data to be transmitted. One task removes redundant data, and the other task transmits the data packets after redundant data removal. Thus, after removing redundant data from one data packet, the transmission task of the data packet after redundant data removal can be performed simultaneously with the removal of redundant data from the next data packet. This reduces the idle time of the transmission thread during redundant data removal and also reduces the idle time on the PC, avoiding the problem of poor real-time image data processing due to excessive idle time on the PC.

[0063] Through the above embodiments, redundant data is removed before the data to be transmitted, avoiding the use of extra bandwidth by redundant data during transmission, thereby improving bandwidth utilization and reducing transmission latency. Simultaneously, redundant data removal and data packet transmission are created as two parallel tasks. While deleting redundant data from the current data packet, the transmission task of the previous data packet, whose redundant data has been removed, can be performed concurrently. This achieves streaming transmission of data packets within the data to be transmitted, solving the problem of long-term idle transmission threads during redundant data removal in existing technologies, improving transmission efficiency, and reducing transmission latency.

[0064] Please see Figure 3 In one exemplary embodiment, step 220 may include the following steps:

[0065] Step 300: Parse the original image data according to the storage format supported by the ARM chip to obtain the data corresponding to each pixel in the original image data.

[0066] Parsing refers to combining the data corresponding to each pixel in the original image data, which is stored separately, into the data corresponding to a complete pixel.

[0067] This explanation uses a RAW10 image as the raw image data. Assuming each pixel in a RAW10 image contains 10 bits of data, then a RAW10 image requires 5 bytes to store the data corresponding to 4 pixels. (Reference) Figure 4This diagram illustrates the format of a RAW10 image and the corresponding data to be transmitted. In a RAW10 image, the high eight bits (A2 to A9) of pixel A occupy one byte, the high eight bits (B2 to B9) of pixel B occupy one byte, the high eight bits (C2 to C9) of pixel C occupy one byte, and the high eight bits (D2 to D9) of pixel D occupy one byte. The low two bits (A0, A1, B0, B1, C0, C1, D0, D1) of these four pixels together occupy one byte. It is evident that, to comply with the MIPI transmission protocol, the data corresponding to each pixel in the original image data is stored separately. Therefore, parsing a RAW10 image according to the storage format supported by the ARM chip (e.g., RAW10) means concatenating the separated A0, A1, and A2 to A9 bits of each pixel in the RAW10 image into a complete pixel (A0 to A9).

[0068] Step 320: Based on the data length of each pixel's corresponding data obtained from parsing, the data corresponding to each pixel is aligned and stored so that the original image data is stored as data to be transmitted containing redundant data.

[0069] Aligned storage refers to byte-aligned storage, which means that the data corresponding to each pixel is stored using an integer number of bytes to conform to the storage format supported by the ARM chip. For example, if the data length of each pixel is less than 1 byte, it is stored as 1 byte; if it is greater than 1 byte but less than 2 bytes, it is stored as 2 bytes.

[0070] refer to Figure 4 Taking RAW10 images as an example, the data length corresponding to each pixel obtained from RAW10 image parsing is 10 bits (e.g., A0 to A9). If two bytes (16 bits) are used to align and store the data corresponding to each pixel, then each pixel in the resulting data to be transmitted will have 6 bits of redundant data. Specifically, as... Figure 5 As shown, step 320 may include the following steps:

[0071] Step 500: Based on the storage format supported by the ARM chip, determine the set data length and set padding method for the data configuration corresponding to each pixel.

[0072] The set data length is related to the data length of each pixel and should be an integer byte length. For example, for a RAW10 image, the data length of each pixel is 10 bits, so the set data length can be two bytes.

[0073] Setting a padding method refers to the padding pattern used when the data length of each pixel is less than a set length, in which case the insufficient portion needs to be padded according to the specified padding pattern. For example, refer to... Figure 4In the two-byte set data length, the data A0 to A9 corresponding to pixel A are filled in the 4th to 13th bits of the two bytes, and the other bits in the two-byte set data length are filled with 0.

[0074] Step 520: For each pixel, if the data length of the data corresponding to the pixel is less than the set data length, then the data corresponding to the pixel is padded with bytes according to the set padding method so that the data length of the data corresponding to each pixel is equal to the set data length.

[0075] Step 540: Obtain the data to be transmitted from the data storage corresponding to each pixel that has been filled.

[0076] refer to Figure 4 The data corresponding to the completed pixel A are 0, 0, A9, A8, A7, A6, A5, A4, A3, A2, A1, A0, 0, 0, 0, and each completed pixel data will contain 6 bits of redundant data.

[0077] Please see Figure 6 In an exemplary embodiment, step 260, "one of the tasks is to remove redundant data from each data packet in the data to be transmitted," includes the following steps:

[0078] Step 600: For each data packet in the data to be transmitted, generate the packet header of each data packet and determine the data corresponding to multiple pixels in each data packet.

[0079] In one possible implementation, the packet header stores BlockId and PacketId. BlockId is 2 bytes and is used to indicate the current channel number. Multiple virtual channel data exist in the same frame, and BlockId is used to distinguish which virtual channel the data belongs to. PacketId is 4 bytes and is used to indicate the current packet ID, allowing the receiving end to determine whether the current image has been transmitted.

[0080] Step 620: Based on the data corresponding to multiple pixels in each data packet, determine the redundant data and actual data corresponding to multiple pixels in each data packet, remove the redundant data, and then splice the actual data to obtain the data payload of each data packet.

[0081] refer to Figure 7 This illustrates a schematic diagram of concatenating actual data in an exemplary embodiment. The actual data corresponding to each pixel refers to D0 to D9, and redundant data refers to 0. In one possible implementation, the lower eight bits of four pixels are concatenated using four consecutive bytes, and the higher two bits of these four pixels are concatenated into one byte as the fifth byte, thus concatenating the actual data of the four pixels into five bytes.

[0082] Step 640: Encapsulate the header and data payload of each data packet to obtain each data packet with redundant data removed.

[0083] In one possible implementation, an additional data packet is added outside each data packet. This packet informs the receiving end of basic information about the current image frame, such as the frame header, frame tail timestamp, and the number of image data packets to be transmitted. For example, this packet may include ImageType, HeadTimestamp, PixelFormat, Width, Height, XOffset, YOffset, Padding_x / padding_y, TailTimestamp, and FrameSequence. ImageType is 4 bytes, indicating the image type; HeadTimestamp is 8 bytes, indicating the frame header timestamp; PixelFormat is 4 bytes, indicating the pixel format; Width is 4 bytes, indicating the image width; Height is 4 bytes, indicating the image height; XOffset is 4 bytes, indicating the X-axis offset coordinate; YOffset is 4 bytes, indicating the Y-axis offset coordinate; Padding_x / padding_y are both 4 bytes, indicating the inner margins in the X / Y directions; TailTimestamp is 8 bytes, indicating the frame tail timestamp; and FrameSequence is 4 bytes, indicating the frame sequence number.

[0084] Please see Figure 8 In an exemplary embodiment, step 620, "after removing redundant data, the data payload of each data packet is obtained by splicing the actual data," includes the following steps:

[0085] Step 800: For each data packet, determine the sign bit data and valid bit data contained in the actual data corresponding to each pixel in the data packet.

[0086] by Figure 7 Taking this as an example, the actual data corresponding to each pixel includes sign bit data D8 and D9, and valid bit data D0 to D7.

[0087] Step 820: Concatenate the valid bit data corresponding to each pixel in the set high bits of the data payload, and concatenate the sign bit data corresponding to each pixel in the set low bits of the data payload to obtain the data payload of the data packet.

[0088] by Figure 7 Taking this example, the valid data D0 to D7 are concatenated in the high-order bytes of the data payload, while the sign data D8 and D9 are concatenated in the low-order bytes of the data payload.

[0089] Please see Figure 9 In an exemplary embodiment, step 260, "another task for sending the data packets after removing redundant data to the host computer," may include the following steps:

[0090] Step 900: For each data packet in the data to be transmitted, while one task is called to remove redundant data from the first data packet, another task is waiting to be called.

[0091] Step 920: If the first data packet completes the deletion of redundant data, then another task is called to send the first data packet after the deletion of redundant data to the host computer, and one of the tasks is called to delete redundant data in the second data packet.

[0092] by Figure 10 To illustrate, we assume that each data packet requires the same amount of time to perform two tasks. When Task 1 is called to remove redundant data from Data Packet 1, Task 2 is idle. Once the redundant data removal of Data Packet 1 is complete, Task 2 can be called to send Data Packet 1 to the host computer, while Task 1 removes redundant data from Data Packet 2.

[0093] It needs to be explained that, Figure 10 The statement that each data packet takes the same amount of time to complete two tasks is for illustrative purposes only; in reality, the time required for each data packet to complete two tasks can be different.

[0094] Step 940 continues until all data packets in the data to be transmitted are sent to the host computer, completing the streaming transmission of each data packet in the data to be transmitted.

[0095] With the cooperation of the above embodiments, deleting redundant data before transmitting the data to be transmitted can avoid redundant data occupying additional transmission bandwidth, reduce transmission latency, and improve transmission bandwidth utilization. At the same time, the data to be transmitted is divided into multiple data packets, and two parallel tasks are created to delete redundant data and transmit the data packets. During the process of deleting redundant data in the current data packet, the transmission task of the previous data packet whose redundant data has been deleted is performed concurrently. This realizes the streaming transmission of data packets in the data to be transmitted, reduces the waiting time of the transmission thread during the redundancy deletion process, solves the problem of long-term idle transmission thread during the redundancy deletion process in the prior art, improves transmission efficiency, and reduces transmission latency.

[0096] The following are embodiments of the apparatus of the present invention, which can be used to execute the image streaming method involved in the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the method embodiments of the image streaming method involved in the present invention.

[0097] Please see Figure 11This invention provides an image streaming device 1100, including but not limited to: a data receiving module 1110, a data storage module 1130, a data packet determination module 1150, and a data transmission module 1170.

[0098] The data receiving module 1110 is used to receive the raw image data output by the chip under test.

[0099] The data storage module 1130 is used to store the original image data as data to be transmitted, according to the storage format supported by the ARM chip. The data to be transmitted includes redundant data generated during the storage process.

[0100] The data packet determination module 1150 is used to determine multiple data packets waiting to be transmitted in the data to be transmitted according to the data packet segmentation rules.

[0101] The data transmission module 1170 is used to create two parallel tasks for transmitting each data packet in each frame of data to be transmitted. One task is used to remove redundant data from each data packet in the data to be transmitted, and the other task is used to send each data packet after removing redundant data to the host computer, so as to realize the streaming transmission of each data packet in the data to be transmitted.

[0102] It should be noted that the image streaming device provided in the above embodiments is only illustrated by the division of the above functional modules when performing image streaming. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the image streaming device will be divided into different functional modules to complete all or part of the functions described above.

[0103] Furthermore, the image streaming device and the image streaming method provided in the above embodiments belong to the same concept, and the specific way in which each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0104] Figure 12 A schematic diagram of the hardware structure of an image acquisition device is shown according to an exemplary embodiment. This image acquisition device is suitable for... Figure 1 The image acquisition device 101 in the implementation environment is shown.

[0105] It should be noted that this image acquisition device is merely an example adapted to this invention and should not be construed as providing any limitation on the scope of use of this invention. Furthermore, this image acquisition device should not be interpreted as requiring or depending on any specific feature. Figure 12 One or more components of the exemplary image acquisition device 2000 shown.

[0106] The hardware structure of the image acquisition device 2000 can vary considerably due to differences in configuration or performance, such as... Figure 12 As shown, the image acquisition device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0107] Specifically, power supply 210 is used to provide operating voltage for the various hardware devices on image acquisition device 2000.

[0108] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, to perform... Figure 1 The diagram illustrates the interaction between the host computer 102 and the image acquisition device 101 in the implementation environment.

[0109] Of course, in other examples of this invention, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 12 As shown, this does not constitute a specific limitation.

[0110] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0111] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the image acquisition device 2000, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0112] Application 253 is a computer program that performs at least one specific task based on operating system 251, and may include at least one module ( Figure 12 (Not shown), each module can contain a computer program for the image acquisition device 2000.

[0113] For example, an image streaming device can be considered as an application 253 deployed on an image acquisition device 2000.

[0114] Data 255 can be photos, pictures, etc. stored on a disk, and stored in memory 250.

[0115] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on massive amounts of data 255 stored in the memory 250. For example, an image streaming method can be implemented by the central processing unit 270 reading a series of computer programs stored in the memory 250.

[0116] Furthermore, the present invention can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of the present invention is not limited to any specific hardware circuit, software, or combination thereof.

[0117] Please see Figure 13 This invention provides an electronic device 4000, which may include an image acquisition card.

[0118] exist Figure 13 The electronic device 4000 includes at least one processor 4001, at least one communication bus 4002, and at least one memory 4003.

[0119] The processor 4001 and memory 4003 are connected, for example, via a communication bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0120] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0121] The communication bus 4002 may include a path for transmitting information between the aforementioned components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0122] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0123] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002.

[0124] When the computer program is executed by the processor 4001, it implements the image streaming methods described in the above embodiments.

[0125] Furthermore, this embodiment of the invention provides a storage medium storing a computer program, which, when executed by a processor, implements the image streaming methods described in the above embodiments.

[0126] This invention provides a computer program product comprising a computer program stored in a storage medium. A processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the image streaming methods described in the above embodiments.

[0127] Compared with related technologies, deleting redundant data before transmitting the data to be transmitted can avoid redundant data occupying additional transmission bandwidth, reduce transmission latency, and improve transmission bandwidth utilization. At the same time, dividing the data to be transmitted into multiple data packets and creating two parallel tasks to delete redundant data and transmit the data packets, while deleting redundant data in the current data packet, the transmission task of the previous data packet with deleted redundant data is carried out concurrently. This realizes the streaming transmission of data packets in the data to be transmitted, reduces the waiting time of the transmission thread during the redundancy deletion process, solves the problem of long-term idle transmission threads during the redundancy deletion process in existing technologies, improves transmission efficiency, and reduces transmission latency.

[0128] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0129] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image streaming method applied to an image acquisition device configured with an ARM chip, characterized in that, The method comprises: receiving original image data output by a chip to be tested; storing the original image data as to-be-transmitted data in a storage format supported by the ARM chip; the to-be-transmitted data contains redundant data generated in the storage process due to the need for byte alignment storage and processing of the original image data; determining, according to a data packet division rule, a plurality of data packets in the to-be-transmitted data waiting for transmission; creating two parallel tasks for transmission of the data packets in the to-be-transmitted data, one task being used to delete redundant data of the data packets in the to-be-transmitted data, and the other task being used to send the data packets after deletion of the redundant data to an upper computer to realize streaming transmission of the data packets in the to-be-transmitted data.

2. The method of claim 1, wherein, The storage of the original image data as to-be-transmitted data in the storage format supported by the ARM chip comprises: parsing the original image data according to the storage format supported by the ARM chip to obtain data corresponding to each pixel in the original image data; based on the data length of the data corresponding to each pixel obtained by parsing, performing aligned storage on the data corresponding to each pixel, so that the original image data is stored as the to-be-transmitted data containing the redundant data.

3. The method of claim 2, wherein, The aligned storage of the data corresponding to each pixel based on the data length of the data corresponding to each pixel obtained by parsing comprises: based on the storage format supported by the ARM chip, determining a set data length and a set padding mode configured for the data corresponding to each pixel; for each pixel, if the data length of the data corresponding to the pixel is less than the set data length, performing byte padding on the data corresponding to the pixel according to the set padding mode, so that the data length of the data corresponding to each pixel is equal to the set data length; storing the data corresponding to each pixel after completion of padding to obtain the to-be-transmitted data.

4. The method of claim 1, wherein, The one task being used to delete the redundant data of the data packets in the to-be-transmitted data comprises: for each data packet in the to-be-transmitted data, generating a packet header of each data packet and determining data corresponding to a plurality of pixels in each data packet; based on the data corresponding to a plurality of pixels in each data packet, determining redundant data and actual data corresponding to a plurality of pixels in each data packet, and splicing the data payload of each data packet from the actual data after the redundant data is removed; encapsulating the packet header and the data payload of each data packet to obtain each data packet after deletion of the redundant data.

5. The method of claim 4, wherein, The splicing of the data payload of each data packet from the actual data after the redundant data is removed comprises: for each data packet, determining sign bit data and valid bit data contained in the actual data corresponding to each pixel in the data packet; splicing the valid bit data corresponding to each pixel at a set high bit of the data payload, and splicing the sign bit data corresponding to each pixel at a set low bit of the data payload to obtain the data payload of the data packet.

6. The method of claim 1, wherein, The other task is used to send the data packets after deleting the redundant data to the host computer, and comprises: For each data packet in the data to be transmitted, when the one task is called to delete the redundant data of the first data packet, the other task is called to wait; If the first data packet completes the deletion of the redundant data, the other task is called to send the first data packet after deleting the redundant data to the host computer, and the one task is called to delete the redundant data of the second data packet; Until all data packets in the data to be transmitted are sent to the host computer, the streaming of each data packet in the data to be transmitted is completed.

7. The method according to any one of claims 1 to 6, characterized in that, The data packet division rule comprises a maximum transmission unit (MTU) limit rule.

8. An image streaming apparatus characterized by comprising: The device comprises: a data receiving module configured to receive original image data output by a chip under test; a data storage module configured to store the original image data as data to be transmitted in a storage format supported by an ARM chip, wherein the data to be transmitted contains redundant data generated in a storage process due to the fact that the original image data needs to be stored and processed according to byte alignment; a data packet determination module configured to determine a plurality of data packets to be transmitted in the data to be transmitted according to a data packet division rule; a data transmission module configured to create two parallel tasks for transmitting each data packet in the data to be transmitted, wherein one task is used to delete the redundant data of each data packet in the data to be transmitted, and the other task is used to send each data packet after deleting the redundant data to a host computer, so as to realize the streaming of each data packet in the data to be transmitted.

9. An electronic device, comprising: comprise: at least one processor, at least one memory, and at least one communication bus, wherein the memory has stored thereon a computer program, and the processor reads the computer program in the memory through the communication bus; the computer program is executed by the processor to implement the image streaming method in any one of claims 1 to 7.

10. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the image streaming method in any one of claims 1 to 7.

Images