Image acquisition method, image acquisition device and semiconductor test equipment

Abstract

The invention provides an image acquisition method, an image acquisition device and semiconductor test equipment, and the method comprises the steps: determining a lock-free annular image acquisition buffer queue based on the hardware information of the semiconductor test equipment under the condition of obtaining an instruction which is sent by an upper computer and is used for collecting continuous image frames, and starting an acquisition thread to carry out the image frame collection; under the condition that the image frames are collected, the collected image frames are stored in a lock-free annular image collection cache queue, and the image frames are taken out from the lock-free annular image collection cache queue; and sending the extracted image frame to the upper computer, so that the size of the lock-free annular image acquisition buffer queue can be allocated according to the hardware performance, and the lock-free annular image acquisition buffer queue is adopted to transmit the image frame data, so that the continuity of the image frame can be ensured, and the transmission efficiency can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of semiconductor testing, and particularly relates to an image acquisition method, an image acquisition device, and a semiconductor testing device. Background Art

[0002] A CMOS image sensor (CIS, Complementary Metal-Oxide-Semiconductor Image Sensor) chip is a core component of a camera, and a camera is an important part of all intelligent technologies. Currently, CIS chips have been widely used in multiple fields such as mobile phones, cameras, automotive electronics, and security. Image sensors are constantly developing towards higher frame rates and higher pixel counts. The requirements for the image quality of chips in end application scenarios are getting higher and higher. It is necessary to ensure that the output images do not drop frames and have no dark or bright spots. For the requirements of front-end CP testing, it is also becoming more and more strict, and the technical requirements for image acquisition and transmission are also getting higher and higher. For the transmission of higher frame rate and high-resolution image data, a higher-bandwidth physical transmission channel is required. In existing technical solutions, it is usually necessary to select a USB3.0 / Thunderbolt / 10 Gigabit Ethernet or higher physical transmission channel. The higher the bandwidth, the higher the performance requirements for the physical transmission channel, and the higher the hardware cost. In order to reduce costs, a software system is used for processing. In a multi-threaded access environment of a single production / single consumption mode, although the locking competition mechanism of the traditional software system can ensure that data access does not conflict, sometimes it harms the system performance more than data copying and other processing, resulting in a lower acquisition and transmission efficiency of image data and making it difficult to meet the requirement of continuous image frames for testing. Summary of the Invention

[0003] Embodiments of this application provide an image acquisition method, an image acquisition device, and a semiconductor testing device, which can allocate the size of a lock-free circular image acquisition buffer queue according to the hardware performance, and use the lock-free circular image acquisition buffer queue to transmit image frame data, which can ensure the continuity of image frames and improve the transmission efficiency.

[0004] In a first aspect, embodiments of this application provide an image acquisition method, which is applied to a semiconductor testing device and includes:

[0005] When receiving an instruction from the host computer to acquire continuous image frames, determining a lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor testing device, and starting an acquisition thread to acquire image frames;

[0006] When an image frame is acquired, storing the acquired image frame in the lock-free circular image acquisition buffer queue, and taking out the image frame from the lock-free circular image acquisition buffer queue;

[0007] Send the retrieved image frame to the host computer.

[0008] In some embodiments, the step of sending the retrieved image frame to the host computer includes:

[0009] Determine the basic information of the retrieved image frame, generate a header packet frame based on the basic information, and send the header packet frame to the host computer;

[0010] Divide the retrieved image frame into multiple image data based on the load data of the transmission bandwidth between the semiconductor test equipment and the host computer, compress the multiple image data to generate intermediate packet frames, and send the intermediate packet frames to the host computer. When one image data is compressed to generate one intermediate packet frame, it is immediately sent to the host computer;

[0011] After the transmission of multiple intermediate packet frames is completed, generate a tail packet frame and send the tail packet frame to the host computer.

[0012] In some embodiments, the step of storing the acquired image frame in a lock-free circular image acquisition buffer queue includes:

[0013] Obtain the current write index of the lock-free circular image acquisition buffer queue;

[0014] Calculate the next write index based on the current write index;

[0015] Determine whether the lock-free circular image acquisition buffer queue is full based on the write index;

[0016] When the lock-free circular image acquisition buffer queue is not full, determine the transmission state of the acquisition thread;

[0017] Determine the priority of storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the transmission state and the frame sequence number of the acquired image frame, and store the acquired image frame in the lock-free circular image acquisition buffer queue based on the priority.

[0018] In some embodiments, the step of determining the priority of storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the transmission state and the frame sequence number of the acquired image frame, and storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the priority includes:

[0019] When the transmission state of the acquisition thread is transmission closed, determine the magnitude relationship of the sequence numbers of the acquired image frames, determine the priority of storing in the lock-free circular image acquisition buffer queue based on the magnitude relationship, and store the acquired image frames in the lock-free circular image acquisition buffer queue based on the priority. The smaller the sequence number of the acquired image frame, the higher the priority of the corresponding image frame;

[0020] When the transmission state of the acquisition thread is transmission enabled, determine the size relationship of the sequence numbers of the acquired image frames, determine the priority for storing in the lock-free circular image acquisition buffer queue based on the size relationship, and store the acquired image frames in the lock-free circular image acquisition buffer queue based on the priority, where the higher the sequence number of the acquired image frame, the higher the priority of the corresponding image frame.

[0021] In some embodiments, taking out an image frame from the lock-free circular image acquisition buffer queue further includes:

[0022] Obtain the readable image frames in the lock-free circular image acquisition buffer queue;

[0023] In the case of multiple readable image frames, select the image with the smallest frame sequence number and transmit the image with the smallest frame sequence number.

[0024] In some embodiments, the method further includes:

[0025] In the case of obtaining an instruction of on-demand request transmission mode sent by the host computer, pause the real-time continuous acquisition and transmission mode and enter the on-demand request transmission mode. Wherein, in the case of entering the on-demand request transmission mode and in the case of obtaining an instruction of acquiring continuous image frames sent by the host computer, determine the lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor test equipment and start an acquisition thread to acquire image frames.

[0026] In some embodiments, the method further includes:

[0027] In the case of obtaining a retransmission instruction of a specified data packet sent by the host computer, retransmit the instruction data packet based on the sequence number of the specified data packet in the retransmission instruction, where the host computer can determine the sequence number of the lost specified data packet based on the header packet frame, middle packet frame, and tail packet frame corresponding to the image frame sent by the semiconductor test equipment, and generate a retransmission instruction of the specified data packet based on the sequence number of the specified data packet.

[0028] In a second aspect, an embodiment of the present application provides an image acquisition device, including:

[0029] A determination module, configured to determine a lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor test equipment and start an acquisition thread to acquire image frames in the case of obtaining an instruction of acquiring continuous image frames sent by the host computer;

[0030] A buffer module, configured to store the acquired image frames in the lock-free circular image acquisition buffer queue and take out image frames from the lock-free circular image acquisition buffer queue when the image frames are acquired;

[0031] A sending module, configured to send the retrieved image frames to the host computer.

[0032] In a third aspect, an embodiment of the present application provides a semiconductor testing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in any one of the above is implemented.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, a semiconductor testing device is enabled to execute the method described in any one of the above.

[0035] The beneficial effects of the embodiment of the present application compared with the prior art are as follows:

[0036] For the image acquisition method provided by the embodiment of the present application, when an instruction to acquire continuous image frames sent by the host computer is obtained, a lock-free circular image acquisition buffer queue is determined based on the hardware information of the semiconductor testing device, and an acquisition thread is started to acquire image frames. When an image frame is acquired, the acquired image frame is stored in the lock-free circular image acquisition buffer queue, and an image frame is retrieved from the lock-free circular image acquisition buffer queue. The retrieved image frame is sent to the host computer. In this way, the size of the lock-free circular image acquisition buffer queue can be allocated according to the hardware performance, and the lock-free circular image acquisition buffer queue is used to transmit image frame data, which can ensure the continuity of the image frames and improve the transmission efficiency.

[0037] It can be understood that the beneficial effects of the above second aspect to fifth aspect can refer to the relevant descriptions in the first aspect above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0039] Figure 1 It is a schematic diagram of the implementation process of an image acquisition method provided by an embodiment of the present application;

[0040] Figure 2A schematic diagram of network communication of a network protocol provided by an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the implementation process of a transmission method provided by an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the index of a lock-free circular image acquisition buffer queue provided by an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the process of an image acquisition method provided by an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the structure of an image acquisition device provided by an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of a semiconductor test device provided by an embodiment of the present application. Detailed implementation manners

[0046] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are set forth in order to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obscuring the description of the present application.

[0047] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0048] It should also be understood that the term "and / or" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrases "if determined" or "if detected" can be interpreted as meaning "once determined", "in response to determining", "once detected", or "in response to detecting" according to the context.

[0050] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", etc. are only used for differential description and should not be construed as indicating or implying relative importance.

[0051] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0052] Based on the technical problems of the related art, the embodiments of the present application provide an image acquisition method that can be applied to mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), semiconductor test devices such as image acquisition cards, etc. The embodiments of the present application do not impose any restrictions on the specific types of semiconductor test devices.

[0053] The embodiments of the present application provide an image acquisition method. Hereinafter, taking a semiconductor test device as an image acquisition card as an example for illustration. Figure 1 For the implementation flow schematic diagram of an image acquisition method provided by the present application, as Figure 1 shown, the image acquisition method includes:

[0054] Step S101, when an instruction to acquire consecutive image frames sent by the host computer is obtained, determine a lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor test device, and start an acquisition thread to acquire image frames.

[0055] In the embodiments of the present application, the network communication between the host computer and the image acquisition card can be an application layer protocol based on the GigE Vision standard UDP transmission protocol. The application layer protocol can include: the GigE Vision Control Protocol (GVCP) and the extended GVSP stream transmission protocol. The extended GVCP control protocol is used for conventional control command interaction, and the extended GVSP stream transmission protocol is used for the transmission of image data streams.

[0056] In the embodiments of the present application, the host computer can send instructions for continuous image frames to the image acquisition card by extending the GVCP control protocol, so that the image acquisition card can obtain the instructions for acquiring continuous image frames sent by the host computer.

[0057] In some embodiments, the network communication between the host computer and the image acquisition card is still based on protocols such as WS-Discovery and ONVIF.

[0058] In the embodiments of the present application, when the instructions from the host computer are obtained, the instructions can be parsed to determine the operation instructions. For example, start acquisition, stop acquisition, obtain images, etc. The instructions may include information on the number of frames of continuous image frames.

[0059] In the embodiments of the present application, the hardware performance can be determined based on the hardware information, and the largest lock-free circular image buffer queue can be allocated according to the hardware performance.

[0060] In the embodiments of the present application, the lock-free circular image acquisition buffer queue is an efficient data structure suitable for the processing of real-time images or data streams, especially in a multi-threaded environment. Through the lock-free circular image acquisition buffer queue, the performance bottleneck brought by the traditional lock mechanism can be avoided, thereby improving the overall performance of the image acquisition system.

[0061] In the embodiments of the present application, the lock-free circular image acquisition buffer queue has a circular structure and adopts a lock-free mechanism. The lock-free circular image acquisition buffer queue uses an array of fixed size to store image data, with the head and tail connected to form a ring. Through the circular structure setting, the memory can be effectively utilized, and frequent memory allocation and release can be avoided.

[0062] In the embodiments of the present application, the lock-free mechanism realizes the enqueue and dequeue operations by using atomic operations (such as CAS, i.e., compare and swap), avoiding thread blocking and improving the concurrency performance.

[0063] In the embodiments of the present application, through the lock-free circular image acquisition buffer queue, the producer (acquisition thread) can continuously put image data into the queue, while the consumer (processing or sending thread) can read the data from the queue for processing. The lock-free mechanism ensures efficient cooperation among multiple threads.

[0064] In the embodiments of the present application, an image acquisition function can be written to run in a new thread, and this thread is responsible for obtaining image data from the image acquisition device. The image acquisition device can be an image sensor such as a camera. A thread library can be used to create and start a new thread to run the image acquisition function, thereby starting the acquisition thread to perform image frame acquisition.

[0065] Step S102, when an image frame is acquired, store the acquired image frame in a lock-free circular image acquisition buffer queue, and retrieve an image frame from the lock-free circular image acquisition buffer queue.

[0066] In the embodiments of the present application, the image acquisition card can access image data through the SDK or API interface of the image acquisition device. Continuously call the camera interface in the acquisition thread to capture images and obtain image frames.

[0067] In the embodiments of the present application, before storing the acquired image frame in the lock-free circular image acquisition buffer queue, it is necessary to determine whether the lock-free circular image acquisition buffer queue is full. If it is not full, store the acquired image frame in the lock-free circular image acquisition buffer queue. If the lock-free circular image acquisition buffer queue is full, the image frame cannot be put in.

[0068] In the embodiments of the present application, before retrieving data from the lock-free circular image acquisition buffer queue, it is necessary to determine whether the lock-free circular image acquisition buffer queue is empty. If it is not empty, data can be retrieved from the lock-free circular image acquisition buffer queue.

[0069] In the embodiments of the present application, due to the sharp increase in the total image bandwidth brought by high resolution and high frame rate, limited by subsequent image format conversion and hardware transmission bandwidth, processing conversion time-consuming and transmission failure will cause frame loss and discontinuity. In the embodiments of the present application, the method of trading space for time can be used. By obtaining the available memory of the hardware, allocate as many image buffers as possible. The acquired images are first cached in the queue and then sent. The image data acquired by the acquisition thread enters the queue buffer from the end of the queue, and the first-entered image data will be retrieved from the head of the queue and packed and transmitted by the transmission thread.

[0070] In the embodiments of the present application, for the single-producer and single-consumer scenario, the memory barrier technology is used to achieve lock-free concurrent access to data dequeue and enqueue.

[0071] Step S103, send the retrieved image frame to the host computer.

[0072] In the embodiments of the present application, the image frame can be sent to the host computer by using a streaming transmission scheme that compresses the image frame in a lossless compression manner with a custom data structure while packing.

[0073] The image acquisition method provided by the embodiments of the present application, when receiving the instruction for acquiring consecutive image frames sent by the host computer, determines a lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor test equipment, and starts an acquisition thread to acquire image frames; when an image frame is acquired, stores the acquired image frame in the lock-free circular image acquisition buffer queue, and extracts an image frame from the lock-free circular image acquisition buffer queue; and sends the extracted image frame to the host computer. In this way, the size of the lock-free circular image acquisition buffer queue can be allocated according to the hardware performance, and the lock-free circular image acquisition buffer queue is used to transmit image frame data, which can ensure the continuity of the image frames and improve the transmission efficiency.

[0074] In some embodiments, step S103 may be implemented through the following steps:

[0075] Step S1031, determine the basic information of the extracted image frame, generate a header packet frame based on the basic information, and send the header packet frame to the host computer.

[0076] In the embodiments of the present application, the network communication between the host computer and the image acquisition board is based on the application layer protocol of the GigE Vision standard UDP transmission protocol. GigE Vision is a standard network protocol on Gigabit Ethernet. Figure 2 For the network communication schematic diagram of a network protocol provided by the embodiments of the present application, as Figure 2 shown, the network communication work is divided into 7 layers, namely the physical layer, data link layer, network layer, transport layer, session layer, presentation layer, and application layer. The UDP protocol is the most basic communication protocol currently used in the network, and each operating system has implemented the UDP protocol stack. According to different programming languages and different operating systems, system designers have provided different programming interfaces to develop software to achieve data interconnection and interoperability. The GigE Vision extended GVCP control protocol is used for conventional control instruction interaction, and the extended GVSP stream transmission protocol is used for the transmission of image data streams. The communication protocol message structure of the user data part of the extended GVCP control protocol request instruction mainly includes two parts, an instruction header and data, and this instruction can be expressed as:

[0077] struct TRequestData

[0078] {

[0079] / / Instruction header:

[0080] struct T_CommandHeader{

[0081] uint8_t Fixed0x42 = 0x42;

[0082] uint8_t Flag;

[0083] uint16_t Command;

[0084] uint16_t Length;

[0085] uint16_t ReqId;

[0086] };

[0087] / / Data

[0088] uint8_t* data;

[0089] }。

[0090] The communication protocol message structure of the user data part of the extended GVCP control protocol request instruction mainly includes two parts, an acknowledgment header and data, and this instruction can be expressed as:

[0091] struct TRequestData

[0092] {

[0093] / / Acknowledgment header:

[0094] struct T_AcknowledgeHeader{

[0095] uint16_t Status;

[0096] uint16_t Acknowledge;

[0097] uint16_t Length;

[0098] uint16_t AckId;

[0099] };

[0100] / / Data

[0101] uint8_t* data;

[0102] }。

[0103] In the embodiments of the present application, the extended GVSP stream transmission protocol for image data stream transmission includes three protocol packet frame formats: a header packet frame, an intermediate packet frame, and a tail packet frame.

[0104] In the embodiments of the present application, the basic information may include: information such as image width, image height, image format and mode, frame sequence number, and timestamps of the frame header and frame tail.

[0105] In the embodiments of the present application, the header packet frame mainly includes content describing the basic information of the image.

[0106] Step S1032: Divide the retrieved image frames into multiple pieces of image data based on the load data of the transmission bandwidth between the semiconductor test equipment and the host computer, compress the multiple pieces of image data to generate intermediate packet frames, and send the intermediate packet frames to the host computer. When one piece of image data is compressed to generate one intermediate packet frame, it is immediately sent to the host computer.

[0107] In the embodiments of the present application, the transmission bandwidth between the image acquisition card and the host computer can be regarded as the transport layer. The image can be divided into several frames according to the load data size of the transport layer. The intermediate packet frame mainly includes: packet sequence number and image data.

[0108] In the embodiments of the present application, before transmission, the image frames can be losslessly compressed in the byte storage format transmitted on the line in alignment with the maximum MTU bytes in a pixel-aligned manner. At the same time, the stored image format needs to be right-shifted and aligned according to pixels according to the platform. Streaming transmission transmits one packet immediately after one packet is processed by the algorithm, minimizing the idle time between algorithm processing and transmission. The host computer program end decompresses and restores the transmitted data to the original data image.

[0109] Figure 3 It is a schematic diagram of the implementation process of a transmission method provided by the embodiments of the present application. As Figure 3 shown, in Figure 3 , after each data packet is packed, one data packet is transmitted.

[0110] In the embodiments of the present application, lossless compression processing mainly improves the bandwidth utilization rate by reducing the transmission of invalid data. At the transmission end, the algorithm processes and packs while transmitting, which can make full use of the concurrent capabilities of the multi-core CPU. Compared with transmitting after the entire picture is processed, it reduces the waiting time during the algorithm processing and improves the transmission efficiency.

[0111] Step S1033: After multiple intermediate packet frames are transmitted, generate a tail packet frame and send the tail packet frame to the host computer.

[0112] In the embodiments of the present application, the tail packet frame mainly indicates the end of the transmission of one piece of image data.

[0113] In the embodiments of the present application, the data stream packet frame protocol message structure mainly includes two parts, a header and data. Exemplarily, the structure of the data stream packet frame protocol message can be expressed as:

[0114] struct TStreamData{

[0115] / / Header

[0116] struct T_PacketHeader{

[0117] uint16_t Status{0};

[0118] uint16_t Flag{0};

[0119] uint8_t EI{0};

[0120] E_GvspPacketFormat PacketFormat{(E_GvspPacketFormat)0};

[0121] uint64_t BlockId{0};

[0122] uint32_t PacketId{0}; / / Packet sequence number

[0123] };

[0124] / / Data

[0125] uint8_t* data;

[0126] }。

[0127] In the embodiments of this application, the host computer application program can start a data receiving thread to receive the image data packets uploaded by the semiconductor test equipment, and restore the data of the image network data packets by parsing according to the three protocol packet frame formats of the custom extended GVSP stream transmission header packet frame, middle packet frame, and tail packet frame and the data stream packet frame protocol message, so as to obtain a complete image one by one.

[0128] In the embodiments of the present application, it is assumed that the user needs to obtain 32 consecutive frame images due to algorithm testing requirements. To reduce the bandwidth load of continuous real-time transmission, the host computer application program will set the on-demand request transmission mode instruction to be packed with GVCP communication protocol packets and sent to the image acquisition card application program through network communication. When a packed user data passes through each layer, different protocol headers will be added by the software protocol stack, and an Ethernet frame header and frame tail will be added at the network interface layer. Finally, it is sent to the image acquisition card application program by the host computer physical network card. After the image acquisition card physical network card receives the data, the protocol headers are removed layer by layer in reverse, and finally the original user data is obtained. The data received by the host computer program during the receiving process is the reverse operation of the sending process. After receiving the instruction, the image acquisition card application program pauses the real-time continuous acquisition and transmission thread, and after the host computer instruction is processed, an ACK is sent back to the host computer test program to confirm the result of the instruction execution. Similarly, then the host computer application program issues a frame number control instruction for acquiring continuous image frames through GVCP. After receiving the instruction, the image acquisition card application program reallocates the maximum custom lock-free circular image acquisition buffer queue according to the hardware performance.

[0129] In some embodiments, step S102 can be implemented through the following steps:

[0130] Step S1021, obtain the current write index of the lock-free circular image acquisition buffer queue.

[0131] In the embodiments of the present application, two variables, WriteIdx and ReadIdx, are used as the enqueue index and dequeue index respectively for the lock-free circular image acquisition buffer queue. Figure 4 This is a schematic diagram of the index of a lock-free circular image acquisition buffer queue provided by the embodiments of the present application. As Figure 4 shown, when enqueuing n frame images, the WriteIdx variable is incremented by n. When dequeuing k data, the ReadIdx variable is incremented by k. out is not allowed to be greater than WriteIdx (when ReadIdx is equal to WriteIdx, it means the fifo is empty). WriteIdx is not allowed to be more than the fifo space larger than ReadIdx (for example, in the following figure, WriteIdx can be at most 8 more than ReadIdx, which means the fifo is full at this time).

[0132] In the embodiments of the present application, the current write index of the lock-free circular image acquisition buffer queue can be obtained.

[0133] Step S1022, calculate the next write index based on the current write index.

[0134] Step S1023, determine whether the lock-free circular image acquisition buffer queue is full based on the write index.

[0135] Step S1024, when the lock-free circular image acquisition buffer queue is not full, determine the transmission state of the acquisition thread.

[0136] In the embodiment of the present application, the value of WriteIdx can be set to the number of elements included in the current PduList. Move the write index to the end of the lock-free circular image acquisition buffer queue to prepare to add new data at the end of the lock-free circular image acquisition buffer queue.

[0137] In the embodiment of the present application, an object with the status of Empty can be searched for. If found, return immediately. If not found, then check the object with the status of Read, continue to search for the object with the status of Full, and finally if no object that meets the conditions is found, return nullptr, thereby determining that the buffer queue is not full.

[0138] In the embodiment of the present application, to search for whether the lock-free circular image acquisition buffer queue is not full, it can be implemented through the following code:

[0139] Search for an object in the writable lock-free circular image acquisition buffer queue that meets the following conditions, and perform the search based on the priority. The priority can include:

[0140] Level 1 (Empty): ReadIndx = WriteIdx = 0. (Basically, this situation only exists when the object is just created).

[0141] Level 2 (Read): ReadIndx >= PduSize, WriteIdx >= PduSize. (The memory data has been sent).

[0142] Level 3 (Full): ReadIndx = 0, WriteIdx >= PduSize. (The memory data has not started to be sent).

[0143] In the embodiment of the present application, if an object that meets the conditions is found, the lock-free circular image acquisition buffer queue is not full. If no object that meets the conditions is found, the lock-free circular image acquisition buffer queue is full.

[0144] Step S1025, determine the priority of storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the transmission state and the frame sequence number of the acquired image frame, and store the acquired image frame in the lock-free circular image acquisition buffer queue based on the priority.

[0145] In the embodiment of the present application, if there are multiple image frames to be written, the storage priority can be determined, and the acquired image frames are stored in the lock-free circular image acquisition buffer queue based on the priority.

[0146] In the embodiments of the present application, when the transmission status of the acquisition thread is transmission closed, the size relationship of the sequence numbers of the acquired image frames is determined, the priority for storing in the lock-free circular image acquisition buffer queue is determined based on the size relationship, and the acquired image frames are stored in the lock-free circular image acquisition buffer queue based on the priority, where the smaller the sequence number of the acquired image frame, the higher the priority of the corresponding image frame.

[0147] In the embodiments of the present application, since the smaller the sequence number of the acquired image frame, the higher the priority of the corresponding image frame, the buffer in the cache is as close as possible to the latest frame output by the sensor. All image frames ImageFrame can be traversed, and the ImageFrame with the smallest frame sequence number can be found.

[0148] When the transmission status of the acquisition thread is transmission open, the size relationship of the sequence numbers of the acquired image frames is determined, the priority for storing in the lock-free circular image acquisition buffer queue is determined based on the size relationship, and the acquired image frames are stored in the lock-free circular image acquisition buffer queue based on the priority, where the larger the sequence number of the acquired image frame, the higher the priority of the corresponding image frame.

[0149] In the embodiments of the present application, the higher the priority of the image frame with the larger sequence number of the acquired image frame can stagger the selection strategy of the transmission thread.

[0150] The method provided by the embodiments of the present application can store image frames in a lock-free circular image acquisition buffer queue.

[0151] In some embodiments, step S102 further includes:

[0152] Step S1026, obtaining the readable image frames in the lock-free circular image acquisition buffer queue.

[0153] Step S1027, in the case of multiple readable image frames, selecting the image with the smallest frame sequence number and transmitting the image with the smallest frame sequence number.

[0154] In the embodiments of the present application, by selecting the image with the smallest frame sequence number and transmitting the image with the smallest frame sequence number in the case of multiple readable image frames, frame loss can be ensured not to occur.

[0155] In some embodiments, after obtaining the image data in the lock-free circular image acquisition buffer queue, the write index can be set to zero to ensure that data can be rewritten.

[0156] In some embodiments, before step S101, the method further includes:

[0157] Step S1011, when receiving the instruction of the on-demand request transmission mode sent by the host computer, pause the real-time continuous acquisition and transmission mode and enter the on-demand request transmission mode. Among them, when entering the on-demand request transmission mode and receiving the instruction of acquiring continuous image frames sent by the host computer, determine a lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor test equipment, and start an acquisition thread to acquire image frames.

[0158] In the embodiments of the present application, the default working mode of the image acquisition card can be the real-time continuous acquisition and transmission mode. By switching the working mode, the processing load of the CPU can be reduced.

[0159] In some embodiments, after step S103, the method further includes:

[0160] Step S104, when receiving the retransmission instruction of the specified data packet sent by the host computer, retransmit the instruction data packet based on the sequence number of the specified data packet in the retransmission instruction. Among them, the host computer can determine the sequence number of the lost specified data packet based on the head packet frame, middle packet frame, and tail packet frame corresponding to the image frame sent by the semiconductor test equipment, and generate the retransmission instruction of the specified data packet based on the sequence number of the specified data packet.

[0161] In the embodiments of the present application, packet loss occasionally occurs during the transmission process due to unstable hardware connection of the bandwidth load. The host computer knows the sequence number of the lost packet by parsing the data stream packet frame protocol message, and notifies the semiconductor test equipment to retransmit the specified data packet, reducing the retransmission of the entire frame of image data, and improving the bandwidth for the transmission of effective frame data on the link to ensure the continuity of the image frame data.

[0162] In the embodiments of the present application, after the host computer receives a complete image frame, it performs continuous frame card control judgment through the frame sequence number parsed from the custom data structure head packet frame. If the frame sequence numbers are not continuous, it restarts the acquisition count according to the current latest frame sequence number until the acquisition of the specified continuous number of frames requested by the user is completed. After the acquisition is completed, the host computer application stops the receiving thread, and notifies the image semiconductor test equipment to stop the acquisition and transmission thread through the GVCP control protocol message, and restores the on-demand request transmission policy to the real-time acquisition and transmission policy. The card control mechanism based on the uploaded frame sequence numbers strictly ensures the continuity of the image frames from acquisition to transmission and then to testing, and ensures the accuracy of the algorithm test for image quality detection in image quality detection.

[0163] Based on the foregoing embodiments, the embodiments of the present application further provide an image acquisition method. Figure 5 It is a schematic flowchart of an image acquisition method provided by the embodiments of the present application, as Figure 5As shown in the figure, it includes: the network communication between the host computer application program and the image acquisition board application program is an application layer protocol based on the UDP transmission protocol of the GigE Vision standard. The extended GVCP control protocol is used for the interaction of general control instructions, and the extended GVSP stream transmission protocol is used for the transmission of image data streams. First, the host computer application program issues an on-demand request transmission instruction through GVCP. After receiving the instruction, the image acquisition card application program pauses the real-time continuous acquisition and transmission thread. Then, the host computer application program issues an instruction for the number of consecutive image frames to be acquired. After receiving the instruction, the image acquisition card application program reallocates the maximum custom lock-free circular image acquisition buffer queue according to the hardware performance, and starts the acquisition thread to perform continuous image frame acquisition operations. The image data collected by the acquisition thread is enqueued into the lock-free circular image buffer queue. At the same time, a transmission thread is started to dequeue the acquired images from the lock-free circular image buffer queue. The dequeued image frame data is transmitted to the host computer application program through GVSP in a streaming transmission scheme of lossless compression while packing. Then, the host computer application program starts a data reception thread to receive the network data packets uploaded by the image acquisition card application program, and parses the data of the network data packets according to the custom data structure. If packet loss occurs during the transmission process, the host computer program notifies the image acquisition card application program to retransmit the lost network data packets until the application program parses the complete image frame data. In this way, the remaining image data is received in a loop. After receiving the complete image frame data, continuous frame judgment is performed based on the frame sequence number parsed in the custom data structure. If the frame sequence numbers are not consecutive, the reception count is restarted according to the current latest frame sequence number until the number of consecutive frames requested by the user is received. After the reception is completed, the host computer application program stops the reception thread, and the image acquisition card application program stops the acquisition and transmission thread, and restores the on-demand request transmission policy to the real-time acquisition and transmission policy.

[0164] The method provided by the embodiment of the present application allocates the maximum custom lock-free circular image buffer queue according to the hardware performance to solve the communication problem of the mismatch between the acquisition and transmission buffer speeds. In terms of transmission, a streaming transmission scheme of lossless compression while packing is adopted using a custom data structure. During the network transmission process, the lost data packets are confirmed according to the header information of the custom data structure and retransmitted in parallel. Through the cooperation of the above three mechanism methods at the acquisition and transmission levels, while ensuring the acquisition and transmission rate performance, the continuity of the image frame data is also taken into account.

[0165] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0166] According to the foregoing embodiments, an image acquisition device is provided in an embodiment of the present application. Each module included in the device, as well as each unit included in each module, may be implemented by a processor in a computer device; of course, it may also be implemented by specific logic circuits. During implementation, the processor may be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing), or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0167] An embodiment of the present application provides an image acquisition device. Figure 6 FIG. is a schematic structural diagram of an image acquisition device provided in an embodiment of the present application. As Figure 6 shown, the image acquisition device 600 includes:

[0168] A determination module 601, configured to, when an instruction to acquire consecutive image frames sent by a host computer is obtained, determine a lock-free circular image acquisition buffer queue based on the hardware information of the semiconductor test device, and start an acquisition thread to perform image frame acquisition;

[0169] A buffer module 602, configured to, when an image frame is acquired, store the acquired image frame in the lock-free circular image acquisition buffer queue, and take out an image frame from the lock-free circular image acquisition buffer queue;

[0170] A sending module 603, configured to send the taken-out image frame to the host computer.

[0171] In some embodiments, the sending module includes:

[0172] A first sending unit, configured to determine basic information of the taken-out image frame, generate a header packet frame based on the basic information, and send the header packet frame to the host computer;

[0173] A second sending unit, configured to divide the taken-out image frame into multiple image data based on load data of the transmission bandwidth between the semiconductor test device and the host computer, compress the multiple image data to generate intermediate packet frames, and send the intermediate packet frames to the host computer, where, when one image data is compressed to generate one intermediate packet frame, it is immediately sent to the host computer;

[0174] A third sending unit, configured to generate a tail packet frame after the transmission of multiple intermediate packet frames is completed, and send the tail packet frame to the host computer.

[0175] In some embodiments, the buffer module includes:

[0176] A first acquisition unit, configured to acquire a current write index of the lock-free circular image acquisition buffer queue;

[0177] A calculation unit, configured to calculate a next write index based on the current write index;

[0178] A determination unit, configured to determine whether the lock-free circular image acquisition buffer queue is full based on the write index;

[0179] A status determination unit, configured to determine a transmission status of an acquisition thread when the lock-free circular image acquisition buffer queue is not full;

[0180] A storage unit, configured to determine a priority of storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the transmission status and a frame sequence number of the acquired image frame, and store the acquired image frame in the lock-free circular image acquisition buffer queue based on the priority.

[0181] In some embodiments, the determining the priority of storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the transmission status and the frame sequence number of the acquired image frame, and storing the acquired image frame in the lock-free circular image acquisition buffer queue based on the priority includes:

[0182] When the transmission status of the acquisition thread is transmission closed, determining a magnitude relationship of sequence numbers of the acquired image frames, determining a priority of storing in the lock-free circular image acquisition buffer queue based on the magnitude relationship, and storing the acquired image frames in the lock-free circular image acquisition buffer queue based on the priority, where the smaller the sequence number of the acquired image frame, the higher the priority of the corresponding image frame;

[0183] When the transmission status of the acquisition thread is transmission open, determining a magnitude relationship of sequence numbers of the acquired image frames, determining a priority of storing in the lock-free circular image acquisition buffer queue based on the magnitude relationship, and storing the acquired image frames in the lock-free circular image acquisition buffer queue based on the priority, where the larger the sequence number of the acquired image frame, the higher the priority of the corresponding image frame.

[0184] In some embodiments, the buffer module further includes:

[0185] A second acquisition unit, configured to acquire readable image frames in the lock-free circular image acquisition buffer queue;

[0186] A selection unit, configured to select an image with the smallest frame sequence number and transmit the image with the smallest frame sequence number when there are multiple readable image frames.

[0187] In some embodiments, the image acquisition device further includes:

[0188] A pause module, configured to pause the real-time continuous acquisition and transmission mode and enter the on-demand request transmission mode when receiving an instruction for the on-demand request transmission mode sent by the host computer. Wherein, when entering the on-demand request transmission mode and receiving an instruction for acquiring continuous image frames sent by the host computer, a lock-free circular image acquisition buffer queue is determined based on the hardware information of the semiconductor test equipment, and an acquisition thread is started to acquire image frames.

[0189] In some embodiments, the image acquisition device further includes:

[0190] A retransmission module, configured to retransmit the instruction data packet based on the sequence number of the specified data packet in the retransmission instruction when receiving a retransmission instruction for the specified data packet sent by the host computer. Wherein, the host computer can determine the sequence number of the lost specified data packet based on the head packet frame, middle packet frame, and tail packet frame corresponding to the image frame sent by the semiconductor test equipment, and generate a retransmission instruction for the specified data packet based on the sequence number of the specified data packet.

[0191] It should be noted that, for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought thereby can be specifically referred to the method embodiment part, and will not be elaborated herein.

[0192] In addition, Figure 6 The shown image acquisition device can be a software unit, a hardware unit, or a unit combining software and hardware. It can also be integrated into the semiconductor test equipment as an independent pendant, or exist as an independent terminal device.

[0193] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments and will not be elaborated herein.

[0194] Figure 7This is a schematic structural diagram of the semiconductor testing device provided by the embodiments of the present application. As Figure 7 shown, the semiconductor testing device 3 of this embodiment may include: at least one processor 30 ( Figure 7 only one processor 30 is shown in Figure 1 ), a memory 31, and a computer program 32 stored in the memory 31 and executable on at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above method embodiments are implemented, such as Figure 6 the steps S101 to S103 in the embodiment shown in

[0195] . Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above device embodiments are implemented, such as Figure 6 the functions of the modules 601 to 603 shown in

[0195] .

[0195] Exemplarily, the computer program 32 may be divided into one or more modules / units. One or more modules / units are stored in the memory 31 and executed by the processor 30 to complete the present application. One or more modules / units may be a series of computer program 32 instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 32 in the semiconductor testing device 3.

[0196] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program 32, and when the computer program 32 is executed by the processor 30, the steps in the above method embodiments can be implemented.

[0197] The embodiments of the present application provide a computer program product. When the computer program product runs on the semiconductor testing device, it enables the vehicle to implement the steps in the above method embodiments when executed.

[0198] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, a computer program 32 can be used to instruct relevant hardware to complete. The computer program 32 can be stored in a computer-readable storage medium. When the computer program 32 is executed by a processor 30, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program 32 includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the terminal, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0199] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0200] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0201] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical or other forms.

[0202] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0203] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image acquisition method, characterized in that: Used in semiconductor test equipment, including: When an instruction to capture continuous image frames is obtained from a host computer, a lock-free annular image capture cache queue is determined based on hardware information of the semiconductor test equipment, and a capture thread is started to capture image frames; When an image frame is acquired, the acquired image frame is stored in a lock-free annular image acquisition buffer queue, and the image frame is taken out from the lock-free annular image acquisition buffer queue; The extracted image frame is sent to the host computer.

2. The method according to claim 1, characterized in that: The step of sending the retrieved image frame to the host computer comprises: Determine basic information of the extracted image frame, generate a header packet frame based on the basic information, and send the header packet frame to the host computer; Dividing the extracted image frame into a plurality of image data based on the load data of the transmission bandwidth between the semiconductor test equipment and the host computer, compressing the plurality of image data to generate an intermediate packet frame, and sending the intermediate packet frame to the host computer, wherein when one image data is compressed to generate an intermediate packet frame, it is immediately sent to the host computer; After the transmission of multiple intermediate packet frames is completed, a tail packet frame is generated, and the tail packet frame is sent to the host computer.

3. The method according to claim 1, characterized in that: The step of storing the acquired image frames in the lock-free annular image acquisition buffer queue comprises: Obtaining the current write index of the lock-free annular image acquisition buffer queue; Based on the current write index, calculate the next write index; Determining whether the lock-free annular image acquisition buffer queue is full based on the write index; When the lock-free annular image acquisition buffer queue is not full, determining the transmission state of the acquisition thread; The priority of storing the image frame in the lock-free annular image acquisition buffer queue is determined based on the transmission status and the frame sequence number of the acquired image frame, and the acquired image frame is stored in the lock-free annular image acquisition buffer queue based on the priority.

4. The method according to claim 3, characterized in that The step of determining the priority of storing the image frame in the lock-free annular image acquisition cache queue based on the transmission state and the frame sequence number of the acquired image frame, and storing the acquired image frame in the lock-free annular image acquisition cache queue based on the priority includes: When the transmission state of the acquisition thread is transmission closed, determining the size relationship of the sequence numbers of the acquired image frames, determining the priority of storing in the lock-free annular image acquisition cache queue based on the size relationship, and storing the acquired image frames in the lock-free annular image acquisition cache queue based on the priority, wherein the smaller the sequence number of the acquired image frame, the higher the priority of the corresponding image frame; When the transmission status of the acquisition thread is transmission on, the size relationship of the serial numbers of the acquired image frames is determined, and the priority of storing in the lock-free annular image acquisition cache queue is determined based on the size relationship. The acquired image frames are stored in the lock-free annular image acquisition cache queue based on the priority, wherein the larger the serial number of the acquired image frame, the higher the priority of the corresponding image frame.

5. The method according to claim 4, characterized in that The step of taking out the image frame from the lock-free annular image acquisition buffer queue further includes: Obtaining readable image frames in the lock-free annular image acquisition buffer queue; When there are multiple readable image frames, the image with the smallest frame number is selected and transmitted.

6. The method according to claim 1, characterized in that The method further comprises: When an instruction for the on-demand request transmission mode is obtained from the host computer, the real-time continuous acquisition transmission mode is suspended and the on-demand request transmission mode is entered, wherein, when entering the on-demand request transmission mode and when an instruction for acquiring continuous image frames is obtained from the host computer, a lock-free circular image acquisition cache queue is determined based on the hardware information of the semiconductor testing equipment, and an acquisition thread is started to perform image frame acquisition.

7. The method according to claim 2, characterized in that The method further comprises: When a retransmission instruction of a specified data packet sent by a host computer is obtained, the instruction data packet is retransmitted based on the sequence number of the specified data packet in the retransmission instruction, wherein the host computer can determine the sequence number of the lost specified data packet based on the head packet frame, middle packet frame and tail packet frame corresponding to the image frame sent by the semiconductor testing equipment, and generate a retransmission instruction of the specified data packet based on the sequence number of the specified data packet.

8. An image acquisition device, characterized in that: include: A determination module, configured to determine a lock-free annular image acquisition cache queue based on hardware information of the semiconductor test equipment and start an acquisition thread to acquire image frames when an instruction to acquire continuous image frames is obtained from a host computer; A cache module, used for storing the acquired image frames in a lock-free annular image acquisition cache queue when the image frames are acquired, and taking out the image frames from the lock-free annular image acquisition cache queue; The sending module is used to send the extracted image frame to the host computer.

9. A semiconductor testing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images