Image processing apparatus and method

Abstract

The application discloses an image processing device and method, and relates to the technical field of image processing. The image processing device of the embodiment of the application comprises an image acquisition module, a chip detection module and a display module; the image acquisition module is configured to acquire an image of a chip, and transmit the image to the chip detection module and the display module; the chip detection module is configured to receive the image, acquire position information of the chip according to the image, and transmit the position information to the display module; and the display module is configured to receive the image and the position information, mark a target region on the image according to the position information, cut out a target region image from the image, and acquire a golden sample. The embodiment of the application can synchronously present the image captured by a camera in real time and a target region containing a chip on a display screen, facilitate cutting out the target region containing the chip from the image captured by the camera, and improve the efficiency of making the golden sample.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing apparatus and method. Background Technology

[0002] After packaging, integrated circuit (IC) chips must undergo rigorous testing to ensure product quality. Chip appearance inspection is an essential and crucial step that directly impacts IC product quality and the smooth operation of subsequent production processes. When using artificial intelligence (AI) technology to inspect chip appearance, it's first necessary to create sample images, often referred to as "gold standard images," for AI model training or image comparison. Currently, creating gold standard images typically requires manually adjusting the chip to the appropriate position and then performing image processing on the target area containing the chip; this process is tedious and time-consuming. Summary of the Invention

[0003] In view of this, this application provides an image processing apparatus and method that can simultaneously display images captured by a camera in real time and a target area containing a chip on a display screen, making it easier to crop the target area containing the chip from the image captured by the camera and improving the efficiency of making gold samples.

[0004] An image processing apparatus according to an embodiment of this application includes an image acquisition module, a chip detection module, and a display module. The display module is connected to the image acquisition module and the chip detection module, and the image acquisition module is connected to the chip detection module. The image acquisition module is configured to: acquire an image of the chip and transmit the image to the chip detection module and the display module; the chip detection module is configured to: receive the image, acquire the location information of the chip based on the image, and transmit the location information to the display module; the display module is configured to: receive the image and the location information, mark a target region on the image based on the location information, and crop the target region image from the image to obtain a gold sample; wherein, the target region refers to the image area covered by the chip.

[0005] In one embodiment, the image acquisition module is configured to transmit the image via the WebSocket protocol.

[0006] In one embodiment, the image acquisition module is configured to: determine whether the connection between the image acquisition module and the chip detection module or the display module is valid before transmitting the image via the WebSocket protocol; if the connection is determined to be invalid, interrupt the image transmission and re-check the connection or notify the user of the connection error.

[0007] In one embodiment, the image acquisition module is configured to preprocess the image before transmitting it via the WebSocket protocol, the preprocessing including at least image scaling or image compression.

[0008] In one embodiment, the chip detection module is configured to transmit the location information via a server-sent event (SSE).

[0009] In one embodiment, the chip detection module is configured to: acquire features of the chip; determine whether the features exist in the image; if the features are determined to exist in the image, mark the position of the chip in the image to acquire the position information.

[0010] In one implementation, the display module is configured to send a request via a RESTful API to receive the image.

[0011] An image processing method according to an embodiment of this application includes: acquiring an image of a chip; acquiring the location information of the chip based on the image; marking a target region on the image based on the location information, wherein the target region refers to the image area covered by the chip; and cropping the target region image from the image to obtain a gold sample.

[0012] In one embodiment, before obtaining the chip's location information based on the image, the method further includes: preprocessing the image, the preprocessing including at least image scaling or image compression.

[0013] In one embodiment, obtaining the chip's location information based on the image includes: obtaining the chip's features; determining whether the features exist in the image; and if the features are determined to exist in the image, marking the chip's location in the image to obtain the location information.

[0014] This application embodiment obtains a gold sample by marking the target area on the image of the chip and cropping the target area image from the image. It can synchronously present the image captured by the camera in real time and the target area containing the chip on the display screen, which facilitates the cropping of the target area containing the chip from the image captured by the camera, improves the efficiency of making gold samples, and reduces the operation time of chip appearance inspection. Attached Figure Description

[0015] Figure 1 This is a block diagram of an image processing apparatus according to an embodiment of this application.

[0016] Figure 2 This is a flowchart of an image processing method according to an embodiment of this application.

[0017] Figure 3 This is a schematic diagram of an image display interface according to one embodiment of this application.

[0018] Figure 4 This is a block diagram of an image processing apparatus according to another embodiment of this application.

[0019] Figure 5 This is a flowchart of an image processing method according to another embodiment of this application.

[0020] Explanation of main component symbols

[0021] 10 Image processing device

[0022] 11 Image Acquisition Module

[0023] 12 Chip Detection Module

[0024] 13 Display Modules

[0025] 41 cameras Detailed Implementation

[0026] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Many specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only some embodiments of this application, and not all embodiments.

[0027] Figure 1 This is a block diagram of an image processing apparatus 10 according to an embodiment of this application.

[0028] Please see Figure 1 The image processing device 10 includes an image acquisition module 11, a chip detection module 12, and a display module 13. These modules can be connected via a bus or directly.

[0029] In this embodiment, the image acquisition module 11 is used to acquire an image of the chip and transmit the image to the chip detection module 12 and the display module 13. In one implementation, firstly, the image acquisition module 11 captures an image of the chip using a camera. Then, the image acquisition module 11 transmits the image to the chip detection module 12 and the display module 13 via the WebSocket protocol.

[0030] The WebSocket protocol refers to a full-duplex communication protocol based on the Transmission Control Protocol (TCP), which allows the server to actively push data to the client, realizing full-duplex communication between the browser and the server. In this embodiment, the image acquisition module 11 transmits the image via the WebSocket protocol, which can save server resources and bandwidth, and enables real-time communication.

[0031] In one embodiment, before transmitting the image via the WebSocket protocol, the image acquisition module 11 further determines whether its connection with the chip detection module 12 or the display module 13 is valid. For example, the image acquisition module 11 can parse the client's (i.e., the chip detection module 12 or the display module 13) identity information from the connection establishment parameters (e.g., client name or address) to determine whether the client is repeatedly connecting. If it is determined that the client is repeatedly connecting, it means that the client already has a valid connection, and this connection is invalid. The image acquisition module 11 interrupts this connection to avoid resource waste caused by repeated connections. If it is determined that the client is not repeatedly connecting, it means that this connection is valid, and the image acquisition module 11 maintains this connection.

[0032] In another implementation, the image acquisition module 11 can determine the validity of its connection with the client by detecting the camera's operating status. For example, if it is determined that the camera is not in a working state (e.g., the camera is not powered on or not connected to the network), it indicates that the connection is invalid. The image acquisition module 11 interrupts the connection and controls the camera to resume its working state (e.g., the camera is powered on again or reconnects to the network), or notifies the user that the camera is malfunctioning (i.e., the camera is not in a working state). If it is determined that the camera is in a working state, it indicates that the connection is valid, and the image acquisition module 11 maintains the connection.

[0033] In one embodiment, the image acquisition module 11 further preprocesses the image before transmitting it via the WebSocket protocol. The preprocessing includes, but is not limited to, image scaling or image compression. The image acquisition module 11 can scale the image according to a preset scaling factor.

[0034] For example, in one embodiment, the image acquisition module 11 captures a first image of the chip using a camera, the first image having a size of 2448*2048. The image acquisition module 11 scales the first image according to a preset scaling factor to obtain a second image. Assuming the scaling factor is 0.5*0.5, the second image would have a size of 1224*1024. In another embodiment, the image acquisition module 11 converts the image into a byte stream for compression, thereby reducing storage space and saving server resources.

[0035] In this embodiment, the chip detection module 12 is used to receive the image, detect the position of the chip from the image, and transmit the detected chip position information to the display module 13. In one implementation, the chip detection module 12 can detect the position of the chip from the image through feature recognition and transmit the position information to the display module 13 through a server-sent event (SSE).

[0036] Here, SSE refers to a one-way transmission channel based on the Hypertext Transfer Protocol (HTTP), which allows the server to push messages to the browser, and the webpage to automatically receive updates from the server. In this embodiment, the chip detection module 12 transmits location information to the display module 13 through SSE, enabling real-time synchronization of the location information to the display module 13.

[0037] Figure 2 This is a flowchart of an image processing method according to an embodiment of this application.

[0038] Please see Figure 2 The chip detection module 12 detects the chip's location from an image of the chip through feature recognition, and may include the following steps:

[0039] S21, acquire the characteristics of the chip.

[0040] In one embodiment, the chip detection module 12 acquires chip features through preset information or historical data. These features may include appearance features or identification features. Appearance features include, but are not limited to, at least one of shape, size, color, and texture. Identification features include, but are not limited to, at least one of text, numbers, letters, and graphic identifiers. For example, the chip detection module 12 may receive preset information input by a user, which includes chip features. Alternatively, the chip detection module 12 may also extract chip features from historical chip detection data.

[0041] S22, determine whether the feature exists in the image of the chip. If the feature exists in the image, proceed to step S23. If the feature does not exist in the image, the process ends.

[0042] In one embodiment, the chip detection module 12 determines whether the chip's features are present in the image by scanning the image. For example, the chip detection module 12 identifies the chip's model by scanning text, numbers, or letters in the image. If the chip detection module 12 identifies the chip's model from the image, it needs to mark the chip's location in the image. If the chip detection module 12 cannot identify the chip's model from the image, it indicates that the image does not meet the sample requirements, and another image can be received, or the image acquisition module 11 can be notified to reacquire the chip's image.

[0043] S23, mark the position of the chip in the image.

[0044] In one embodiment, the chip detection module 12 can establish a coordinate system based on the image, and then mark the position of the chip on the coordinate system. For example, the chip detection module 12 can establish a Cartesian coordinate system on the image with a pixel as the origin, two mutually perpendicular directions as the horizontal and vertical axes, and a fixed number of pixels as the unit spacing. Then, the chip detection module 12 can draw the outline of the chip on the Cartesian coordinate system and mark the coordinate points of the outline edges. The set of coordinate points can represent the position of the chip in the image.

[0045] In this embodiment, the chip detection module 12 is further configured to transmit position information to the display module 13. The position information may include the set of coordinate points and information about the Cartesian coordinate system.

[0046] In this embodiment, the display module 13 is used to receive the image and the position information, reconstruct the outline of the chip based on the position information, and display the outline of the chip on the image. It can be understood that by displaying the outline of the chip on the image, the display module 13 facilitates cropping a target area containing the chip from the image. The target area refers to the image area covered by the chip.

[0047] In one embodiment, after the display module 13 displays the outline of the chip on the image, it performs image cropping or notifies the user to crop the image to obtain a gold sample.

[0048] In another embodiment, the display module 13 can detect the chip's position from the chip's image through feature recognition to determine whether the chip is within the target area. If the chip is determined to be within the target area, the display module 13 can send a request to the image acquisition module 11 to acquire the image from the image acquisition module 11. If the chip is determined not to be within the target area, the display module 13 can send an instruction to the image acquisition module 11 to control the image acquisition module 11 to reacquire the image of the chip.

[0049] It is understood that the display module 13 detects the position of the chip from the chip image through feature recognition. For details, please refer to the description of steps S21-S23 above, which will not be repeated here.

[0050] Figure 3 This is a schematic diagram of an image display interface according to one embodiment of this application.

[0051] Please see Figure 3 The display module 13 synchronously displays the outline of a chip with the model number "abcd222" on the image. The image can be cropped along the outline of the chip, and the target area image containing the chip is retained after cropping. The target area image is the gold sample of the chip.

[0052] In one embodiment, the display module 13 sends a request to the image acquisition module 11 via a RESTful API to obtain an image of the chip.

[0053] The RESTful API, as mentioned here, refers to an Application Programming Interface (API) that conforms to the Representational State Transfer (REST) ​​principle. A RESTful API is a practice of the HTTP protocol. In a RESTful architecture, data and functionality are treated as resources, accessed using Uniform Resource Identifiers (URIs), and executed through predefined operations. Clients and servers exchange representations of resources through standardized interfaces and protocols (such as HTTP). For example, text resources can be represented in formats such as Hyper Text Markup Language (HTML), Extensible Markup Language (XML), or JavaScript Object Notation (JSON), while image resources can be represented in formats such as Portable Network Graphics (PNG) or Joint Photographic Experts Group (JPEG).

[0054] In this embodiment, the display module 13 sends a request to the image acquisition module 11 through a RESTful API. The image acquisition module 11 can receive the request through a unified response interface, without having to define different response interfaces for different request expressions.

[0055] Figure 4 This is a block diagram of an image processing apparatus 10 according to another embodiment of this application.

[0056] Please see Figure 4 The image acquisition module 11, the chip detection module 12, and the display module 13 are respectively mounted on three host computers, which are interconnected via a local area network. The image acquisition module 11 is also connected to a camera 41, and a tray for holding the chip is located below the lens of the camera 41. The display module 13 can simultaneously display the image captured by the camera 41 and the outline of the chip through a browser's user interface (UI), and crop the image along the outline of the chip to obtain a gold sample of the chip.

[0057] It is understood that the host can be any of a computer, tablet computer, server or workstation, or any of a microcontroller, processor or application-specific integrated circuit.

[0058] In this embodiment, the image acquisition module 11, the chip detection module 12, and the display module 13 are respectively mounted on three host computers, which can distribute computing resources and improve the efficiency of data processing.

[0059] Figure 5 This is a flowchart of an image processing method according to another embodiment of this application. It can be understood that... Figure 5 The specific implementation of the steps shown can be referred to the above description of the implementation of the image acquisition module 11, the chip detection module 12 and the display module 13, and will not be repeated here.

[0060] Please see Figure 5 The image processing method includes the following steps:

[0061] S501, the image acquisition module 11 determines whether its connection with the chip detection module 12 and the display module 13 is valid. If the connection is determined to be valid, step S502 is executed. If the connection is determined to be invalid, the process ends.

[0062] The image acquisition module 11 determines whether its connection with the chip detection module 12 and the display module 13 is valid. If the connection is valid, the image acquisition module 11 can acquire a first image and a second image of the chip and transmit the first image and the second image to the chip detection module 12 and the display module 13. If the connection is invalid, the image acquisition module 11 re-checks the connection or notifies the user of the connection error.

[0063] S502, the image acquisition module 11 acquires the first image of the chip.

[0064] The image acquisition module 11 can capture a first image of the chip using a camera, thereby acquiring the first image from the camera.

[0065] S503, the image acquisition module 11 preprocesses the first image to obtain the second image.

[0066] The preprocessing includes at least image scaling or image compression. Specifically, the image acquisition module 11 can scale the first image according to a preset scaling factor. Alternatively, the image acquisition module 11 can convert the first image into a byte stream for compression.

[0067] S504, the image acquisition module 11 transmits the second image via the WebSocket protocol.

[0068] The image acquisition module 11 can transmit the second image to the chip detection module 12 and the display module 13 via the WebSocket protocol.

[0069] S505, the chip detection module 12 obtains the position information of the chip from the second image.

[0070] The chip detection module 12 can detect the position of the chip from the second image through feature recognition, thereby obtaining the chip's position information.

[0071] S506, the chip detection module 12 transmits the chip's location information through a server-sent event (SSE).

[0072] The chip detection module 12 can transmit the chip's location information to the display module 13 via a server-sent event (SSE).

[0073] S507, the display module 13 marks the target area on the second image according to the position information.

[0074] The target area refers to the image area covered by the chip. The display module 13 can reconstruct the outline of the chip based on the position information and display the outline of the chip on the second image.

[0075] S508, the display module 13 displays the second image and the target area through a user interface (UI).

[0076] The display module 13 can simultaneously display the second image and the target area through the browser's user interface (UI).

[0077] S509, the display module 13 determines whether the chip is in the target area. If the chip is determined to be in the target area, step S510 is executed. If the chip is determined not to be in the target area, the process returns to step S502.

[0078] In one embodiment, the display module 13 can detect the position of the chip on the second image through feature recognition, thereby determining whether the chip is located in the target area. If the chip is determined to be in the target area, the display module 13 can send a request to the image acquisition module 11 to obtain the first image from the image acquisition module 11. If the chip is determined not to be in the target area, the display module 13 can send an instruction to the image acquisition module 11 to control the image acquisition module 11 to reacquire the first image of the chip.

[0079] S510, the display module 13 sends a request via a RESTful API to obtain the first image.

[0080] The display module 13 can send a request to the image acquisition module 11 via a RESTful API. In response to the request, the image acquisition module 11 transmits the first image to the display module 13.

[0081] S511, the display module 13 crops out the target area image from the first image to obtain the gold sample.

[0082] The target area image refers to the portion of the first image that covers the chip. The display module 13 can crop the first image along the outline of the chip, retaining the target area image after cropping. The target area image is the gold sample of the chip.

[0083] This embodiment of the application, through the information interaction between the image acquisition module 11, the chip detection module 12 and the display module 13, can synchronously present the image captured by the camera in real time and the target area containing the chip on the display screen, which makes it easier to cut out the target area containing the chip from the image captured by the camera, improves the efficiency of making gold samples, and reduces the operation time of chip appearance inspection.

[0084] It is understood that although a logical order is shown in the flowchart above, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. The methods disclosed in the embodiments of this application include one or more steps or actions for implementing the method. Method steps and / or actions may be interchanged with each other without departing from the scope of the claims.

[0085] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. An image processing apparatus, characterized in that, The image processing device includes an image acquisition module, a chip detection module, and a display module. The display module is connected to the image acquisition module and the chip detection module, and the image acquisition module is connected to the chip detection module. The image acquisition module is configured as follows: A first image of the chip is acquired, the first image is preprocessed to obtain a second image, and the second image is transmitted to the chip detection module and the display module via the WebSocket protocol; The chip detection module is configured as follows: The system receives the second image, obtains the position information of the chip based on the second image, and transmits the position information to the display module by sending an event SSE through the server. The display module is configured as follows: The system receives the second image and the location information, marks the target area on the second image according to the location information, displays the second image and the target area through the user interface (UI), and when it is determined that the chip is in the target area, sends a request to the image acquisition module through a RESTful API to obtain the first image, and crops the target area image from the first image to obtain the gold sample; wherein, the target area refers to the image area covered by the chip.

2. The image processing apparatus as claimed in claim 1, characterized in that, The image acquisition module is configured as follows: Before transmitting the image via the WebSocket protocol, it is determined whether the connection between the image acquisition module and the chip detection module or the display module is valid; If the connection is determined to be invalid, image transmission is interrupted, and the connection is rechecked or the user is notified of the connection error.

3. The image processing apparatus as described in claim 1 or 2, characterized in that, The preprocessing includes at least image scaling or image compression.

4. The image processing apparatus as described in claim 1 or 2, characterized in that, The chip detection module is configured as follows: Obtain the features of the chip; Determine whether the feature exists in the second image; If the feature is determined to exist in the second image, the position of the chip in the second image is marked to obtain the position information.

5. An image processing method, characterized in that, The method includes: Acquire the first image of the chip; The first image is preprocessed to obtain the second image; The location information of the chip is obtained based on the second image; Based on the location information, a target region is marked on the second image, where the target region refers to the image area covered by the chip; The second image and the target area are displayed through a user interface (UI). When it is determined that the chip is located in the target area, the target area image is cropped from the first image to obtain a gold sample.

6. The image processing method as described in claim 5, characterized in that, The preprocessing includes at least image scaling or image compression.

7. The image processing method as described in claim 5 or 6, characterized in that, The step of obtaining the chip's location information based on the second image includes: Obtain the features of the chip; Determine whether the feature exists in the second image; If the feature is determined to exist in the second image, the position of the chip in the second image is marked to obtain the position information.

Images