Target identification method and device and computer storage medium
By identifying the marking positions of auxiliary pointing marks in the microscopic system, the problem of difficulty in identifying microstructure targets is solved, and fast and accurate target positioning and recognition is achieved.
Patent Information
- Application Number
- CN202411400055.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2025-02-18
- Estimated Expiration
- Not applicable · inactive patent
Smart Images
Figure CN119478026A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of optical technology, and in particular to a target recognition method, device and computer storage medium. Background Art
[0002] When some tiny structures such as devices, alignment marks or research samples that are very small in themselves and require empty areas around them are characterized in the microscope system, or when the coarse alignment accuracy of the alignment system in the overlay scene is insufficient or temporarily fails, the field of view of the microscope imaging is very small, and finding samples or target alignment marks is often like looking for a needle in a haystack. If the spiral trial search method is used, although finding the sample is not a problem, it often takes a lot of time. At this time, the introduction of auxiliary pointing marks can greatly shorten the necessity of searching and reduce the time required to find the sample. Some common auxiliary pointing marks, such as arrows, coordinate numbers, and symbol text, often have more complex structures, multiple topologically hollow structures, or multiple small-sized isolated graphics. Complex structures such as arrows have more oblique lines, so when exposing such graphics, the difficulty of mask-light source optimization is often more difficult than horizontal and vertical graphics. Topological hollow structures such as the number 0 or the number 8 are not easy to fall off during peeling after exposure and development, evaporation or deposition. Once they do not fall off, the internal details cannot be distinguished, which easily causes recognition difficulties. Small-sized isolated graphics, such as Braille, Morse code dots and dashes, etc., are easy to fall off or lose integrity after washing, corrosion and wear during various processes, causing difficulty in identification. Summary of the invention
[0003] The present application provides a target recognition method, device and computer storage medium.
[0004] In order to solve the above technical problems, the present application proposes a target recognition method, which includes: obtaining a target recognition image for a target to be identified and an auxiliary pointing mark graphic, wherein the relative position of the target to be identified and the auxiliary pointing mark graphic is fixed; in the target recognition image, obtaining the marking position of any auxiliary pointing mark in the auxiliary pointing mark graphic; based on the auxiliary pointing information of the marking position, obtaining the target position of the target to be identified.
[0005] Wherein, each auxiliary pointing mark in the auxiliary pointing mark pattern has different transverse pointing marks and / or longitudinal pointing marks.
[0006] Among them, the method of obtaining the marking position of any auxiliary pointing mark in the auxiliary pointing mark graphic includes: obtaining the target transverse pointing mark and the target longitudinal pointing mark of the target auxiliary pointing mark in the auxiliary pointing mark graphic; determining the coordinate quadrant of the target auxiliary pointing mark based on the extension direction of the target transverse pointing mark and the extension direction of the target longitudinal pointing mark; determining the transverse coordinate difference based on the extension length of the target transverse pointing mark; determining the longitudinal coordinate difference based on the extension length of the target longitudinal pointing mark; determining the marking position of the target auxiliary pointing mark based on the transverse coordinate difference, the longitudinal coordinate difference, and the coordinate quadrant.
[0007] Wherein, determining the lateral coordinate difference based on the extension length of the target lateral pointing mark includes: determining the number of lateral grids based on the extension length of the target lateral pointing mark; and determining the lateral coordinate difference based on the number of lateral grids and a preset mark interval.
[0008] Wherein, before obtaining the target transverse pointing mark and the target longitudinal pointing mark of the target auxiliary pointing mark in the auxiliary pointing mark graphic, the target identification method also includes: obtaining a candidate transverse pointing mark and a candidate longitudinal pointing mark of at least two candidate auxiliary pointing marks in the auxiliary pointing mark graphic; and taking the candidate auxiliary pointing mark corresponding to the minimum value of the sum of the extended length of the candidate transverse pointing mark and the extended length of the candidate longitudinal pointing mark as the target auxiliary pointing mark.
[0009] Wherein, the obtaining of the marking position of any one auxiliary pointing mark in the auxiliary pointing mark graph includes: obtaining the coding information of the target auxiliary pointing mark in the auxiliary pointing mark graph; and determining the marking position of the target auxiliary pointing mark based on the coding information.
[0010] Wherein, the pixel points of the auxiliary pointing mark in the auxiliary pointing mark graphic are set in a discrete manner.
[0011] Among them, after obtaining the target position of the target to be identified based on the auxiliary pointing information of the mark position, the target identification method also includes: obtaining a movement instruction using the target position and the mark position; moving the target to be identified according to the movement instruction, and re-identifying it.
[0012] In order to solve the above technical problems, the present application proposes a target recognition device based on a tiny structure, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above target recognition method.
[0013] In order to solve the above technical problems, the present application proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above target identification method.
[0014] Different from the prior art, the beneficial effect of the present application is that: the target recognition device obtains a target recognition image for the target to be recognized and the auxiliary pointing mark pattern, wherein the relative position of the target to be recognized and the auxiliary pointing mark pattern is fixed; in the target recognition image, the mark position of any auxiliary pointing mark in the auxiliary pointing mark pattern is obtained; based on the auxiliary pointing information of the mark position, the target position of the target to be recognized is obtained. In the above manner, the distance of the current mark can be accurately measured, thereby quickly estimating the coordinate difference from the target position, and accurately reaching the target position. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0016] Figure 1 is a flowchart of a first embodiment of a target recognition method provided by the present application;
[0017] Figure 2 is a schematic diagram of a first embodiment of an auxiliary pointing mark provided by the present application;
[0018] Figure 3 The target recognition method provided by this application Figure 1 Schematic diagram of the sub-step flow chart of step S13;
[0019] Figure 4 is a schematic diagram of a second embodiment of the auxiliary pointing mark provided by the present application;
[0020] Figure 5 is a schematic diagram of a third embodiment of the auxiliary pointing mark provided by the present application;
[0021] Figure 6 is a schematic diagram of a fourth embodiment of the auxiliary pointing mark provided by the present application;
[0022] Figure 7 is a schematic diagram of a fifth embodiment of the auxiliary pointing mark provided by the present application;
[0023] Figure 8 is a schematic diagram of a sixth embodiment of the auxiliary pointing mark provided by the present application;
[0024] Fig. 9 This is a schematic diagram of the auxiliary pointing mark pattern provided in the present application when the corners are rounded due to the characteristics of the exposure process;
[0025] Fig.10 It is a structural schematic diagram of an embodiment of a target recognition device provided by the present application;
[0026] Fig.11 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] The target recognition method of the present application is applied to a target recognition device, wherein the target recognition device of the present application can be a server, or a system composed of a server and a local terminal cooperating with each other. Accordingly, the various parts of the target recognition device, such as various units, subunits, modules, and submodules, can all be set in the server, or can be set in the server and the local terminal respectively.
[0029] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules for providing distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here. In some possible implementations, the target identification method of the embodiment of the present application can be implemented by a processor calling a computer-readable instruction stored in a memory.
[0030] See also Figure 1 , Figure 1 It is a flowchart of the first embodiment of the target recognition method provided by the present application.
[0031] like Figure 1 As shown, the specific steps are as follows:
[0032] Step S11: Acquire a target recognition image for the target to be recognized and the auxiliary pointing mark graphic.
[0033] In the embodiment of the present application, the target to be identified may be a mask plate, a glass slide, or other device whose structure is very small and requires an empty space around it.
[0034] In the embodiment of the present application, the relative position of the target to be identified and the auxiliary pointing mark graphic is fixed.
[0035] In an embodiment of the present application, the pixel points of the auxiliary pointing mark in the auxiliary pointing mark graphic are discretely arranged, a grid arrangement is arranged on the target to be identified, and each auxiliary pointing mark is located at a grid point of the grid arrangement.
[0036] See also Figure 2 , Figure 2 It is a schematic diagram of the first embodiment of the auxiliary pointing mark provided by the present application, wherein each auxiliary pointing mark is arranged in a └ shape, the └ shape includes figures in both horizontal and vertical directions, each turning point of the └ shape is a right angle, and the └ shape can be rotated 360 degrees along the axial symmetry center, and the specific rotation angle is determined by the quadrant where each auxiliary pointing mark is located.
[0037] Step S12: in the target recognition image, obtaining the marking position of any auxiliary pointing mark in the auxiliary pointing mark graphic.
[0038] Step S13: Based on the auxiliary pointing information of the marked position, the target position of the target to be identified is obtained.
[0039] Furthermore, the present application proposes steps S131 to S135 as sub-steps of step S13 to obtain the position of the target to be identified. For details, please refer to Figure 3 , Figure 3 The target recognition method provided by this application Figure 1 Schematic diagram of the sub-step flow chart of step S13.
[0040] like Figure 3 As shown, the specific steps are as follows:
[0041] Step S131: obtaining a target transverse pointing mark and a target longitudinal pointing mark of a target auxiliary pointing mark in the auxiliary pointing mark graphic.
[0042] like Figure 4 As shown, Figure 4 It is a schematic diagram of the second embodiment of the auxiliary pointing mark provided by the present application. In the embodiment of the present application, the directions of the auxiliary pointing marks in different quadrants are different. The first quadrant is a "└ shape", and the target is located at the lower left; the second quadrant is a "┘ shape", and the target is located at the lower right; the third quadrant is a "┐ shape", and the target is located at the upper right; the fourth quadrant is a "┌ shape", and the target is located at the lower right.
[0043] Wherein, each auxiliary pointing mark in the auxiliary pointing mark pattern has different transverse pointing marks and / or longitudinal pointing marks.
[0044] Furthermore, in an embodiment of the present application, before the target identification device obtains the target transverse pointing mark and the target longitudinal pointing mark of the target auxiliary pointing mark in the auxiliary pointing mark graphic, the target identification device obtains the candidate transverse pointing mark and the candidate longitudinal pointing mark of at least two candidate auxiliary pointing marks in the auxiliary pointing mark graphic; and takes the candidate auxiliary pointing mark corresponding to the minimum value of the sum of the extension length of the candidate transverse pointing mark and the extension length of the candidate longitudinal pointing mark as the target auxiliary pointing mark.
[0045] Step S132: Determine the coordinate quadrant where the target auxiliary pointing mark is located based on the extension direction of the target transverse pointing mark and the extension direction of the target longitudinal pointing mark.
[0046] Specifically, since the auxiliary pointing mark patterns in different quadrants are different, the extension direction of the target transverse pointing mark and the extension direction of the target longitudinal pointing mark of the target identification device can determine the coordinate quadrant where the target auxiliary pointing mark is located.
[0047] like Figure 4 As shown, the half-integer grid points are:
[0048]
[0049] Where x is the horizontal coordinate, y is the vertical coordinate, T1 is the horizontal marking distance, and T2 is the vertical marking distance.
[0050] like Figure 5-8 As shown, Figure 5 is a schematic diagram of a third embodiment of the auxiliary pointing mark provided by the present application; Figure 6 is a schematic diagram of a fourth embodiment of the auxiliary pointing mark provided by the present application; Figure 7 is a schematic diagram of a fifth embodiment of the auxiliary pointing mark provided by the present application; Figure 8 It is a schematic diagram of the sixth embodiment of the auxiliary pointing mark provided in the present application.
[0051] in, Figure 5-Figure 8 The auxiliary pointing marks in the figure are auxiliary pointing marks in different quadrants, with line widths of ω1 and ω2 in the horizontal and vertical directions respectively, and the lengths of the two arms of the L-shape in the horizontal and vertical directions respectively (m+1)ω1 and (n+1)ω2, thereby determining the coordinates (x, y) of the axial symmetry center of each auxiliary pointing mark.
[0052] The length of the auxiliary marker in both directions determines its distance from the target position. The longer the length, the farther the distance in the corresponding coordinate axis direction. This kind of distance-giving marker can be of great help to automated search. Since image recognition can accurately measure the distance of the current marker, it can quickly estimate the coordinate difference from the target position and reach the target position accurately quantitatively. The error magnitude is close to the recognition accuracy. As long as any marker in the grid array is seen in the field of view, only one move is required to reach the target without further searching.
[0053] Step S133: Determine the lateral coordinate difference based on the extension length of the target lateral pointing mark.
[0054] Specifically, the target recognition device determines the number of transverse grids based on the extension length of the target transverse pointing mark; and determines the transverse coordinate difference based on the number of transverse grids and a preset mark interval.
[0055] Step S134: Determine the longitudinal coordinate difference based on the extension length of the target longitudinal pointing mark.
[0056] Specifically, the target recognition device determines the number of longitudinal grids based on the extension length of the target longitudinal pointing mark; and determines the longitudinal coordinate difference based on the number of transverse grids and a preset mark interval.
[0057] Step S135: Determine the mark position of the target auxiliary pointing mark based on the horizontal coordinate difference, the vertical coordinate difference, and the coordinate quadrant.
[0058] Specifically, the target recognition device obtains the coding information of the target auxiliary pointing mark in the auxiliary pointing mark graphic; based on the coding information, determines the marking position of the target auxiliary pointing mark.
[0059] In the embodiment of the present application, the encoding information can be any data encoding method, including but not limited to using numbers, coordinates, etc.
[0060] The length of the auxiliary pointing mark in two directions determines its distance from the target position. The longer the length, the farther the distance in the corresponding coordinate axis direction. This kind of mark that gives the distance can provide great help for automated search. Since image recognition can accurately measure the distance of the current mark, it can quickly estimate the coordinate difference from the target position, so as to reach the target position accurately quantitatively. The error magnitude is close to the recognition accuracy. As long as any mark in the grid array is seen in the field of view, only one move is required to reach the target without further searching.
[0061] like Fig. 9 As shown, Fig. 9 This is a schematic diagram of the auxiliary pointing mark pattern provided in the present application when rounded corners occur due to the characteristics of the exposure process.
[0062] Since the size of the mark often reaches the diffraction limit of optical imaging, when the exposure causes severe rounding, the pattern will become Fig. 9 In the situation shown, the present application is still able to identify the information carried by the mark.
[0063] Since the auxiliary pointing mark graphics designed in the embodiment of the present application are discretized, the measurement data only needs to be compared with these discrete values, and the closest one is the result value. This discretization digitizes the position information carried by the mark, so that it has strong anti-noise ability. It has strong adaptability to different exposure parameters and different illumination wavelengths, so that its stability in practical applications is guaranteed.
[0064] In some other embodiments of the present application, the auxiliary pointing mark graphics also include some variations, such as replacing the half-integer in the basic version with other grid points, such as whole points or other rational or irrational numbers; for example, changing the equidistant grid into a non-equidistant grid, such as a logarithmic grid or a grid with Fibonacci sequence spacing, etc.; for example, only part of the grid is used, such as leaving a space in the middle or only using half a quadrant or a quarter of a quadrant, etc.; for example, reversing the yin and yang of the entire graphic, and changing positive engraving to negative engraving, etc.
[0065] Furthermore, after the target recognition device obtains the target position of the target to be recognized based on the auxiliary pointing information of the mark position, the target recognition device uses the target position and the mark position to obtain a moving instruction; moves the target to be recognized according to the moving instruction, and performs re-recognition.
[0066] The present application designs an auxiliary pointing mark pattern that uses only horizontal and vertical graphics, and does not have topological holes or topological islands, so it has advantages in micro-nano processing, and is easy to optimize exposure, easy to peel off after evaporation deposition, and not easy to be damaged or dropped. Its area is also small, which can meet the requirement of no interference around the sample. The auxiliary pointing mark pattern can provide an indication of the target position when locating the sample in a haystack. At the same time, the position pointing information it carries is digital, and it has good stability and noise resistance in recognition.
[0067] In order to implement the target recognition method of the above embodiment, the present application also provides a target recognition device. Fig.10 , Fig.10 It is a structural schematic diagram of an embodiment of a target recognition device provided by the present application.
[0068] like Fig.10 As shown, the target recognition device 600 of this embodiment includes a processor 61 , a memory 62 , an input / output device 63 , and a bus 64 .
[0069] The processor 61 , the memory 62 , and the input / output device 63 are respectively connected to the bus 64 . The memory 62 stores a computer program, and the processor 61 is used to execute the computer program to implement the target recognition method of the above embodiment.
[0070] In this embodiment, the processor 61 may also be referred to as a CPU (Central Processing Unit). The processor 61 may be an integrated circuit chip having the ability to process signals. The processor 61 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 61 may also be a GPU (Graphics Processing Unit), also known as a display core, a visual processor, or a display chip, which is a microprocessor that is specifically used for image computing on computers, workstations, game consoles, and some mobile devices (such as tablet computers, smart phones, etc.). The purpose of the GPU is to convert and drive the display information required by the computer system, and to provide a line scan signal to the display to control the correct display of the display. It is an important component that connects the display and the computer motherboard. As an important component of the computer host, the graphics card is responsible for outputting display graphics. The general-purpose processor may be a microprocessor or the processor 61 may also be any conventional processor, etc.
[0071] The present application also provides a computer storage medium, such as Fig.11 As shown, the computer storage medium 700 is used to store a computer program 71. When the computer program 71 is executed by a processor, it is used to implement the method described in the target identification method embodiment of the present application.
[0072] The method involved in the target identification method embodiment of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0073] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A target recognition method based on microstructures, characterized in that: The target recognition method comprises: Acquire a target recognition image for a target to be recognized and an auxiliary pointing mark pattern, wherein the relative positions of the target to be recognized and the auxiliary pointing mark pattern are fixed; In the target recognition image, obtaining a marking position of any one of the auxiliary pointing marks in the auxiliary pointing mark graphic; Based on the auxiliary pointing information of the marked position, the target position of the target to be identified is obtained.
2. The target recognition method according to claim 1, characterized in that: Each auxiliary pointing mark in the auxiliary pointing mark pattern has different transverse pointing marks and / or longitudinal pointing marks.
3. The target recognition method according to claim 2, characterized in that: The obtaining of the marking position of any one of the auxiliary pointing marks in the auxiliary pointing mark graphic comprises: Acquire a target transverse pointing mark and a target longitudinal pointing mark of the target auxiliary pointing mark in the auxiliary pointing mark graphic; Determining the coordinate quadrant where the auxiliary target pointing mark is located based on the extension direction of the target transverse pointing mark and the extension direction of the target longitudinal pointing mark; Determining a lateral coordinate difference based on an extension length of the target lateral pointing mark; Determining a longitudinal coordinate difference based on an extension length of the target longitudinal pointing mark; Based on the horizontal coordinate difference, the vertical coordinate difference, and the coordinate quadrant, a mark position of the target auxiliary pointing mark is determined.
4. The target recognition method according to claim 3, characterized in that: The determining of the lateral coordinate difference based on the extension length of the target lateral pointing mark comprises: Determining the number of lateral grids based on the extension length of the target lateral pointing mark; The lateral coordinate difference is determined based on the lateral grid number and a preset marking interval.
5. The target recognition method according to claim 3, characterized in that: Before obtaining the target transverse pointing mark and the target longitudinal pointing mark of the target auxiliary pointing mark in the auxiliary pointing mark graphic, the target recognition method further includes: Acquire a candidate transverse pointing mark and a candidate longitudinal pointing mark of at least two candidate auxiliary pointing marks in the auxiliary pointing mark pattern; The candidate auxiliary directional marker corresponding to the minimum value of the sum of the extension length of the candidate transverse directional marker and the extension length of the candidate longitudinal directional marker is used as the target auxiliary directional marker.
6. The target recognition method according to claim 1, characterized in that: The obtaining of the marking position of any one of the auxiliary pointing marks in the auxiliary pointing mark graphic comprises: Obtaining the coding information of the target auxiliary pointing mark in the auxiliary pointing mark graphic; Based on the coding information, a marking position of the target auxiliary pointing mark is determined.
7. The target recognition method according to claim 1, characterized in that: The pixel points of the auxiliary pointing mark in the auxiliary pointing mark graphic are set in a discrete manner.
8. The target recognition method according to claim 1, characterized in that: After obtaining the target position of the target to be identified based on the auxiliary pointing information of the marking position, the target identification method further includes: Using the target position and the mark position, obtaining a movement instruction; The target to be identified is moved according to the movement instruction, and re-identified.
9. A target recognition device based on microstructures, characterized in that: The target identification device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the target recognition method according to any one of claims 1 to 8.
10. A computer storage medium, characterized in that: The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the target recognition method according to any one of claims 1 to 8.