Automatic picking apparatus and operating method thereof
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- CURIOSIS CO LTD
- Filing Date
- 2026-02-03
- Publication Date
- 2026-08-06
Smart Images
Figure KR2026001998_06082026_PF_FP_ABST
Abstract
Description
Automatic picking device and its operation method
[0001] The present disclosure relates to an automatic picking device, and specifically to an automatic picking device configured to perform accurate picking of a target material.
[0002] In automated bioanalytical equipment or colony picking equipment, high-precision position control technology is essential to accurately recognize the location of target substances and to precisely extract and transport them using picking means such as needles. This position control is generally performed based on image-based recognition results using a vision module and requires alignment between the control coordinates of the needle and the image recognition coordinates.
[0003] However, in actual equipment, a minute offset may occur between the physical center coordinates of the needle and the coordinates recognized by the vision module due to various factors, such as manufacturing or assembly errors of core components, or minute bending or thermal expansion of the structure. Such errors hinder the needle from accurately reaching the target point and can cause picking failure, contamination, or sample damage, especially in bio experiments requiring precision in the order of several micrometers (μm).
[0004] Therefore, there is a need for technology capable of controlling the needle to accurately reach the target coordinates by precisely measuring the center coordinates of the needle and quantitatively calculating and correcting the coordinate error between the vision module and the needle. Furthermore, there is an increasing need for a high-precision coordinate correction system that can ensure repeatability and experimental reliability through the automation of the error correction process.
[0005] One technical objective of the present disclosure is to provide an automatic picking device capable of performing accurate picking by correcting the coordinate error between a vision module and a needle.
[0006] The technical problems to be solved in this disclosure are not limited to those mentioned above, and various unmentioned technical problems can be inferred by a person skilled in the art from this disclosure.
[0007] An automatic picking device according to embodiments of the present disclosure may include one or more processors, one or more memories storing at least one instruction executed by the one or more processors, a vision module including a camera configured to acquire an image of a target substance, a picking module including a needle configured to pick the target substance, and a position measuring module configured to acquire first position information of the camera and second position information of the needle. The one or more processors may be configured to acquire first position information of the camera through the position measuring module and acquire second position information of the needle through the position measuring module by executing the at least one instruction, and to calculate the relative position of the camera and the needle based on the first position information and the second position information.
[0008] In one embodiment, the position measuring module may include a laser sensor that generates a first laser beam and a second laser beam that intersect each other. The processor may be configured to acquire the coordinates of the needle when either the first laser beam and the second laser beam are at least partially blocked by the needle.
[0009] In one embodiment, the laser sensor may include a first light-emitting unit that generates the first laser beam, a first light-receiving unit that receives the first laser beam, a second light-emitting unit that generates the second laser beam, and a second light-receiving unit that receives the second laser beam. The laser sensor may be configured to generate a cutoff signal when the intensity of the first laser beam received by the first light-receiving unit or the second laser beam received by the second light-receiving unit decreases to a value or less.
[0010] In one embodiment, the processor may be configured to acquire the coordinates of the needle when the blocking signal is generated.
[0011] In one embodiment, obtaining second position information of the needle may include rotating the needle clockwise around the intersection point of the first and second laser beams, and rotating the needle counterclockwise around the intersection point of the first and second laser beams.
[0012] In one embodiment, the second position information of the needle can be calculated by considering clockwise measurement coordinates obtained while rotating the needle clockwise and counterclockwise measurement coordinates obtained while rotating the needle counterclockwise.
[0013] In one embodiment, the clockwise measurement coordinates may include a first clockwise measurement coordinate and a second clockwise measurement coordinate obtained by the needle blocking the first laser beam, and a third clockwise measurement coordinate and a fourth clockwise measurement coordinate obtained by the needle blocking the second laser beam. The counterclockwise measurement coordinates may include a first counterclockwise measurement coordinate and a second counterclockwise measurement coordinate obtained by the needle blocking the first laser beam, and a third counterclockwise measurement coordinate and a fourth counterclockwise measurement coordinate obtained by the needle blocking the second laser beam. The first clockwise measurement coordinate and the first counterclockwise measurement coordinate, the second clockwise measurement coordinate and the second counterclockwise measurement coordinate, the third clockwise measurement coordinate and the third counterclockwise measurement coordinate, and the fourth clockwise measurement coordinate and the fourth counterclockwise measurement coordinate may each be adjacent to one another.
[0014] In one embodiment, the processor may be configured to calculate a first measurement coordinate from the first clockwise measurement coordinate and the first counterclockwise measurement coordinate, calculate a second measurement coordinate from the second clockwise measurement coordinate and the second counterclockwise measurement coordinate, calculate a third measurement coordinate from the third clockwise measurement coordinate and the third counterclockwise measurement coordinate, and calculate a fourth measurement coordinate from the fourth clockwise measurement coordinate and the fourth counterclockwise measurement coordinate.
[0015] In one embodiment, the processor may be configured to determine a first straight line passing through the first measurement coordinate and the second measurement coordinate, determine a second straight line passing through the third measurement coordinate and the fourth measurement coordinate, calculate the coordinates of the intersection point of the first straight line and the second straight line, and calculate the second position information of the needle from the coordinates of the intersection point of the first straight line and the second straight line and the coordinates of the intersection point of the first laser beam and the second laser beam.
[0016] In one embodiment, obtaining the second position information of the needle may further include moving the needle onto the intersection point of the first and second laser beams and then lowering the needle.
[0017] In one embodiment, the position measuring module may further include a light source formed below the intersection point of the first laser beam and the second laser beam. Acquiring the first position information of the camera may include capturing light generated from the light source with the camera.
[0018] In one embodiment, the processor may be configured to obtain the first position information of the camera from the coordinates of the camera when the light generated from the light source is located at the center of the camera.
[0019] In one embodiment, the picking module includes a plurality of needles, and the processor may be configured to calculate second position information for all of the plurality of needles.
[0020] In one embodiment, the picking module includes a plurality of needles arranged in a grid shape, and the processor may be configured to acquire second position information for at least some of the plurality of needles or for a needle placed at a vertex of the grid.
[0021] A method of operation of an automatic picking device according to embodiments of the present disclosure may include the steps of: obtaining first position information of a camera included in the automatic picking device through a position measurement module in a processor of the automatic picking device; obtaining second position information of a needle included in the automatic picking device through the position measurement module in the processor; and calculating the relative position of the camera and the needle based on the first position information and the second position information in the processor.
[0022] According to embodiments of the present disclosure, the relative position between the camera and the needle can be precisely calculated through a position measurement module, and picking of a target material can be performed based thereon. Accordingly, the picking accuracy of the target material can be improved.
[0023] The effects according to the technical concept of the present disclosure are not limited to the effects mentioned above, and various unmentioned effects can be clearly understood by a person skilled in the art from the present disclosure.
[0024] FIGS. 1 and 2 are drawings for explaining the configuration of an automatic picking device according to one embodiment of the present disclosure.
[0025] FIG. 3 is a drawing for explaining the configuration of a position measurement module according to one embodiment of the present disclosure.
[0026] FIG. 4 is a drawing for explaining the operation of a laser sensor according to one embodiment of the present disclosure.
[0027] FIG. 5 is a flowchart for explaining the operation of an automatic picking device according to embodiments of the present disclosure.
[0028] FIG. 6 is a flowchart for explaining step S200 of FIG. 5 according to the present disclosure.
[0029] FIGS. 7a to 7d are drawings for explaining steps S210, S220, S230, and S240 of FIG. 6 according to the present disclosure, respectively.
[0030] FIG. 8 is a drawing for explaining a method for measuring the coordinates of the tip of a needle according to embodiments of the present disclosure.
[0031] Hereinafter, specific embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The embodiments of the present disclosure are illustrative for the purpose of explaining the embodiments. Various modifications may be made to the embodiments, and the scope of the present application is not limited by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.
[0032] In the accompanying drawings, identical or similar components are assigned the same reference numbers. Additionally, when describing embodiments of the present disclosure, descriptions of identical or similar components may be omitted to avoid redundant descriptions. However, such omission of description is not intended to imply that the component is not included in a particular embodiment.
[0033] Unless otherwise defined, the terms used in this disclosure may have the meaning generally understood by those skilled in the art.
[0034] In the present disclosure, the expressions “each of a plurality of A” or “each of a plurality of A” may refer to each of all elements included in a plurality of A, or may refer to each of some elements of a plurality of A.
[0035] In the present disclosure, the expression “one or more A” may mean a set of one or more A's unless the context clearly indicates otherwise.
[0036] In this disclosure, expressions such as "first," "second," or "first," "second," etc., do not limit the order, importance, etc., of the components they modify unless the context clearly indicates otherwise. These expressions may be used to distinguish one component from another.
[0037] In the present disclosure, expressions such as “A, B, or C”, “A, B, and / or C”, “at least one of A, B, and C”, “at least one of A, B, or C”, “at least one of A, B, and / or C”, “at least one selected from A, B, and C”, “at least one selected from A, B, or C”, and “at least one selected from A, B, and / or C” may mean each or all possible combinations thereof. For example, “at least one of A or B” may refer to at least one A, at least one B, at least one A, and at least one B.
[0038] In the present disclosure, expressions such as "connected" and "connected" should be understood to mean that while one component may be directly connected to or connected to another component, a new component may exist between them.
[0039] In this disclosure, expressions such as "below," "lower side," "above," and "upper side" should be understood to include cases where a component may be located "below," "lower side," "above," or "upper side" by directly contacting another component, but also cases where a new component exists between them and the component is indirectly located "below," "lower side," "above," or "upper side." These expressions are used to describe the relationships between components illustrated in the drawings. These terms are relative concepts and are described based on the directions indicated in the drawings.
[0040] In this disclosure, expressions such as "comprising," "having," and "having" imply the presence of the relevant features (e.g., functions, operations, or components, etc.) and do not exclude the presence of other additional features. That is, these expressions should be understood as open-ended terms that leave open the possibility of including other embodiments.
[0041] In the present disclosure, expressions such as "configured to..." may have meanings such as "set to...", "capable of...", "modified to...", "made to...", or "capable of..." depending on the context. These expressions are not limited to "specifically designed in hardware." For example, a processor configured to perform a specific operation may refer to a general-purpose processor capable of performing that operation through software execution, or a special-purpose computer structured through programming to perform the said specific operation.
[0042] FIGS. 1 and 2 are drawings for explaining the configuration of an automatic picking device according to one embodiment of the present disclosure.
[0043] Referring to FIGS. 1 and 2, the automatic picking device (10) may be a device for picking a target substance placed in a work place (not shown) using a needle (210). For example, the target substance may be a colony cultured in a medium.
[0044] The automatic picking device (10) may include a vision module (100), a picking module (200), a position measuring module (300), a processor (400), and a memory (500). Some components of the automatic picking device (10) that are not essential to explaining the contents of the present disclosure may not be shown in the drawings or may be shown in a simplified manner, and the description of such components may also be omitted or simplified. For example, the configuration of the picking module in FIG. 2 is shown in a simplified manner.
[0045] In relation to the configuration and operation of the automatic picking device (10), a first direction (D1), a second direction (D2), and a third direction (D3) that intersect each other may be defined. In one embodiment, the first direction (D1) and the second direction (D2) may be substantially parallel to the ground and substantially perpendicular to each other. Additionally, the third direction (D3) may be substantially perpendicular to the ground. Hereinafter, the third direction (D3) and the perpendicular direction may refer to the same direction.
[0046] As illustrated in FIG. 2, the vision module (100) and the picking module (200) may be components included in the head assembly (50). The head assembly (50) may be positioned at a predetermined distance from the work place in a third direction (D3) and may move in a first direction (D1), a second direction (D2), and a third direction (D3). When the head assembly (50) moves in the first direction (D1), the second direction (D2), and the third direction (D3), the vision module (100) including the camera (110) and the picking module (200) including a plurality of needles (210) may also move together.
[0047] The vision module (100) may be positioned at a predetermined distance from the workplace in a third direction (D3). The vision module (100) may include a camera (110) configured to acquire an image of a target material placed on the workplace.
[0048] The vision module (100) and the camera (110) included therein can be configured to move along a first direction (D1), a second direction (D2), and a third direction (D3) to photograph a target material placed on a workspace and acquire an image of the target material. The vision module (100) can transmit the acquired image to a processor (400).
[0049] The picking module (200) may be positioned at a predetermined distance from the work place in a third direction (D3). As illustrated in FIG. 1, the picking module (200) may include a plurality of needles (210) configured to pick a target material placed on the work place and a needle support (220) supporting the plurality of needles (210). For example, the plurality of needles (210) may be arranged in a grid shape as illustrated in FIG. 1.
[0050] A picking module (200) and a needle (210) included therein may be configured to move along a first direction (D1), a second direction (D2), and a third direction (D3) to pick a target material placed on a workpiece. Specifically, a head assembly (50) may be moved so that any one of the selected needles is positioned on the target material, and then the selected needle may move downward along the third direction (D3) to perform picking of the target material. The picking module (200) may include a needle operating unit (not shown) configured to operate the needle (210) individually. The needle operating unit may be configured to select any one of a plurality of needles and move it up and down along the third direction (D3). Accordingly, picking of the target material may be performed by the selected needle moving downward to pick the target material placed on the workpiece and then moving upward to return. Target material picking in this manner can be performed sequentially on multiple target materials using multiple needles (210).
[0051] The picking module (200) may be detachable from the head assembly (50). For example, various types of picking modules (200) may be provided depending on the type, arrangement, number, etc. of the needles, and various types of picking modules (200) may be detachably installed on the head assembly (50).
[0052] A position measurement module (300) may be placed on a workspace. The position measurement module (300) may be configured to acquire position information of a camera (110) included in a vision module (100) (hereinafter also referred to as "first position information") and position information of an individual needle (210) included in a picking module (200) (hereinafter also referred to as "second position information").
[0053] FIG. 3 is a drawing for explaining the configuration of a position measuring module according to one embodiment of the present disclosure. FIG. 4 is a drawing for explaining the operation of a laser sensor according to one embodiment of the present disclosure.
[0054] Referring further to FIG. 3, the position measuring module (300) may include a laser sensor (310) that generates a first laser beam (LB1) and a second laser beam (LB2) that intersect each other. The first laser beam (LB1) and the second laser beam (LB2) may be substantially parallel to the ground and may intersect each other to form an intersection point (LB_C).
[0055] The laser sensor (310) may include a first light-emitting part (312) that generates a first laser beam (LB1), a first light-receiving part (314) disposed opposite the first light-emitting part (312) to receive the first laser beam (LB1), a second light-emitting part (316) that generates a second laser beam (LB2), and a second light-receiving part (318) disposed opposite the second light-emitting part (316) to receive the second laser beam (LB2).
[0056] The laser sensor (310) may be configured to obtain position information of individual needles (210).
[0057] Specifically, by operating the head assembly (50) and the needle operating part, the end of the needle (210) to acquire position information can move to the position measurement module (300) and can move through the first laser beam (LB1) and the second laser beam (LB2).
[0058] The laser sensor (310) may be configured to generate a blocking signal when the intensity of the first laser beam (LB1) received by the first light receiving unit (314) or the intensity of the second laser beam (LB2) received by the second light receiving unit decreases to a value or less. The processor (400) may be configured to receive the blocking signal from the laser sensor (310) and, at this time, acquire the coordinates of the needle (210). In other words, the processor (400) may be configured to acquire the coordinates of the needle (210) when either the first laser beam (LB1) or the second laser beam (LB2) is at least partially blocked by the needle (210). As previously described, the needle (210) may move together with the head assembly (50), and thus, in one embodiment, the coordinates of the head assembly (50) when the blocking signal is generated may be acquired as the coordinates of the corresponding needle (210).
[0059] In this regard, referring to FIG. 4, an example is illustrated in which the coordinates of the needle (210) are obtained as the needle (210) passes through the laser beam (LB). From a planar perspective, when the needle (210) passes through the laser beam (LB) in a direction perpendicular to the direction in which the laser beam (LB) is irradiated (i.e., the direction from the emitting part toward the receiving part) (i.e., the direction of the arrow in FIG. 4), as the needle (210) moves, the intensity of the laser beam (LB) received by the receiving part (314 or 318) may gradually decrease, and if the intensity of the laser beam (LB) received by the receiving part (314 or 318) decreases below a predetermined value, a blocking signal may be generated. At this time, as shown in FIG. 4, a blocking signal may be generated in advance before the needle (210) is positioned at the center of the laser beam, and accordingly, it may be difficult to accurately measure the coordinates of the needle (210) based on the blocking signal of the laser sensor (310).
[0060] In the present disclosure, recognizing this problem, the needle (210) is configured to pass a laser beam (LB) from two opposite directions and calculate the measured coordinates of the needle by considering the two measured coordinates. This allows the accuracy of the needle coordinate measurement to be increased. This will be explained in more detail below with reference to FIGS. 7a to 7d.
[0061] Referring again to FIG. 3, the position measuring module (300) may further include a light source unit (320). The light source unit (320) may be formed below the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2).
[0062] The light source unit (320) may be configured to obtain information from the camera (110).
[0063] Specifically, the light source unit (320) can generate light that is irradiated in a third direction (D3). When acquiring position information of the camera (110), the camera (110) can move onto the position measurement module (300) to capture the light generated from the light source unit (320). Specifically, the camera (110) can move so that the light generated from the light source unit (320) is positioned at the center of the captured image of the camera (110), and the coordinates of the camera (110) at this time can be acquired as position information of the camera (110). As previously described, the camera (110) can move together with the head assembly (50), and thus, in one embodiment, the coordinates of the head assembly (50) when the light generated from the light source unit (320) is positioned at the center of the captured image of the camera (110) can be acquired as the coordinates of the corresponding camera (110).
[0064] A processor (400) can perform the operation of an automatic picking device (10) according to FIG. 5 or FIG. 6. The processor (400) may include one or more processors. The processor (400) can control at least one component of the automatic picking device (10) by running software (e.g., instructions, programs, etc.). The processor (400) can read data, etc. from memory (500) or write data, etc. to memory (500). Additionally, the processor (400) can perform various operations such as calculation, processing, data generation or processing according to an embodiment of the present disclosure by executing at least one instruction stored in memory (500).
[0065] The memory (500) may include one or more memory units. The memory (500) may write or read various data upon request from the processor (400), etc. The memory (500) may store at least one instruction executed by the processor (400). The memory (500) may store an image received from the vision module (100) or position information of the camera (110) and / or position information of the needle (210) received from the position measurement module (300) by a command from the processor (400). Additionally, the memory (500) may store an operating system, an application, etc. for controlling the operation of the automatic picking device (10). The memory (500) may include volatile memory or non-volatile memory, but the present disclosure is not limited thereto.
[0066] In the memory (500), design position information of the needle (210) according to the type of picking module (200) may be stored. The design position information of the needle (210) may include information regarding the position of each needle (210) (hereinafter also referred to as the "design position" of the needle) when the picking module (200) is manufactured as designed without processing or assembly errors and installed in the head assembly (50).
[0067] However, in the case of the actual picking module (200), errors may occur during the manufacturing and installation stages, and accordingly, the actual location where the needle (210) is installed in the head assembly (50) (hereinafter also referred to as the "actual location" of the needle) may differ from the design location of the needle (210). Such errors may be a factor that hinders accurate picking. Accordingly, in the present disclosure, information regarding the actual location of the needle (210) is obtained through the position measuring module (300) described above, and the accuracy of picking can be improved by storing and utilizing this information in the memory (500).
[0068] In the present disclosure, the expressions “at least one instruction stored in memory” or “a program stored in memory” may be used to refer to an operating system, an application for controlling the resources of an automatic picking device (10), or middleware that provides various functions to an application so that the application can utilize the resources of the automatic picking device (10). In the embodiments, when the processor (400) performs a specific operation, the memory (500) may store instructions that are performed by the processor (400) and correspond to the specific operation.
[0069] FIG. 5 is a flowchart for explaining the operation of an automatic picking device according to embodiments of the present disclosure. FIG. 6 is a flowchart for explaining step S200 of FIG. 5 according to the present disclosure. FIG. 7a to 7d are drawings for explaining steps S210, S220, S230, and S240 of FIG. 6 according to the present disclosure, respectively. FIG. 8 is a drawing for explaining a method for measuring the coordinates of the tip of a needle according to embodiments of the present disclosure.
[0070] Referring to FIG. 5, the automatic picking device (10) can obtain first position information of the camera (110) and second position information of the needle (210) through the position measurement module (300), and can calculate the relative position of the camera (110) and the needle (210) based on this.
[0071] In step S100, the processor (400) can obtain first position information of the camera (110) through the position measurement module (300).
[0072] Referring to FIGS. 2 and 3, the light source unit (320) included in the position measurement module (300) can generate light directed toward a third direction (D3) (i.e., a vertical direction). The processor (400) can move the head assembly (50) based on an image captured by the camera (110) so that the light generated by the light source unit (320) is positioned at the center of the captured image of the camera (110), and the coordinates of the camera (110) (or the coordinates of the head assembly (50)) at that time can be obtained as first position information of the camera (110).
[0073] In other words, the processor (400) can obtain the coordinates of the camera (110) (or the coordinates of the head assembly (50)) when the light generated from the light source (320) is positioned at the center of the captured image of the camera (110) as the first position information of the camera (110). The processor (400) can store the obtained first position information of the camera (110) in the memory (500).
[0074] In step S200, the processor (400) can obtain second position information of the needle (210) through the position measurement module (300).
[0075] Referring further to FIG. 6, obtaining second position information of the needle (210) may include rotating the needle (210) clockwise to obtain clockwise measurement coordinates (S210), rotating the needle (210) counterclockwise to obtain counterclockwise measurement coordinates (S220), calculating measurement coordinates based on the clockwise measurement coordinates and the counterclockwise measurement coordinates (S230), determining two straight lines based on the measurement coordinates (S240), and calculating second position information of the needle (210) from the intersection point of the two determined straight lines and the intersection point (LB_C) of the first and second laser beams (LB1, LB2) (S250). Each step is described in detail below.
[0076] Referring further to FIG. 7a, in step S210, the needle (210) can be controlled to rotate clockwise around the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2).
[0077] In step S210, the needle (210) can be controlled based on design position information. Specifically, based on the design position of the individual needle (210) included in the design position information of the corresponding picking module (200), the needle (210) can rotate one full clockwise along a predetermined circle centered on the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2). At this time, the coordinates of the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2) can be defined as the coordinates when the needle (210) is located at the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2), according to the design position information of the needle (210).
[0078] Meanwhile, as described above, the actual position of the needle (210) may differ from the design position of the needle (210) due to errors during the manufacturing and installation stages. Therefore, in step S210, the actual center of rotation of the needle (210) may not exactly coincide with the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2). For example, in step S210, the actual center of rotation of the needle (210) may deviate from the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2) by the aforementioned error.
[0079] In this regard, the fact that the needle (210) rotates around the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2) in this disclosure may include cases where the actual center of rotation of the needle (210) does not exactly coincide with the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2) due to such error. This may also be applied in step S220 described later.
[0080] As the needle (210) rotates one full clockwise, four blocking signals may be generated, and accordingly, four coordinates for the needle (210) (hereinafter also referred to as "clockwise measurement coordinates") may be obtained. Specifically, as shown in FIG. 7a, when the needle (210) rotates one full clockwise, it may block the first laser beam (LB1) by passing through it twice, and accordingly, the first clockwise measurement coordinate (CW1) and the second clockwise measurement coordinate (CW2) may be obtained. Likewise, when the needle (210) rotates one full clockwise, it may block the second laser beam (LB2) by passing through it twice, and accordingly, the third clockwise measurement coordinate (CW3) and the fourth clockwise measurement coordinate (CW4) may be obtained. The first to fourth clockwise measurement coordinates (CW1, CW2, CW3, CW4) may be two-dimensional coordinates obtained from a planar perspective.
[0081] Since the first to fourth clockwise measured coordinates (CW1, CW2, CW3, CW4) are coordinates measured as the needle (210) moves clockwise, they may be coordinates measured clockwise forward (or counterclockwise backward) from the width direction center of the laser beam (LB1, LB2) as shown in FIG. 7a.
[0082] Referring further to FIG. 7b, in step S220, the needle (210) can be controlled to rotate counterclockwise around the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2).
[0083] In step S220, the needle (210) can be controlled based on design position information. Specifically, based on the design position of the individual needle (210) included in the design position information of the corresponding picking module (200), the needle (210) can rotate one full turn counterclockwise along a predetermined circle centered on the intersection point (LB_C) of the first laser beam (LB1) and the second laser beam (LB2).
[0084] As the needle (210) rotates one full turn in a counterclockwise direction, four blocking signals may be generated, and accordingly, four coordinates for the needle (210) (hereinafter also referred to as "counterclockwise measurement coordinates") may be obtained. Specifically, as shown in FIG. 7b, when the needle (210) rotates one full turn in a counterclockwise direction, it may block the first laser beam (LB1) by passing through it twice, and accordingly, the first counterclockwise measurement coordinate (CCW1) and the second counterclockwise measurement coordinate (CCW2) may be obtained. Likewise, when the needle (210) rotates one full turn in a counterclockwise direction, it may block the second laser beam (LB2) by passing through it twice, and accordingly, the third counterclockwise measurement coordinate (CCW3) and the fourth counterclockwise measurement coordinate (CCW4) may be obtained. The first to fourth counterclockwise measured coordinates (CCW1, CCW2, CCW3, CCW4) may be two-dimensional coordinates obtained from a planar perspective.
[0085] Since the first to fourth counterclockwise measured coordinates (CCW1, CCW2, CCW3, CCW4) are coordinates measured as the needle (210) moves counterclockwise, they may be coordinates measured counterclockwise forward (or clockwise backward) from the width direction center of the laser beam (LB1, LB2) as shown in FIG. 7b.
[0086] Referring further to FIG. 7c, in step S230, measurement coordinates (P1, P2, P3, P4) can be calculated based on clockwise measurement coordinates (CW1, CW2, CW3, CW4) and counterclockwise measurement coordinates (CCW1, CCW2, CCW3, CCW4).
[0087] Clockwise measurement coordinates (CW1, CW2, CW3, CW4) and counterclockwise measurement coordinates (CCW1, CCW2, CCW3, CCW4) may form adjacent (or corresponding) pairs. Specifically, as illustrated in FIGS. 7a and 7b, the first clockwise measurement coordinate (CW1) and the first counterclockwise measurement coordinate (CCW1), the second clockwise measurement coordinate (CW2) and the second counterclockwise measurement coordinate (CCW2), the third clockwise measurement coordinate (CW3) and the third counterclockwise measurement coordinate (CCW3), and the fourth clockwise measurement coordinate (CW4) and the fourth counterclockwise measurement coordinate (CCW4) may each be adjacent to each other and may form pairs.
[0088] More specifically, as illustrated in FIGS. 7a and 7b, the first and second laser beams (LB1, LB2) can be divided into four parts (LB1_a, LB1_b, LB2_a, LB2_b) based on the intersection point (LB_C). At this time, clockwise measurement coordinates (CW1, CW2, CW3, CW4) and counterclockwise measurement coordinates (CCW1, CCW2, CCW3, CCW4) that are measured in contact with the same part (LB1_a, LB1_b, LB2_a, LB2_b) of the laser beam (LB1, LB2) can be defined as adjacent (or corresponding) coordinate pairs. Specifically, the first clockwise measurement coordinate (CW1) and the first counterclockwise measurement coordinate (CCW1) measured in contact with the first part (LB1_a) of the first laser beam (LB1), the second clockwise measurement coordinate (CW2) and the second counterclockwise measurement coordinate (CCW2) measured in contact with the second part (LB1_b) of the first laser beam (LB1), the third clockwise measurement coordinate (CW3) and the third counterclockwise measurement coordinate (CCW3) measured in contact with the first part (LB2_a) of the second laser beam (LB2), and the fourth clockwise measurement coordinate (CW4) and the fourth counterclockwise measurement coordinate (CCW4) measured in contact with the second part (LB2_b) of the second laser beam (LB2) can each form a pair adjacent to each other.
[0089] At this time, the clockwise and counterclockwise measurement coordinates that form a pair with each other may be coordinates that are measured when the needle (210) passes through substantially the same point of the laser beam (LB1, LB2), but with only the direction of movement opposite. For example, the first clockwise measurement coordinate (CW1) and the first counterclockwise measurement coordinate (CCW1) may be measured when the needle (210) passes through substantially the same point of the laser beam (LB1, LB2), wherein the first clockwise measurement coordinate (CW1) is measured while the needle (210) moves clockwise and the first counterclockwise measurement coordinate (CCW1) is measured while the needle (210) moves counterclockwise.
[0090] Accordingly, from adjacent (or paired) clockwise measurement coordinates (CW1, CW2, CW3, CW4) and counterclockwise measurement coordinates (CCW1, CCW2, CCW3, CCW4), the coordinates (hereinafter also referred to as "measurement coordinates") at which the needle (210) is located in the center of the width direction of the laser beam (LB1, LB2) can be calculated.
[0091] Specifically, a first measurement coordinate (P1) can be calculated from a first clockwise measurement coordinate (CW1) and a first counterclockwise measurement coordinate (CCW1). The first measurement coordinate (P1) is an intermediate coordinate between the first clockwise measurement coordinate (CW1) and the first counterclockwise measurement coordinate (CCW1), and can be calculated by finding the average value of the two coordinates.
[0092] A second measurement coordinate (P2) can be calculated from a second clockwise measurement coordinate (CW2) and a second counterclockwise measurement coordinate (CCW2). The second measurement coordinate (P2) is an intermediate coordinate between the second clockwise measurement coordinate (CW2) and the second counterclockwise measurement coordinate (CCW2), and can be calculated by finding the average value of the two coordinates.
[0093] A third measurement coordinate (P3) can be calculated from the third clockwise measurement coordinate (CW3) and the third counterclockwise measurement coordinate (CCW3). The third measurement coordinate (P3) is an intermediate coordinate between the third clockwise measurement coordinate (CW3) and the third counterclockwise measurement coordinate (CCW3), and can be calculated by finding the average value of the two coordinates.
[0094] A fourth measurement coordinate (P4) can be calculated from the fourth clockwise measurement coordinate (CW4) and the fourth counterclockwise measurement coordinate (CCW4). The fourth measurement coordinate (P4) is an intermediate coordinate between the fourth clockwise measurement coordinate (CW4) and the fourth counterclockwise measurement coordinate (CCW4), and can be calculated by finding the average value of the two coordinates.
[0095] Referring further to FIG. 7d, in step S240, two straight lines (LL1, LL2) can be determined based on the measured coordinates. The two straight lines (LL1, LL2) can each correspond to two laser beams (LB1, LB2).
[0096] Specifically, a first straight line (LL1) passing through the first measurement coordinates (P1) and the second measurement coordinates (P2) can be determined. The first measurement coordinates (P1) and the second measurement coordinates (P2) may be coordinates obtained when the needle (210) meets the first laser beam (LB1), and thus the first straight line (LL1) obtained therefrom may correspond to the first laser beam (LB1).
[0097] A second straight line (LL2) passing through the third measurement coordinates (P3) and the fourth measurement coordinates (P4) can be determined. The third measurement coordinates (P3) and the fourth measurement coordinates (P4) may be coordinates obtained when the needle (210) meets the second laser beam (LB2), and thus the second straight line (LL2) obtained therefrom may correspond to the second laser beam (LB2).
[0098] The coordinates of the intersection point (LL_C) can be calculated from the determined first and second straight lines (LL1, LL2).
[0099] In step S250, by considering the intersection point (LL_C) of the first and second straight lines (LL1, LL2) and the intersection point (LB_C) of the first and second laser beams (LB1, LB2), second position information regarding the actual position of the needle (210) can be calculated.
[0100] Specifically, when the design position and the actual position of the needle (210) are the same, the coordinates of the intersection point (LB_C) of the first and second laser beams (LB1, LB2) and the coordinates of the intersection point (LL_C) of the first and second straight lines (LL1, LL2) may match. In this case, the design position of the needle (210) may correspond to the actual position of the needle (210), and thus the design position of the needle (210) may be calculated as second position information for the actual position of the needle (210).
[0101] In contrast, if the design position and the actual position of the needle (210) are different, the coordinates of the intersection point (LB_C) of the first and second laser beams (LB1, LB2) may not match the coordinates of the intersection point (LL_C) of the first and second straight lines (LL1, LL2). In this case, the difference between the coordinates of the intersection point (LL_C) of the first and second straight lines (LL1, LL2) and the coordinates of the intersection point (LB_C) of the first and second laser beams (LB1, LB2) may correspond to the difference between the design position and the actual position of the needle (210). Therefore, by reflecting the difference between the coordinates of the intersection point (LL_C) of the first and second straight lines (LL1, LL2) and the coordinates of the intersection point (LB_C) of the first and second laser beams (LB1, LB2) in the design position of the needle (210), second position information regarding the actual position of the needle (210) can be calculated.
[0102] The calculation of the second position information of the needle (210) according to steps S210 to S250 can be performed individually for at least some of the plurality of needles (210) included in the picking module (200).
[0103] In one embodiment, the calculation of second position information of the needle (210) according to steps S210 to S250 may be performed for all of the plurality of needles (210) included in the picking module (200). For example, when a new picking module (200) is coupled to the head assembly (50), the processor (400) may calculate the second position information individually for all of the plurality of needles (210) included in the picking module (200).
[0104] In one embodiment, the calculation of second position information of the needle (210) according to steps S210 to S250 may be performed for some of the plurality of needles (210) included in the picking module (200). For example, if there is no replacement of the picking module (200) but verification of the needle (210) position is required, the processor (400) may calculate second position information only for some of the plurality of needles (210) included in the picking module (200). For example, if the plurality of needles (210) are arranged in a grid shape, second position information may be calculated only for the needles placed at the vertices of the grid.
[0105] In one embodiment, the second position information of the needle (210) may include position information of the end of the needle (210). In this regard, referring further to FIG. 8, the position of the end of the needle (210) can be obtained through a position measurement module (300). Specifically, the position information of the end of the needle can be obtained by moving the needle (210) onto the intersection point (LB_C) between the laser beam (LB) of the position measurement module (300) (e.g., the first laser beam (LB1) and the second laser beam (LB2)), then lowering the needle (210) and obtaining the coordinates of the needle when a blocking signal is generated at the light receiving unit (314, 318).
[0106] The processor (400) can store the second position information of the needle (210) calculated in step S200 in memory (500).
[0107] Referring again to FIG. 5, in step S300, the relative positions of the camera (110) and the needle (210) can be calculated based on the first position information of the camera (110) and the second position information of the needle (210) obtained through the position measurement module (300). Additionally, in step S400, the calculated relative positions of the camera (110) and the needle (210) can be stored in the memory (500).
[0108] In the automatic picking device (10), a camera (110) is moved over a target substance to acquire an image of the target substance, and then analyzed to detect the exact location of the target substance, and a needle (210) is moved over the detected target substance to perform picking of the target substance.
[0109] In this regard, in the embodiments of the present disclosure, the relative position between the camera (110) and the needle (210) can be precisely calculated through the position measuring module (300), and picking of the target material can be performed based thereon. Accordingly, the picking accuracy of the target material can be improved.
[0110] As explained above, a person skilled in the art of the present disclosure will recognize that the present disclosure may be implemented in various forms without altering its technical principles or core features. Accordingly, it should be understood that the above embodiments are merely illustrative and do not limit the scope of the present disclosure. The scope of the present disclosure is defined by the claims below rather than the detailed description, and all variations or modifications based on the meaning and scope of the claims and equivalent concepts should be interpreted as being included within the scope of the present disclosure.
[0111] The features and advantages described herein are only partial, and further features and advantages will become apparent to those skilled in the art upon reference to the drawings, specification, and claims. Additionally, it should be noted that the language used herein is chosen for readability and illustrative purposes and is not necessarily chosen to limit or describe the subject matter of this disclosure.
[0112] The description of the above embodiments is provided for illustrative purposes only and is not intended to limit the scope of the disclosure in its exact form. A person skilled in the art will understand that various modifications and variations are possible from the content of the disclosure.
[0113] Therefore, the scope of the present disclosure is not limited by the detailed description but is defined by the claims of this specification. Accordingly, the embodiments of the present disclosure are exemplary and do not limit the scope of the present disclosure as set forth in the claims below.
Claims
1. In an automatic picking device, One or more processors; One or more memories storing at least one instruction executed by the above one or more processors; A vision module including a camera configured to acquire an image of a target substance; A picking module comprising a needle configured to pick the above target substance; and It includes a position measurement module configured to acquire first position information of the camera and second position information of the needle, and The above one or more processors execute the above at least one instruction, Acquire first position information of the camera through the above position measurement module, and The second position information of the needle is obtained through the above position measurement module, and An automatic picking device configured to calculate the relative position of the camera and the needle based on the first position information and the second position information.
2. In Paragraph 1, The above position measuring module includes a laser sensor that generates a first laser beam and a second laser beam that intersect each other, and An automatic picking device configured such that the processor acquires the coordinates of the needle when either the first laser beam and the second laser beam is at least partially blocked by the needle.
3. In Paragraph 2, The above laser sensor is: A first light-emitting unit that generates the first laser beam; A first light receiving unit that receives the first laser beam; A second light-emitting unit that generates the second laser beam; and It includes a second light receiving unit that receives the second laser beam, and The above laser sensor is configured to generate a blocking signal when the intensity of the first laser beam received by the first light receiving unit or the second laser beam received by the second light receiving unit decreases to a predetermined value or less, in an automatic picking device.
4. In Paragraph 3, An automatic picking device configured such that the processor is configured to acquire the coordinates of the needle when the blocking signal is generated.
5. In Paragraph 2, Acquiring the second position information of the above needle is: Rotating the needle clockwise around the intersection point of the first and second laser beams; and An automatic picking device comprising rotating the needle counterclockwise around the intersection point of the first and second laser beams.
6. In Paragraph 5, An automatic picking device, wherein the second position information of the above needle is calculated by considering clockwise measurement coordinates obtained while rotating the above needle clockwise and counterclockwise measurement coordinates obtained while rotating the above needle counterclockwise.
7. In Paragraph 6, The above clockwise measurement coordinates are: First clockwise measurement coordinates and second clockwise measurement coordinates obtained by the needle blocking the first laser beam; and The above needle includes third clockwise measurement coordinates and fourth clockwise measurement coordinates obtained by blocking the second laser beam, and The above counterclockwise measurement coordinates are: First counterclockwise measurement coordinates and second counterclockwise measurement coordinates obtained by the needle blocking the first laser beam; and The above needle includes third counterclockwise measurement coordinates and fourth counterclockwise measurement coordinates obtained by blocking the second laser beam, and The first clockwise measuring coordinate and the first counterclockwise measuring coordinate, the second clockwise measuring coordinate and the second counterclockwise measuring coordinate, the third clockwise measuring coordinate and the third counterclockwise measuring coordinate, and the fourth clockwise measuring coordinate and the fourth counterclockwise measuring coordinate are each adjacent to each other, in an automatic picking device.
8. In Paragraph 7, The above processor is: Calculate the first measurement coordinate from the first clockwise measurement coordinate and the first counterclockwise measurement coordinate, and Calculate the second measurement coordinates from the second clockwise measurement coordinates and the second counterclockwise measurement coordinates, and Calculate the third measurement coordinates from the above third clockwise measurement coordinates and the above third counterclockwise measurement coordinates, and, An automatic picking device configured to calculate a fourth measurement coordinate from the fourth clockwise measurement coordinate and the fourth counterclockwise measurement coordinate.
9. In Paragraph 8, The above processor is: Determine a first straight line passing through the first measurement coordinates and the second measurement coordinates, and Determine a second straight line passing through the third measurement coordinate and the fourth measurement coordinate, and Calculate the coordinates of the intersection point of the first straight line and the second straight line, and An automatic picking device configured to calculate the second position information of the needle from the coordinates of the intersection point of the first straight line and the second straight line and the coordinates of the intersection point of the first laser beam and the second laser beam.
10. In Paragraph 5, An automatic picking device for obtaining second position information of the above needle, further comprising moving the needle onto the intersection point of the first and second laser beams and then lowering the needle.
11. In Paragraph 2, The above position measuring module further includes a light source formed below the intersection point of the first laser beam and the second laser beam, and An automatic picking device, wherein acquiring the first position information of the above camera includes capturing light generated from the light source unit with the above camera.
12. In Paragraph 11, The above processor is an automatic picking device configured to obtain the first position information of the camera from the coordinates of the camera when the light generated from the light source is located at the center of the camera.
13. In Paragraph 1, The above picking module includes a plurality of needles, and The above processor is an automatic picking device configured to calculate second position information for all of the plurality of needles.
14. In Paragraph 1, The above picking module includes a plurality of needles arranged in a grid shape, and The above processor is configured to acquire second position information for at least some of the plurality of needles or for needles placed at the vertices of the grid, an automatic picking device.
15. As a method of operating an automatic picking device, In the processor of the automatic picking device, a step of obtaining first position information of a camera included in the automatic picking device through a position measurement module; In the above processor, the step of obtaining second position information of a needle included in the automatic picking device through the position measuring module; and A method of operating an automatic picking device comprising the step of calculating the relative position of the camera and the needle based on the first position information and the second position information in the processor above.