Feature recognition guided robot dispensing method, computing device, and storage medium
By using feature recognition to guide the robotic arm to adjust the light source and camera parameters, the coordinates of the fiber optic coupler can be accurately identified, solving the accuracy and consistency problems caused by differences in part features during the dispensing process of optical components, and achieving a highly efficient and precise dispensing effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN RAYCUS FIBER LASER TECHNOLOGY CO LTD
- Filing Date
- 2023-12-27
- Publication Date
- 2026-05-29
AI Technical Summary
In the existing process of dispensing optical components, it is difficult to adapt to the differences in characteristics of different parts, resulting in insufficient dispensing position accuracy and poor consistency of glue dot shape, which fails to meet the requirements of material consistency and tooling consistency of optical components.
By using feature recognition to guide the robotic arm, the light source and camera parameters are adjusted according to the characteristics of different parts of the fiber optic coupler, the coordinates of the parts are accurately identified, and the dispensing coordinates of the robotic arm are calculated to achieve precise dispensing.
It improves the accuracy and efficiency of dispensing, meets the dispensing needs of different parts of optical devices, and enhances the accuracy of dispensing position and the consistency of glue dots.
Smart Images
Figure CN117732677B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent dispensing by robotic arms, and more specifically to a method for feature recognition and positioning guidance of dispensing by a robotic arm, a computing device, and a storage medium. Background Technology
[0002] Currently in manufacturing, the packaging process of optical components, such as fiber optic couplers, involves dispensing adhesive to different parts of the optical components. These different parts of the optical components have significant differences in characteristics. Existing optical component packaging methods often involve manual packaging with tooling assistance or single-machine multi-axis dispensing, using high-speed chains, roller lifts, vision positioning stations, and UV curing processes. This makes it difficult to control the consistency of adhesive quantity and shape, and each dispensing station can only perform a single dispensing operation. This approach cannot adapt to the significant differences in characteristics between different parts of the optical components, resulting in problems such as poor consistency of incoming materials and insufficient consistency of tooling boards. Consequently, this leads to insufficient dispensing position accuracy and poor consistency of adhesive dot shape. Summary of the Invention
[0003] This application provides a feature recognition-guided robotic arm dispensing method, computing device, and storage medium. For different parts of the coupler, the light source is adjusted according to the lighting conditions that match the features of the parts, so that the coordinate recognition of the feature coordinates is more accurate and the coordinates of the robotic arm dispensing are precise.
[0004] In a first aspect, embodiments of this application provide a method for dispensing adhesive using a feature recognition-guided robotic arm, comprising:
[0005] Based on the image of the tooling plate captured by the camera, the characteristics of the fiber optic coupler are identified, and the fiber optic coupler is placed on the tooling plate.
[0006] When the feature of the part is identified as a cone feature, the light source is adjusted according to the preset lighting conditions corresponding to the cone, the cone image is captured by the camera, and the center pixel coordinates of the cone are identified based on the cone image;
[0007] When the feature of the part is identified as a pipe cut-out feature, the light source is adjusted according to the preset lighting conditions corresponding to the pipe cut-out, the image of the pipe cut-out is captured by the camera, and the pixel coordinates of the pipe cut-out are identified based on the image of the pipe cut-out.
[0008] When the feature of the part is identified as a glass tube opening feature, the light source is adjusted according to the preset lighting conditions corresponding to the glass tube opening, the image of the glass tube opening is captured by the camera, and the pixel coordinates of the glass tube opening are identified based on the glass tube opening image.
[0009] The dispensing coordinates of the robot arm are calculated based on the pixel coordinates of the center of the cone area, the pixel coordinates of the tube opening, and the pixel coordinates of the glass tube opening.
[0010] The robotic arm controls the dispensing machine to dispense adhesive at the dispensing coordinates.
[0011] In one embodiment, when the feature of the part is identified as a cone-shaped feature, adjusting the light source according to the preset lighting conditions corresponding to the cone-shaped feature includes:
[0012] When the feature of the area is identified as a cone-shaped feature, the light source is turned off, and the exposure time of the camera is increased, or...
[0013] When the feature of the area is identified as a cone-shaped feature, the light source is turned off, the exposure time of the camera is increased, and the natural light around the lens above the fiber coupler is partially blocked.
[0014] In one embodiment, the light source includes a coaxial light source, and the step of adjusting the light source according to preset lighting conditions corresponding to the cut-off point when the feature of the part is identified as a cut-off point includes:
[0015] When the feature of the part is identified as a cut-off pipe feature, the coaxial light source is turned on and the camera exposure time is reduced.
[0016] In one embodiment, the light source includes a strip light source and a coaxial light source. When the feature of the part is identified as a glass tube opening, adjusting the light source according to preset lighting conditions corresponding to the glass tube opening includes:
[0017] When the feature of the part is identified as a glass tube opening, the coaxial light source and the strip light source are turned on, and the camera exposure time is reduced.
[0018] In one embodiment, calculating the dispensing coordinates of the robot arm based on the pixel coordinates of the cone center, the pixel coordinates of the scissor opening, and the pixel coordinates of the glass tube opening includes:
[0019] Calculate the midpoint coordinates of the center pixel coordinates of the cone region and the pixel coordinates of the cut-off nozzle; obtain the first glue dot coordinates based on the midpoint coordinates;
[0020] The second adhesive coordinates are obtained based on the pixel coordinates of the glass tube opening.
[0021] In one embodiment, controlling the dispensing machine to dispense adhesive at the dispensing coordinates via the robotic arm includes:
[0022] The robotic arm controls the dispensing machine to fold the glue at the first dispensing coordinate point;
[0023] The robotic arm controls the dispensing machine to fold the glue at the second dispensing coordinate point.
[0024] In one embodiment, there are multiple tooling plates, each tooling plate has a unique number, and the method further includes:
[0025] The camera captures an image of the tooling plate and identifies the number. The robot arm then executes actions according to the robot arm action instructions corresponding to the number.
[0026] In one embodiment, the method further includes:
[0027] The edge pixel coordinates of the glass cover are identified based on the image of the tooling plate captured by the camera. The third dispensing coordinates are obtained based on the edge pixel coordinates of the glass cover. The dispensing machine is controlled by the robot arm to dispense glue at the third dispensing coordinates.
[0028] The dispensing machine needle applies adhesive to the surface of the glass cover plate.
[0029] In a second aspect, embodiments of this application provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the feature recognition-guided robotic arm dispensing method described in the first aspect.
[0030] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the feature recognition-guided robotic arm dispensing method as described in the first aspect.
[0031] This application adjusts the light source and camera parameters to match the different features of the coupler, based on the characteristics of different parts, to accurately identify the coordinates of the different features and calculate the precise coordinates of the robot arm dispensing glue, thereby improving the accuracy and efficiency of glue dispensing. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 This is a schematic diagram of the structure of the feature recognition-guided robotic arm dispensing device provided in the embodiments of this application;
[0034] Figure 2 This is a schematic flowchart of an embodiment of the feature recognition-guided robotic arm dispensing method provided in this application.
[0035] Figure 3 This is a schematic diagram of a fiber optic coupler in one embodiment of this application;
[0036] Figure 4 This is a schematic diagram of the structure of a mobile phone in one embodiment of this application;
[0037] Figure 5 This is a schematic diagram of a server structure in one embodiment of this application.
[0038] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation
[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0040] In the following description, specific embodiments of this application will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of this application are described in the foregoing text, which is not intended to be limiting, and those skilled in the art will understand that many of the steps and operations described below can also be implemented in hardware.
[0041] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.
[0042] This application provides a feature recognition-guided robotic dispensing method, a computing device, and a storage medium.
[0043] Please see Figure 1 , Figure 1 This is a schematic diagram of a feature recognition-guided robotic dispensing device provided in an embodiment of this application. The device may include a four-axis robotic arm (1), a dispensing machine (2), a camera (3), a strip light source (4), and a coaxial light source (5). The four-axis robotic arm (1) and the dispensing machine (2) are connected via a connecting sleeve (6). The strip light source (4) and the coaxial light source (5) are collectively referred to as light sources. The light sources are connected to the camera (3) mounting base via sheet metal parts. The height of the light sources and the camera (3) is adjustable. It should be noted that the coaxial light source and the strip light source are merely examples. It is understood that there may be one or more light sources, and they may be other types of light sources, such as circular light sources. Specific details are not limited here.
[0044] Optionally, the feature recognition-guided dispensing device for the robotic arm may also include a host computer. The camera and the host computer are connected via Ethernet, and the host computer and the robotic arm communicate via Ethernet. The light source is connected to the robotic arm via I / O hardwire or bus. The I / O is used for interactive input and output control signals, and the Ethernet communication is used for interactive visual recognition data.
[0045] It should be noted that, Figure 1 The schematic diagram of the feature recognition-guided robotic arm dispensing device shown is merely an example. The feature recognition-guided robotic arm dispensing device and scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of feature recognition-guided robotic arm dispensing devices and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.
[0046] Based on different part features, this application embodiment adjusts the light source and camera parameters to match the different part features of the coupler, accurately identifies the coordinates of different part features, calculates the precise coordinates of the robot arm dispensing glue, and improves the accuracy and efficiency of glue dispensing.
[0047] The following detailed description is based on specific embodiments.
[0048] In this embodiment, the method of guiding a robotic arm to dispense adhesive will be described from the perspective of feature recognition.
[0049] This application provides a feature recognition-guided robotic dispensing method, which includes: identifying features of an optical fiber coupler based on an image of a tooling plate captured by a camera, wherein the optical fiber coupler is placed on the tooling plate; when the feature is identified as a conical region, adjusting the light source according to preset lighting conditions corresponding to the conical region, capturing an image of the conical region with the camera, and identifying the center pixel coordinates of the conical region based on the image; when the feature is identified as a tube opening feature, adjusting the light source according to preset lighting conditions corresponding to the tube opening, capturing an image of the tube opening with the camera, and identifying the tube opening pixel coordinates based on the image; when the feature is identified as a glass tube opening feature, adjusting the light source according to preset lighting conditions corresponding to the glass tube opening, capturing an image of the glass tube opening with the camera, and identifying the glass tube opening pixel coordinates based on the image; calculating the dispensing coordinates of the robotic arm based on the center pixel coordinates of the conical region, the tube opening pixel coordinates, and the glass tube opening pixel coordinates; and controlling the dispensing machine to dispense adhesive at the dispensing coordinates using the robotic arm.
[0050] Please see Figure 2 , Figure 2 This is a schematic flowchart of an embodiment of the feature recognition-guided robotic arm dispensing method in this application. The feature recognition-guided robotic arm dispensing method includes the following steps 201 to 206:
[0051] 201. Based on the image of the tooling plate captured by the camera, identify the features of the fiber optic coupler and place the fiber optic coupler on the tooling plate.
[0052] The camera can be an industrial camera, offering higher image stability and transmission capabilities. The camera is connected via a camera mounting plate. When capturing an image, the camera automatically adjusts the exposure based on the ambient natural light conditions. The fixture plate is used to place the fiber optic coupler on the packaging production line. Optionally, the feature recognition-guided robotic dispensing method's related devices can be mounted on one side of the fixture plate and connected via a robotic arm mounting base. The fiber optic coupler to be dispensed is placed on the fixture plate; please refer to [link to relevant documentation]. Figure 3 The structure of the fiber optic coupler is as follows Figure 3 As shown, the stripped section, tapered section, cut end, and glass tube end are placed on the base. The fiber optic coupler can be composed of multiple coupled fibers or a single coupled fiber. When the fixture plate is passed under the camera, a shooting command is sent to the camera, and the camera takes a picture of the fixture plate to obtain an image. For example, when the fixture plate is passed under the camera, the camera receives a shooting command, automatically triggers the shutter button to take a picture, and obtains an image of the fixture plate.
[0053] 202. When the feature of the part is identified as a cone feature, the light source is adjusted according to the preset lighting conditions corresponding to the cone, the cone image is captured by the camera, and the center pixel coordinates of the cone are identified based on the cone image.
[0054] Specifically, when a cone-shaped feature is detected in an image captured by the camera, the light source is adjusted according to preset lighting conditions corresponding to the cone. The cone image features are pre-set in a relevant device. By performing feature recognition and comparison on the captured image, it is determined whether the cone feature exists. For example, if the cone has a continuously changing diameter in one direction, after capturing the image, the presence of the cone feature is identified, and the preset feature image and the captured image are compared to determine whether the cone feature exists in the captured image. The preset lighting conditions are the most suitable lighting conditions for identifying the cone. The lighting conditions may include the on / off state of the light source, camera parameters and the light source, and the camera's height and position. After adjusting the light source according to the preset lighting conditions, the image of the cone is recaptured by the camera, and the pixel coordinates of the cone's center are identified from the cone image.
[0055] 203. When the feature of the part is identified as a pipe cut-out, the light source is adjusted according to the preset lighting conditions corresponding to the pipe cut-out, and the image of the pipe cut-out is captured by the camera. The pixel coordinates of the pipe cut-out are identified based on the image of the pipe cut-out.
[0056] Specifically, when a cut-off pipe feature is detected in an image captured by the camera, the light source is adjusted according to preset lighting conditions corresponding to the cut-off pipe. The cut-off pipe image feature is pre-set in a relevant device. By performing feature recognition and comparison on the captured image, it is determined whether the cut-off pipe feature exists in the captured image. Specifically, the cut-off pipe has a large diameter and irregular broken stripes. The preset lighting conditions are the most suitable lighting conditions for identifying the cut-off pipe. The lighting conditions can include the on / off state of the light source, camera parameters and the light source, and the camera's height position. After adjusting the light source according to the preset lighting conditions, the image of the cut-off pipe is re-captured by the camera, and the pixel coordinates of the cut-off pipe are identified from the image.
[0057] 204. When the feature of the part is identified as a glass tube opening feature, the light source is adjusted according to the preset lighting conditions corresponding to the glass tube opening, the image of the glass tube opening is captured by the camera, and the pixel coordinates of the glass tube opening are identified based on the image of the glass tube opening.
[0058] Specifically, when the presence of a glass tube opening feature is detected in an image captured by the camera, the light source is adjusted according to preset lighting conditions corresponding to the glass tube opening. The glass tube opening image features are pre-set in a relevant device; by performing feature recognition and comparison on the captured image, it is determined whether the glass tube opening feature exists in the captured image. Specifically, compared to the conical area and the sheared tube opening, the glass tube opening has the largest diameter and is covered with glue. The preset lighting conditions are the most suitable lighting conditions for identifying the glass tube opening. These lighting conditions can include the on / off state of the light source, camera parameters and the light source, and the camera's height and position. After adjusting the light source according to the preset lighting conditions, the image of the glass tube opening is re-captured by the camera, and the pixel coordinates of the glass tube opening are identified from this image.
[0059] Among them, the cone region image, the cut tube opening image, and the glass tube opening image can be identified by different operators and then undergo coordinate transformation.
[0060] 205. Calculate the dispensing coordinates of the robot arm based on the pixel coordinates of the center of the cone area, the pixel coordinates of the tube opening, and the pixel coordinates of the glass tube opening.
[0061] Specifically, the dispensing position coordinates can be calculated based on the pixel coordinates of the cone center, the pixel coordinates of the cut tube opening, and the pixel coordinates of the glass tube opening. Then, the dispensing coordinates of the robot arm can be obtained through coordinate transformation.
[0062] 206. The dispensing machine is controlled by the robotic arm to dispense adhesive at the dispensing coordinates.
[0063] Specifically, after coordinate conversion, the dispensing coordinates of the robotic arm are obtained. The robotic arm and the dispensing machine are connected by a connecting sleeve. By controlling the movement of the robotic arm, the dispensing machine can be controlled to dispense glue at the dispensing coordinates.
[0064] This application adjusts the light source and camera parameters to match the different features of the coupler, based on the characteristics of different parts, to accurately identify the coordinates of the different features and calculate the precise coordinates of the robot arm dispensing glue, thereby improving the accuracy and efficiency of glue dispensing.
[0065] Specifically, in one embodiment of this application, when the feature of the part is identified as a cone-shaped feature, adjusting the light source according to the preset lighting conditions corresponding to the cone-shaped feature includes:
[0066] (1) When the feature of the part is identified as a cone feature, the light source is turned off and the exposure time of the camera is increased.
[0067] Specifically, when the feature of the area is identified as a cone, the best lighting conditions that are suitable for the cone are to turn off all light sources, such as turning off coaxial light sources and bar light sources, while adjusting the camera's shooting parameters, such as adjusting the camera's exposure time parameters and increasing the camera's exposure time.
[0068] (2) When the feature of the part is identified as a cone feature, the light source is turned off, the exposure time of the camera is increased, and the natural light around the lens above the fiber coupler is partially blocked.
[0069] Optionally, when a cone-shaped feature is identified, all light sources can be turned off, the camera's exposure time parameters adjusted to increase the exposure time, and the camera lens partially obscured. For example, if the camera lens has a protective case, the lens can be partially obscured by partially opening the case, or a baffle positioned below the camera lens can be used to partially obscure it. Optionally, the height and position of the light source and camera can be adjusted adaptively according to the cone-shaped feature position on the tooling plate image.
[0070] This application embodiment utilizes the method of turning off the light source, adjusting camera parameters, and partial occlusion to provide matching lighting conditions for identifying the cone area. The features of the cone area are clearer, and the pixel coordinates of the cone area can be identified more accurately.
[0071] In one embodiment of this application, the light source includes a coaxial light source, and the step of adjusting the light source according to preset lighting conditions corresponding to the cut-out when the feature of the part is identified as a cut-out feature includes:
[0072] When the feature of the part is identified as a cut-off pipe feature, the coaxial light source is turned on and the camera exposure time is reduced.
[0073] Specifically, the light source can include a coaxial light source. When the identified feature is a cut-off pipe opening, the lighting conditions adapted to the cut-off pipe opening can be to turn on the coaxial light source, turn off the bar light source, and simultaneously adjust the camera's image capture parameters, such as adjusting the camera's exposure time parameter and lowering the camera's exposure time. Optionally, the light source also includes a bar light source, and the lighting conditions adapted to the cut-off pipe opening can also be to turn on both the bar light source and the coaxial light source, and simultaneously adjust the camera's image capture parameters, such as adjusting the camera's exposure time parameter and lowering the camera's exposure time. Optionally, the height and position of the light source and camera can be adjusted adaptively according to the position of the cone area on the tooling plate image.
[0074] This application embodiment utilizes the light source to adjust camera parameters to provide matching lighting conditions for identifying the cut-off pipe opening, making the features of the cut-off pipe opening clearer and enabling more accurate identification of the center coordinates of the cut-off pipe opening.
[0075] In one embodiment of this application, the light source includes a strip light source and a coaxial light source. The step of adjusting the light source according to preset lighting conditions corresponding to the glass tube opening when the feature is identified as a glass tube opening includes:
[0076] When the feature of the part is identified as a glass tube opening, the coaxial light source and the strip light source are turned on, and the camera exposure time is reduced.
[0077] Specifically, the light source can include a strip light source and a coaxial light source. When the identified feature is a glass tube opening, the lighting conditions adapted to the glass tube opening could be to turn on both the strip light source and the coaxial light source, while adjusting the camera's shooting parameters, such as adjusting the camera's exposure time parameter and lowering the camera's exposure time. Optionally, the lighting conditions adapted to the tube opening could also be to turn on only the coaxial light source and turn off the strip light source, while adjusting the camera's shooting parameters, such as adjusting the camera's exposure time parameter and lowering the camera's exposure time. Optionally, the height and position of the light source and camera can be adjusted adaptively according to the position of the cone area on the tooling plate image.
[0078] This application embodiment utilizes the opening of a light source and the adjustment of camera parameters to provide matching lighting conditions for identifying the glass tube opening. The features of the glass tube opening are clearer, and the pixel coordinates of the glass tube opening can be identified more accurately.
[0079] In one embodiment of this application, the step of calculating the dispensing coordinates of the robot arm based on the pixel coordinates of the cone center, the pixel coordinates of the scissor opening, and the pixel coordinates of the glass tube opening includes:
[0080] (1) Calculate the midpoint coordinates of the center pixel coordinates of the cone area and the pixel coordinates of the cut tube opening; obtain the first glue coordinates based on the midpoint coordinates.
[0081] Specifically, the center pixel coordinates of the cone region and the pixel coordinates of the cut-off nozzle are obtained, and the midpoint coordinates of the two coordinates are calculated to obtain the midpoint coordinates (x, y). 中 y 中 The midpoint coordinates are transformed using the coordinate system calibrated by the camera to obtain the first point coordinate (x) relative to the robot arm. 中′ , y中′ ).
[0082] (2) Obtain the second glue coordinates based on the pixel coordinates of the glass tube opening.
[0083] Specifically, the pixel coordinates (x, y) of the glass tube opening are obtained. 管口 y 管口 The pixel coordinates of the glass tube opening are converted to robot arm coordinates using the coordinate transformation specified by the camera calibration, resulting in the second adhesive dot coordinates (x, y) relative to the robot arm. 管口′ y 管口′ ).
[0084] It should be noted that the coordinate labels in the above embodiments are only examples. It is understood that different coordinate labels may be used in other embodiments, such as the first glue coordinate (x1, y1) and the second glue coordinate (x2, y2). The specifics are not limited here.
[0085] In this embodiment, after obtaining the pixel coordinates of different features of the fiber coupler, the dispensing coordinates are converted into dispensing coordinates relative to the robotic arm through coordinate transformation calibrated by the camera, thereby controlling the robotic arm to precisely dispense adhesive.
[0086] In one embodiment of this application, the step of controlling the dispensing machine to dispense adhesive at the dispensing coordinates via the robotic arm includes:
[0087] 1. The dispensing machine is controlled by the robotic arm to fold the glue at the first dispensing coordinate point.
[0088] Specifically, after obtaining the dispensing coordinates of the robotic arm, the robotic arm controls the dispensing machine at the first dispensing coordinate (x... 中′ y 中′ The dispensing process involves connecting the robotic arm and the dispensing machine via a connecting sleeve, ensuring that the movements of the robotic arm and the dispensing machine are synchronized.
[0089] 2. The dispensing machine is controlled by the robotic arm to fold the glue at the second dispensing coordinate point.
[0090] Specifically, after obtaining the dispensing coordinates of the robotic arm, the robotic arm controls the dispensing machine at the second dispensing coordinate (x... 管口′ y 管口′ That is, the glue is applied at the pixel coordinates of the glass tube opening. The robot arm and the glue dispensing machine are connected by a connecting sleeve, and the movement of the robot arm and the movement of the glue dispensing machine are synchronized.
[0091] This application embodiment uses different robotic arms to control the dispensing machine to dispense glue at different coordinate points, thereby meeting various dispensing needs and improving dispensing efficiency.
[0092] In one embodiment of this application, there are multiple tooling plates, each tooling plate has a unique number, and the method further includes:
[0093] The camera captures an image of the tooling plate and identifies the number. The robot arm then executes actions according to the robot arm action instructions corresponding to the number.
[0094] Specifically, there are multiple tooling plates, each with a unique number. The optical components placed on each tooling plate can be different, and the assembly line operations performed on each tooling plate can also be different. For example, tooling plate 1 holds an optical fiber tray, and the corresponding operation for tooling plate 1 is optical fiber tray loading; tooling plate 2 holds an optical fiber coupler, and the corresponding operation for tooling plate 2 is dispensing adhesive. Optionally, the tooling plates can be white, and the numbering can be black. Optionally, the numbering can also be in the form of a QR code or barcode, and the camera can obtain the tooling plate number by scanning the code. Optionally, each tooling plate number corresponds to a set of robotic arm motion instructions. For example, the robotic arm motion instruction for tooling plate 1 is optical fiber assembly, and the robotic arm motion instruction for tooling plate 2 is dispensing adhesive. The robotic arm motion instructions corresponding to each tooling plate number are pre-set.
[0095] This application embodiment eliminates the positioning errors of different tooling plates during processing and assembly by numbering the tooling plates and pre-setting the robot movement instructions corresponding to the tooling plates at different workstations, thereby improving the efficiency of tooling plate assembly and dispensing.
[0096] In one embodiment of this application, the method further includes:
[0097] The edge pixel coordinates of the glass cover plate are identified based on the image of the tooling plate captured by the camera. The third dispensing coordinates are obtained based on the edge pixel coordinates of the glass cover plate. The dispensing machine is controlled by the robotic arm to dispense adhesive at the third dispensing coordinates. The needle of the dispensing machine dispenses adhesive while in contact with the surface of the glass cover plate.
[0098] Specifically, the coordinates of pixels on the edge of the glass cover can be identified from the image of the tooling plate captured by the camera. After coordinate transformation calibrated by the camera, the pixel coordinates of the glass cover edge are converted into third dispensing coordinates relative to the robotic arm. The robotic arm then controls the dispensing machine to dispense adhesive at these third dispensing coordinates. Specifically, since the dispensing between the edges of the glass cover and the base is height-sensitive, the height of the adhesive must be carefully controlled during dispensing. Optionally, the dispensing machine can be a height-responsive dispensing device, where the dispensing needle can dispense adhesive while remaining flush with the surface of the glass cover.
[0099] This application embodiment improves the dispensing effect at the edge of the glass cover by identifying the pixel coordinates of the glass cover and using a height-following dispensing device to make the dispensing needle stick to the surface of the glass cover for dispensing.
[0100] When the computing device in this application embodiment is a terminal device, this application embodiment also provides a terminal device, such as... Figure 4As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POs), in-vehicle computers, etc. Taking a mobile phone as an example:
[0101] Figure 4 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 4 The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that... Figure 4 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0102] The following is combined with Figure 4 A detailed introduction to each component of a mobile phone:
[0103] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRs), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).
[0104] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0105] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0106] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 based on the type of touch event. Although in Figure 4 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.
[0107] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.
[0108] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.
[0109] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 4 The Wi-Fi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.
[0110] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 1080.
[0111] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.
[0112] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.
[0113] In this embodiment of the application, the processor 1080 included in the mobile phone also has the function of controlling and executing the above-mentioned feature recognition-guided robotic arm dispensing method process executed by the related device of the feature recognition-guided robotic arm dispensing method.
[0114] This application also provides a server; please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 may be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.
[0115] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.
[0116] The steps in the feature recognition-guided robotic dispensing method in the above embodiments can be based on this. Figure 5 The structure of server 1100 is shown.
[0117] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0119] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.
[0120] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0121] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0122] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.
Claims
1. A method for dispensing adhesive using a feature recognition-guided robotic arm, characterized in that, The method includes: Based on the image of the tooling plate captured by the camera, the characteristics of the fiber optic coupler are identified, and the fiber optic coupler is placed on the tooling plate. When the feature of the part is identified as a cone feature, the light source is adjusted according to the preset lighting conditions corresponding to the cone, the cone image is captured by the camera, and the center pixel coordinates of the cone are identified based on the cone image; When the feature of the part is identified as a pipe cut-out feature, the light source is adjusted according to the preset lighting conditions corresponding to the pipe cut-out, the image of the pipe cut-out is captured by the camera, and the pixel coordinates of the pipe cut-out are identified based on the image of the pipe cut-out. When the feature of the part is identified as a glass tube opening feature, the light source is adjusted according to the preset lighting conditions corresponding to the glass tube opening, the image of the glass tube opening is captured by the camera, and the pixel coordinates of the glass tube opening are identified based on the glass tube opening image. The dispensing coordinates of the robot arm are calculated based on the pixel coordinates of the center of the cone area, the pixel coordinates of the tube opening, and the pixel coordinates of the glass tube opening. The robotic arm controls the dispensing machine to dispense adhesive at the dispensing coordinates.
2. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, When the feature of the part is identified as a cone-shaped feature, adjusting the light source according to the preset lighting conditions corresponding to the cone-shaped feature includes: When the feature of the area is identified as a cone-shaped feature, the light source is turned off, and the exposure time of the camera is increased, or... When the feature of the area is identified as a cone-shaped feature, the light source is turned off, the exposure time of the camera is increased, and the natural light around the lens above the fiber coupler is partially blocked.
3. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, The light source includes a coaxial light source. When the feature of the part is identified as a cut-off pipe feature, adjusting the light source according to preset lighting conditions corresponding to the cut-off pipe feature includes: When the feature of the part is identified as a cut-off pipe feature, the coaxial light source is turned on and the camera exposure time is reduced.
4. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, The light source includes a strip light source and a coaxial light source. When the feature of the part is identified as a glass tube opening, adjusting the light source according to preset lighting conditions corresponding to the glass tube opening includes: When the feature of the part is identified as a glass tube opening, the coaxial light source and the strip light source are turned on, and the camera exposure time is reduced.
5. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, The calculation of the dispensing coordinates of the robot arm based on the pixel coordinates of the cone center, the pixel coordinates of the slit opening, and the pixel coordinates of the glass tube opening includes: Calculate the midpoint coordinates of the center pixel coordinates of the cone region and the pixel coordinates of the cut-off nozzle; obtain the first glue dot coordinates based on the midpoint coordinates; The second adhesive coordinates are obtained based on the pixel coordinates of the glass tube opening.
6. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, The step of controlling the dispensing machine to dispense adhesive at the dispensing coordinates via the robotic arm includes: The robotic arm controls the dispensing machine to fold the glue at the first dispensing coordinate point; The robotic arm controls the dispensing machine to fold the glue at the second dispensing coordinate point.
7. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, There are multiple tooling plates, each with a unique number, and the method further includes: The camera captures an image of the tooling plate and identifies the number. The robot arm then executes actions according to the robot arm action instructions corresponding to the number.
8. The feature recognition-guided dispensing method for a robotic arm according to claim 1, characterized in that, The method further includes: The camera identifies the pixel coordinates of the glass cover edge in the image of the tooling plate, and obtains the third glue dispensing coordinates based on the pixel coordinates of the glass cover edge. The robotic arm controls the dispensing machine to dispense glue at the third glue dispensing coordinates. The dispensing machine needle applies adhesive to the surface of the glass cover plate.
9. A computing device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the feature recognition-guided robotic dispensing method as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, It includes instructions that, when executed on a computer, cause the computer to perform the feature recognition-guided robotic dispensing method as described in any one of claims 1 to 8.