Pin taking position recognition method and system based on image recognition and storage medium

Abstract

The invention belongs to the technical field of ultrasonic welding, and particularly relates to a needle taking position recognition method and system based on image recognition and a storage medium. The method comprises the following steps that S1, image information of multiple photos of positioning holes in a needle tray is sequentially collected through a camera; s2, obtaining photographing parameters and image information, and carrying out preprocessing and feature extraction on an image to obtain a circular contour of the positioning hole; s3, fitting the circular contours of the positioning holes in the plurality of pictures, and calculating to obtain the best fitting picture coordinates of the positioning holes; s4, the picture coordinates of the positioning holes are converted into mechanical coordinates, and the position information of the first needle taking hole of the needle material disc in the mechanical coordinates is obtained, so that the multiple needle taking holes of the needle material disc are positioned in sequence; according to the method, the pin taking position is automatically recognized through image recognition, positioning errors caused by manual operation are reduced, the accuracy of the pin taking position is ensured, the positioning precision is improved, and PIN welding failures caused by pin taking position deviation are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of ultrasonic welding, and particularly relates to a method, a system and a storage medium for identifying the needle-taking position based on image recognition. Background Art

[0002] Insulated Gate Bipolar Transistor (IGBT) is the most advanced power semiconductor device in the third generation of power electronic devices. It has the advantages of high frequency, high voltage, large current, etc., and is the core device in the fields of flexible AC / DC power transmission, new energy power generation, power quality governance, etc. At present, it has been widely used in related industries. In the IGBT module packaging, the PIN needles are fixed to the specified positions by welding to achieve the transmission of electricity and signals.

[0003] Before welding, the PIN needles in the needle tray need to be taken out sequentially by the suction nozzle. Since the positioning of the suction nozzle and the PIN needles is inaccurate, it is impossible to ensure that the moving position of the suction nozzle corresponds to each PIN needle. Usually, manual operation is required, which is prone to positioning errors, unable to ensure the accuracy of the needle-taking position, thus affecting the welding quality of the PIN needles. Moreover, manually identifying the needle-taking position takes a long time, reducing the overall production efficiency and unable to meet the requirements of modern production for high efficiency.

[0004] With the improvement of the automation level, images are usually collected by a camera to identify the positions of the PIN needles in the needle tray. However, the inventor found that there are at least the following technical problems in the existing technology:

[0005] On the one hand, when taking pictures of multiple needle-taking holes on the needle tray with the camera lens, due to the existence of multiple needle-sucking holes, it is impossible to distinguish the position of the first needle-taking hole on the needle tray, thus resulting in inaccurate positioning of the positions of several other needle-sucking holes on the needle tray; on the other hand, when replacing the needle tray, the PIN needles do not correspond to the needle-taking holes or there is a position deviation, resulting in needle-taking failure, and thus the welding of the PIN needles fails.

[0006] Based on this, it is necessary to improve the defects existing in the existing technology to overcome the deficiencies existing in practical applications. Summary of the Invention

[0007] Based on the above-mentioned shortcomings and deficiencies existing in the existing technology, one of the purposes of the present invention is to at least solve one or more of the above-mentioned problems existing in the existing technology. In other words, one of the purposes of the present invention is to provide a method, a system and a storage medium for identifying the needle-taking position based on image recognition that meet one or more of the foregoing requirements.

[0008] In order to achieve the above-mentioned invention purpose, the present invention adopts the following technical solutions:

[0009] The present invention provides a method for identifying the needle-taking position based on image recognition, comprising the following steps:

[0010] S1. Sequentially collect the image information of multiple photos of the positioning holes on the needle material tray through a camera;

[0011] S2. Obtain the photographing parameters and image information, and perform preprocessing and feature extraction on the image to obtain the circular contour of the positioning hole;

[0012] S3. Fit the circular contours of the positioning holes in multiple pictures, and calculate the picture coordinates of the best fit of the positioning holes;

[0013] S4. Convert the picture coordinates of the positioning holes into mechanical coordinates, and obtain the position information of the first needle-taking hole on the needle material tray in the mechanical coordinates, so as to sequentially position multiple needle-taking holes on the needle material tray.

[0014] As a preferred solution, before step S1, it further includes:

[0015] Input the photographing parameters of the camera into the PLC control system, and control the camera to move to the shooting position through the PLC control system to collect images of the needle material tray.

[0016] As a preferred solution, step S1 includes:

[0017] Collect images of the positioning holes on the needle material tray through a camera, and identify the picture coordinates of the positioning holes on the needle material tray through image processing.

[0018] As a preferred solution, step S2 includes:

[0019] S21. Perform filtering, noise reduction, and enhancement processing on the image to achieve preprocessing of the image;

[0020] S22. Perform feature extraction on the image, identify the picture coordinates of the positioning hole, and calculate the circular contour of the positioning hole according to the picture coordinate position.

[0021] As a preferred solution, step S3 includes:

[0022] S31. Fit the circular contours of the positioning holes in multiple pictures, and calculate the center and radius of the positioning hole;

[0023] S32. Calculate the picture coordinates of the best fit of the positioning hole according to the center and radius of the positioning hole.

[0024] As a preferred solution, step S4 includes:

[0025] S41. Convert the picture coordinates of the positioning hole into mechanical coordinates to obtain the position information of the needle-taking hole in the mechanical coordinates;

[0026] S42. Obtain the position information of the first needle-taking hole of the needle material tray in the mechanical coordinates to achieve the positioning of multiple needle-taking holes of the needle material tray in sequence.

[0027] As a preferred solution, after the step S4, the following steps are further included:

[0028] Input the position information of the first needle-taking hole of the needle material tray in the mechanical coordinates into the PLC control system for retrieving the position information instruction when taking needles from the needle-taking holes subsequently.

[0029] The present invention further provides a needle-taking position recognition system based on image recognition, which applies the method described in any of the above solutions, and includes:

[0030] An image acquisition module, which is used to sequentially acquire the image information of multiple photos of the positioning holes on the needle material tray through a camera;

[0031] An acquisition module, which is used to acquire the photographing parameters and image information, and perform preprocessing and feature extraction on the image to obtain the circular contour of the positioning hole;

[0032] An image fitting module, which is used to fit the circular contours of the positioning holes in multiple pictures and calculate the picture coordinates of the best fit of the positioning holes;

[0033] A coordinate conversion module, which is used to convert the picture coordinates of the positioning hole into mechanical coordinates and obtain the position information of the first needle-taking hole of the needle material tray in the mechanical coordinates to achieve the positioning of multiple needle-taking holes of the needle material tray in sequence.

[0034] The present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any of the above solutions are implemented.

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] The present invention provides a needle-taking position recognition method based on image recognition. By recognizing the positions of the positioning holes on the needle material tray through camera images, the mechanical coordinates of multiple needle-taking holes are obtained and positioned in sequence, reducing the positioning errors caused by manual operations, ensuring the accuracy of the needle-taking positions, improving the positioning accuracy, and reducing the failure of PIN needle soldering caused by the deviation of the needle-taking positions.

[0037] The present invention provides a needle-taking position recognition method based on image recognition. Through automated processing, the dependence on manual operations is reduced, the labor cost is lowered, it is beneficial to reduce operation errors, and it can be applied to different production environments and types of needle material trays, having good versatility and adaptability. Description of the Drawings

[0038] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings of embodiments can be obtained based on these drawings.

[0039] Figure 1 is a schematic flowchart of the needle-taking position recognition method based on image recognition according to an embodiment of the present invention;

[0040] Figure 2 is an image of the positioning holes of the needle stock tray collected by the camera according to an embodiment of the present invention;

[0041] Figure 3 is an image of the positioning holes of the needle stock tray from another perspective collected by the camera according to an embodiment of the present invention;

[0042] Figure 4 is a schematic structural diagram of the needle stock tray according to an embodiment of the present invention;

[0043] Figure 5 is a top view of the needle stock tray according to an embodiment of the present invention;

[0044] Figure 6 is a schematic structural diagram of the needle-taking position recognition system based on image recognition according to an embodiment of the present invention. Detailed Embodiments

[0045] To more clearly illustrate the embodiments of the present application, the following will describe the specific embodiments of the present application with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, and other embodiments can be obtained.

[0046] In the description of the embodiments of the present application, the orientation or positional relationships such as "upper", "lower", "front", "rear", etc. are based on the orientation or positional relationships shown in the accompanying drawings. It is only for the convenience of description and simplifying the operation, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, so it cannot be understood as a limitation to the present application. In addition, the terms "first", "second", etc. are only used for distinction in description and do not have special meanings.

[0047] In the prior art, when taking photos of multiple needle-taking holes on the needle material tray inside the camera lens, due to the existence of multiple needle-sucking holes, it is impossible to identify the position of the first needle-taking hole on the needle material tray, resulting in inaccurate positioning of the positions of several other needle-sucking holes on the needle material tray; on the other hand, when replacing the needle material tray, the PIN needles do not correspond to the needle-taking holes or there is a position deviation, resulting in needle-taking failure, and further leading to the welding failure of the PIN needles.

[0048] To solve the above technical problems, according to some embodiments of the present application, please refer to Figures 1 to 5 As shown, there is provided a method for identifying the needle-taking position based on image recognition, including the following steps:

[0049] S1. Sequentially collect the image information of multiple photos of the positioning holes on the needle material tray through a camera.

[0050] Specifically, the step S1 includes:

[0051] Collect multiple images of the positioning holes on the needle material tray through a camera, and identify the picture coordinates of the positioning holes on the needle material tray through image processing. Calibrate the camera through nine-point calibration to obtain the conversion matrix for calculating the drawing coordinates and the mechanical coordinates. During the calibration process, establish a camera model by taking point images through the camera, measure the coordinates of the calibration points in the image, and calibrate by calculating the external parameters of the camera to calculate the geometric transformation relationship, that is, the affine transformation matrix. Affine transformation is a linear mapping and can be represented by the following matrix:

[0052]

[0053] Among them, (X, Y) is the original point coordinate, (x, y) is the transformed point coordinate, a ij is the affine matrix element, t x , t y is the translation vector.

[0054] Furthermore, collect nine images of the needle material tray through the camera. The images contain the calibration points of the positioning holes, identify the picture coordinates of the positioning holes, and use the least squares method to fit the affine transformation matrix based on the actual coordinates of the positioning holes and the coordinates extracted from the image. Substitute the nine-point coordinates into the equation to obtain:

[0055]

[0056] Solving this system of equations can obtain the values of a 11 , a 12 , a 21 , a 22 , t x , t y . After the affine transformation matrix is generated, serialize the matrix in reverse and save it for subsequent calls, and then the saved matrix can be used.

[0057] S2. Obtain the photographing parameters and image information, and preprocess and extract features from the image to obtain the circular contour of the positioning hole.

[0058] Specifically, the step S2 includes:

[0059] S21. Filter, denoise, and enhance the image to preprocess the image;

[0060] S22. Extract features from the image, identify the picture coordinates of the positioning hole, and calculate the circular contour of the positioning hole according to the picture coordinate positions.

[0061] Furthermore, first rotate the image to the standard direction through the image rotation function for subsequent feature recognition. In this process, threshold processing is applied to binarize the image to highlight the region of interest.

[0062]

[0063] Among them, f(x, y) is the pixel value of the original image, T is the set threshold, and G(x, y) is the processed image.

[0064] Subsequently, process the extracted region of interest through dilation and erosion to eliminate noise and smooth the contour.

[0065] The formula for dilation is: D(x, y) = max{I(x + i, y + j)|(i, j) ∈ S}; the formula for erosion is: E(x, y) = min{I(x + i, y + j)|(i, j) ∈ S}; where I(x, y) is the input image and S is the structuring element.

[0066] Furthermore, perform opening operation to clarify features, effectively remove small noise points and smooth the contour. The formula for the opening operation is: O(x, y) = D(E(x, y)); where O(x, y) represents the circular contour of the positioning hole; E(x, y) represents the eroded image; D(x, y) represents the dilated image; O(x, y) is the finally processed image after the opening operation. After the noise is removed, the boundary of the target region is smoother and is used for further circle fitting of the positioning hole.

[0067] S3. Fit the circular contours of the positioning holes in multiple pictures and calculate the picture coordinates of the best fit of the positioning hole.

[0068] Specifically, the step S3 includes:

[0069] S31. Fit the circular contours of the positioning holes in multiple pictures and calculate the center and radius of the positioning hole;

[0070] S32. Calculate the picture coordinates of the best fit for the positioning hole based on the center and radius of the positioning hole.

[0071] To obtain the best fit of the circle, we can reformulate the objective function as minimizing the sum of the squares of the residuals. Define the residual as the distance from each point to the fitted circle:

[0072]

[0073] The objective is to minimize the following expression: Thereby obtaining the center and radius of the best fit.

[0074] S4. By converting the picture coordinates of the positioning hole into mechanical coordinates and obtaining the position information of the first needle picking hole on the needle material tray in the mechanical coordinates, the positioning of multiple needle picking holes on the needle material tray is achieved in sequence.

[0075] Specifically, the step S4 includes:

[0076] S41. Convert the picture coordinates of the positioning hole into mechanical coordinates to obtain the position information of the needle picking hole in the mechanical coordinates;

[0077] S42. Obtain the position information of the first needle picking hole on the needle material tray in the mechanical coordinates to achieve the sequential positioning of multiple needle picking holes on the needle material tray.

[0078] During the coordinate transformation process, first read and invert the affine transformation matrix to convert the points in the picture coordinate system into the mechanical coordinate system.

[0079] For each point (X, Y) to be transformed, calculate its new coordinates (x, y) in the target coordinate system using the affine transformation matrix. In the specific coordinate transformation, the calculation can be performed through the following formula:

[0080]

[0081] After obtaining the mechanical coordinates, further offset and correct the coordinates by calculating the actual angle of the workpiece to obtain the final mechanical coordinates.

[0082] According to some embodiments of the present application, before the step S1, it further includes:

[0083] Input the camera shooting parameters into the PLC control system, and control the camera to move to the shooting position through the PLC control system to perform image acquisition on the needle material tray.

[0084] The photographing parameters such as the exposure value, the coordinate X (Circle_X0, Circle_X1) of the camera photographing position, the coordinate Y (Circle_Y0, Circle_Y1) of the camera photographing position, the coordinate Z (Circle_Z) of the camera photographing position, the picture coordinates X1, Y1 of the needle suction hole, and the picture coordinates X2, Y2 of the positioning hole. This step ensures the accuracy of the basic data for subsequent calculations.

[0085] Write the input photographing parameters (Circle_X0, Circle_X1, Circle_Y0, Circle_Y1, Circle_Z) into the PLC control system through the data transmission interface. After the PLC receives these coordinates, it mobilizes the servo motor through control instructions to accurately move to the preset position. After the servo motor is positioned at the specified position, it triggers the camera shooting mechanism to shoot the images of the two positioning holes. This step aims to ensure that the obtained images are clear and accurate, laying a foundation for subsequent image processing and needle picking position recognition. Figures 2 to 3 The figure shows an example of the picture taken by the camera.

[0086] According to some embodiments of the present application, after the step S4, the following is further included:

[0087] Input the position information of the first needle picking hole of the needle stock tray in the mechanical coordinates into the PLC control system for subsequent retrieval of the position information instruction when picking needles from the needle picking hole.

[0088] During subsequent needle picking, the PLC control system will control the servo motor to accurately move to the needle picking position according to the previously stored needle suction position coordinates. At this time, the PLC control system will send corresponding instructions to the servo motor. After the servo motor completes the position adjustment, the needle picking is completed and the automatic welding of the PIN needle is performed.

[0089] As Figures 4 to 5 shown in the figure, which is a schematic structural diagram of the needle stock tray. Positioning holes 1 are respectively arranged at both ends of the needle stock tray. By identifying the picture coordinates of the positioning holes 1 and through algorithms, the mechanical coordinates of the first needle picking hole 2 are obtained to realize the sequential positioning of multiple needle picking holes 2.

[0090] A needle picking position recognition method based on image recognition provided by some embodiments of the present application. By using camera image recognition to identify the positions of the positioning holes on the needle stock tray, the mechanical coordinates of multiple needle picking holes are obtained and sequentially positioned, reducing the positioning error caused by manual operation, ensuring the accuracy of the needle picking position, improving the positioning accuracy, reducing the failure of PIN needle welding caused by the deviation of the needle picking position, reducing the dependence on manual operation through automated processing, reducing the labor cost, being conducive to reducing operation errors, and being applicable to different production environments and types of needle stock trays, having good versatility and adaptability.

[0091] According to some embodiments of the present application, as Figure 6 shown, a needle picking position recognition system based on image recognition is further provided, which applies the method described in any of the above solutions, and includes:

[0092] An image acquisition module, configured to sequentially acquire image information of multiple photos of positioning holes on a needle stock tray through a camera;

[0093] An acquisition module, configured to acquire photographing parameters and image information, and perform preprocessing and feature extraction on the image to obtain a circular contour of the positioning hole;

[0094] An image fitting module, configured to fit the circular contours of the positioning holes in multiple pictures, and calculate the picture coordinates of the best fit of the positioning holes;

[0095] A coordinate conversion module, configured to convert the picture coordinates of the positioning hole into mechanical coordinates, and obtain the position information of the first needle picking hole on the needle stock tray in the mechanical coordinates, so as to sequentially position multiple needle picking holes on the needle stock tray.

[0096] According to some embodiments of the present application, a computer-readable storage medium is further provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0097] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0098] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installation", "connection", "connection", "fixation", etc. should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can also be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific situations.

[0099] The above description only details the preferred embodiments and principles of the present application. For those of ordinary skill in the art, according to the idea provided by the present invention, there will be changes in the specific implementation manners, and these changes should also be regarded as the protection scope of the present application.

Claims

1. A needle removal position recognition method based on image recognition, characterized in that: The following steps are involved: S1, sequentially collecting image information of multiple photos of the positioning holes on the needle material disk through a camera; S2, obtaining photographing parameters and image information, and performing preprocessing and feature extraction on the image to obtain a circular contour of the positioning hole; S3, fitting the circular contours of the positioning holes in the multiple images, and calculating the image coordinates of the best fit of the positioning holes; S4. By converting the image coordinates of the positioning hole into mechanical coordinates, and obtaining the position information of the first needle extraction hole of the needle material disk in the mechanical coordinates, the multiple needle extraction holes of the needle material disk are positioned in sequence.

2. The method for identifying a needle removal position based on image recognition according to claim 1, characterized in that: Before step S1, the following steps are also included: The camera's shooting parameters are input into the PLC control system, and the camera is controlled to move to the shooting position through the PLC control system to capture images of the needle material tray.

3. The method for identifying a needle removal position based on image recognition according to claim 1, characterized in that: The step S1 comprises: The camera collects multiple images of the positioning holes on the needle material disc, and the image coordinates of the positioning holes on the needle material disc are identified through image processing.

4. The method for identifying a needle removal position based on image recognition according to claim 1, characterized in that: The step S2 comprises: S21, filtering, denoising and enhancing the image to achieve preprocessing of the image; S22, extracting features from the image, identifying the image coordinates of the positioning hole, and calculating the circular contour of the positioning hole according to the image coordinate positions.

5. The method for identifying a needle removal position based on image recognition according to claim 1, characterized in that: The step S3 comprises: S31, fitting the circular contours of the positioning holes in the multiple images, and calculating the center and radius of the positioning holes; S32, calculating the best-fitting image coordinates of the positioning hole according to the center and radius of the positioning hole.

6. The method for identifying a needle removal position based on image recognition according to claim 1, characterized in that: The step S4 comprises: S41, converting the image coordinates of the positioning hole into mechanical coordinates to obtain the position information of the pinhole in the mechanical coordinates; S42, obtaining the position information of the first needle extraction hole of the needle material tray in the mechanical coordinates, so as to realize sequential positioning of the multiple needle extraction holes of the needle material tray.

7. The method for identifying a needle removal position based on image recognition according to claim 1, characterized in that: After step S4, the following steps are also included: The position information of the first needle removal hole of the needle material tray in the mechanical coordinate is input into the PLC control system so that the position information instruction can be called for the subsequent needle removal from the needle removal hole.

8. A needle removal position recognition system based on image recognition, characterized in that: The method according to any one of claims 1 to 7 is applied, comprising: An image acquisition module, used to sequentially acquire image information of multiple photos of the positioning holes on the needle material tray through a camera; An acquisition module is used to acquire photographing parameters and image information, and to preprocess and extract features of the image to obtain a circular contour of the positioning hole; An image fitting module is used to fit the circular contours of the positioning holes in multiple pictures and calculate the image coordinates of the best fit of the positioning holes; The coordinate conversion module is used to convert the image coordinates of the positioning hole into mechanical coordinates and obtain the position information of the first needle extraction hole of the needle material tray in the mechanical coordinates, so as to realize the positioning of multiple needle extraction holes of the needle material tray in sequence.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images