A method and system for identifying the in and out of the warehouse of an SMT patch device tray, and an intelligent terminal

Abstract

This application relates to the field of material tray entry and exit technology, and in particular to a method, system, and intelligent terminal for identifying the entry and exit of SMT component trays. The method includes: reading tray information, including tray size and standard tray spacing; acquiring a tray image, the image content of which includes a target area and a background area; tray image processing, separating the target area and background area of ​​the tray image, the target area reflecting the edge contour of the tray; extracting image feature information from the target area to obtain the maximum spacing between the tray edge contours; detection and analysis, performing deformation analysis based on the tray information and image feature information to determine the tray status information, including normal and deformed states; matching warehouse location information based on the tray status information and outputting tray storage information. This application effectively reduces the occurrence of tray misalignment or dropping during storage due to tray deformation.

Description

Technical Field

[0001] This application relates to the field of material tray entry and exit technology, and in particular to a method, system and intelligent terminal for identifying the entry and exit of SMT component trays. Background Technology

[0002] Electronic components are the fundamental building blocks of electronic devices. Surface Mount Technology (SMT) is a circuit assembly technology that mounts leadless or short-lead surface mount components onto the surface of a printed circuit board or other substrate, and then assembles them using methods such as reflow soldering or dip soldering. It is currently one of the most popular technologies and processes in the electronics assembly industry. With the increasing adoption of this technology in electronic products, the SMT industry needs to utilize intelligent automated storage and retrieval systems (AS / RS) for intelligent and automated material storage management, thereby meeting the industry's demands for specialization, centralization, and automation.

[0003] In SMT (Surface Mount Technology), electronic components are stored on trays via tape. These trays serve as storage units. Since there are many types of electronic components, and different types vary in size, SMT trays come in various sizes, including 7-inch and 13-inch, with each diameter further subdivided into thicknesses such as 8mm, 12mm, 16mm, and 24mm. To ensure efficient tray storage and minimize space usage, the clamps are designed with storage compartments tailored to the tray dimensions. The height of each compartment is only slightly greater than the tray's thickness, with gaps typically between 0.5 and 1.0 cm. This design ensures smooth tray storage without excessive space waste.

[0004] Regarding the above technical solution, the inventors believe that since material trays will deform to some extent after long-term use, and during the intelligent placement process, the existing system can only store material trays according to the original specifications of the material trays corresponding to the QR code or barcode information. When the material trays are deformed significantly, misalignment and jamming are likely to occur during storage, resulting in the material trays being unable to be stored normally or falling off due to misplacement. Summary of the Invention

[0005] To reduce the problem of material trays being unable to be placed in their original locations due to deformation, this application provides a method, system, and intelligent terminal for identifying the entry and exit of SMT component trays.

[0006] In a first aspect, this application provides a method for identifying the entry and exit of SMT component reels, employing the following technical solution: A method for identifying the entry and exit of SMT component reels, comprising:

[0007] Read the tray information, which includes the tray size and standard tray spacing;

[0008] Acquire a tray image, wherein the imaging content of the tray image includes a target area and a background area;

[0009] The tray image processing separates the target area and the background area of ​​the tray image, wherein the target area is used to reflect the edge contour of the tray;

[0010] Extract image feature information from the target area to obtain the maximum spacing between the edge contours of the material tray;

[0011] The detection and analysis process involves deformation analysis based on tray information and image feature information to determine tray status information, which includes normal and deformed states.

[0012] Based on the status information of the material tray, the storage location information is matched, and the storage information of the material tray is output.

[0013] By adopting the above technical solution, the outline of the material tray is obtained by acquiring and processing the image of the material tray. The maximum distance between the edge outlines of the material tray is obtained by analyzing the outline of the material tray, thereby determining whether the material tray has been deformed. The storage location information of the material tray is selected according to the status information of the material tray, so that the height of the storage location is selected according to the actual situation of the material tray, thereby reducing the occurrence of the material tray being unable to be stored in the corresponding storage location or falling down due to error when the material tray is deformed.

[0014] Preferably, the specific method of the tray image processing step includes:

[0015] A comparison image is obtained based on the material tray information, wherein the comparison image is a cropped background image in the state without a material tray;

[0016] The material tray image is cropped based on the size of the comparison image to obtain a preprocessed image, wherein the preprocessed image and the comparison image are positioned in the same position in the material tray image;

[0017] The preprocessed image is converted into a grayscale image, and the gradient field of the grayscale image is calculated. The gradient field information of the comparison image is obtained, and the target region and background region of the grayscale image are divided according to the gradient field information.

[0018] By adopting the above technical solution, the material tray image is processed and cropped according to the size and position of the comparison image, which can effectively reduce the amount of material tray image data to be processed and speed up the material tray image processing efficiency. By comparing the gradient field values ​​of the preprocessed image and the comparison image, the target area and the background area can be effectively and accurately separated, thereby reducing the recognition error.

[0019] Preferably, the specific method for extracting image feature information in the target area and obtaining the maximum spacing between the edge contours of the tray includes:

[0020] The target region is input into the feature extraction model to obtain the overall contour of the target region, and mapped to the preset planar coordinate system in the feature extraction model to obtain the maximum distance between the edge contours of the tray; the image feature information reflects the maximum distance between the edge contours of the tray.

[0021] By adopting the above technical solution, the feature extraction model extracts feature points of the target area to determine the outline of the tray. Then, the tray outline is mapped to a plane coordinate system, and coordinate values ​​are assigned to each point of the tray outline through the plane coordinate system. Thus, the maximum spacing between the edge outlines of the tray can be quickly obtained through the coordinate values.

[0022] Preferably, the y-axis of the planar coordinate system passes through the center of the material tray, and the image feature information includes left spacing and right spacing;

[0023] The specific method for obtaining the actual maximum spacing between the edge contours of the material tray includes:

[0024] Obtain the point with the largest y-coordinate and the point with the smallest y-coordinate on the negative half-axis of the x-coordinate axis, and determine the left spacing. The left spacing reflects the maximum spacing between the edge contours of the material tray on one side of the negative half-axis of the x-coordinate axis.

[0025] Obtain the point with the largest y-coordinate and the point with the smallest y-coordinate on the positive half-axis of the x-coordinate axis, and determine the right spacing. The right spacing reflects the maximum spacing between the edge contours of the material tray located on one side of the positive half-axis of the x-coordinate axis.

[0026] By adopting the above technical solution, the center of the material tray is set on the y-axis so that the left and right sides of the material tray are located on the positive and negative x-axis respectively, so as to facilitate the acquisition and comparison of deformation data of the left and right sides of the material tray.

[0027] Preferably, before the step of matching storage location information based on the material tray status information and outputting material tray storage information, the method further includes determining whether to output rotation control information based on deformation analysis. The rotation control information is used to control the horizontal rotation of the material tray to achieve orientation change.

[0028] The deformation analysis includes deformation of the left side of the tray, deformation of the right side of the tray, overall deformation of the tray, and no deformation of the tray.

[0029] When the deformation analysis indicates deformation on the left or right side of the tray, rotation control information is output.

[0030] By adopting the above technical solution, based on the state of the material tray when it is stored, one side of the material tray will contact the clamp first. Once this side of the material tray is successfully placed into the storage location, even if the other side undergoes significant deformation, it will not affect the storage of the clamp. By setting up rotation control information based on deformation analysis, when one side of the clamp deforms, the system determines whether rotation control information needs to be output based on a comparison between the orientation of the clamp deformation and its orientation when stored. Through rotation control information, the undeformed side of the clamp is switched to the side that first contacts the clamp when it is inserted into the storage location. This better ensures that clamps at other storage location heights are not over-occupied, while preventing excessive waste of material trays at the lowest storage location height.

[0031] Preferably, the specific method of the detection and analysis step includes:

[0032] Standard spacing thresholds are obtained based on tray information;

[0033] Based on the standard spacing threshold, determine whether the maximum spacing is greater than the standard spacing threshold and output the determination result;

[0034] Based on the judgment result, the status information of the material tray is determined.

[0035] By adopting the above technical solution, the conditions for judging the deformation of the material tray are made more extensive by setting a standard spacing threshold, and the requirements for the system's recognition accuracy are also reduced. The state of the material tray is determined by comparing the maximum spacing and the standard spacing threshold, so as to quickly determine whether the material tray has deformed beyond the standard spacing threshold.

[0036] Secondly, this application provides an SMT component tray inbound / outbound system, which adopts the following technical solution: An SMT component tray inbound / outbound system, comprising:

[0037] The information reading module is used to read tray information, including tray size and standard tray spacing.

[0038] An image acquisition module is used to acquire images of the material tray; the imaging content of the material tray image includes a target area and a background area;

[0039] The image processing module is used for tray image processing, separating the target area and background area of ​​the tray image; the target area is used to reflect the edge contour of the tray.

[0040] The feature extraction module is used to extract image feature information in the target area and obtain the maximum spacing between the edge contours of the tray; the analysis module is used for detection and analysis, performing deformation analysis based on tray information and image feature information to determine tray status information; the tray status information includes normal state and deformed state;

[0041] The matching module is used to match storage location information based on the status information of the material tray and output the storage information.

[0042] Preferably, it further includes:

[0043] A clamping module is used to clamp the tray so that the information reading module can read the tray information and the image acquisition module can acquire the tray image.

[0044] A rotating module is used to drive the gripping module to rotate;

[0045] The storage module is used to transfer and store the material tray according to the storage information output by the matching module.

[0046] Thirdly, this application provides a smart terminal, which adopts the following technical solution:

[0047] A smart terminal includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in the first aspect for identifying warehouse entry and exit.

[0048] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0049] The computer-readable storage medium stores a computer program that can be loaded by a processor and executed as described in the first aspect for identifying the entry and exit of a warehouse.

[0050] In summary, the present invention has at least one of the following beneficial technical effects:

[0051] 1. By acquiring images of the material trays and performing recognition and analysis on the acquired images, the status of the material trays can be determined. This facilitates matching the storage location height of the material trays according to their status, thereby reducing the occurrence of situations where the material trays cannot be stored in the corresponding storage location or fall out due to errors when the material trays are deformed.

[0052] 2. By setting the center of the material tray on the y-axis of the planar coordinate system, the material tray is divided into left and right sides by the planar coordinate system. The maximum distance between the edge contours of the left and right sides of the material tray is identified and analyzed to determine whether the left and right sides of the material tray have deformed. The state of the material tray is determined based on the deformation of the left and right sides of the material tray. Thus, when only one side of the material tray is deformed, the orientation of the material tray can be adjusted so that the material tray can still be matched with the storage location according to the standard spacing of the material tray, thereby reducing the occupation of storage space for material trays of other height specifications. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall process of the SMT component tray entry and exit identification method of this application;

[0054] Figure 2This is a partial flowchart of a method for identifying the entry and exit of SMT component trays in this application;

[0055] Figure 3 This is a pre-processed image after cropping, used in a method for identifying the entry and exit of SMT component trays in this application.

[0056] Figure 4 This refers to the state of the tray outline mapped in a planar coordinate system in the SMT component tray entry and exit identification method of this application, where the image feature information consists of two pixels;

[0057] Figure 5 This is the state of the tray outline mapped in the planar coordinate system in the SMT component tray entry and exit identification method of this application, and the image feature information is four pixels;

[0058] Figure 6 This is a partial flowchart of a method for identifying the entry and exit of SMT component trays in this application;

[0059] Figure 7 This is a schematic diagram of the SMT component tray inbound and outbound system of this application;

[0060] Figure 8 This is a schematic diagram of the overall structure of the SMT component tray loading and unloading system of this application from one perspective;

[0061] Figure 9 This is a partial structural diagram of the SMT component tray loading and unloading system of this application, mainly showing the structure of the lifting module;

[0062] Figure 10 This is a schematic diagram of the overall structure of the SMT component tray loading and unloading system of this application from another perspective.

[0063] In the diagram, 11 is the information reading module; 12 is the image acquisition module; 13 is the image processing module; 14 is the feature extraction module; 15 is the analysis module; 16 is the matching module; 21 is the lifting module; 211 is the lifting screw; 212 is the lifting frame; 22 is the clamping module; 23 is the rotation module; 24 is the horizontal and vertical movement module; 241 is the synchronous belt; 242 is the screw slide table; 25 is the clamping module; and 26 is the bad disc rack. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the related objects before and after it have an "or" relationship.

[0066] Furthermore, the labels for each step in this embodiment are for illustrative purposes only and do not represent a limitation on the execution order of each step. In practical applications, the execution order of each step can be adjusted or performed simultaneously as needed, and such adjustments or substitutions are all within the protection scope of this invention.

[0067] The following is in conjunction with the instruction manual appendix. Figures 1-10 The embodiments of this application will be described in further detail.

[0068] Example 1:

[0069] This application provides a method for identifying the entry and exit of SMT (Surface Mount Technology) component reels. (Refer to...) Figure 1 The main process of the identification method is described below.

[0070] S100, Read tray information.

[0071] Specifically, the tray information refers to the existing information data of the tray obtained by the information reading module through scanning the tray's label. This includes the tray size, standard tray spacing, and the type, model, and quantity of SMD surface mount devices. The tray size is the diameter of the tray being read, such as a 7-inch tray or a 13-inch tray. The standard tray spacing is the standard height (tray thickness) of the tray being read, i.e., the height of the tray in its undeformed state, such as 8mm, 12mm, 16mm, or 24mm trays. The information reading module can be a barcode scanner or an industrial CCD. Since the tray is disc-shaped, depending on its state when entering or leaving the warehouse, when the tray is gripped by the clamping module, the radial direction of the tray is parallel to the horizontal plane, and the axial direction is perpendicular to the horizontal plane. The tray label is usually placed on the end face of the tray. Therefore, the information reading module should be located above or below the tray, with its scanning direction facing the end face of the tray. In this embodiment, the information reading module is positioned above the tray.

[0072] When a material tray needs to be put into storage, the clamping module clamps the tray and moves it to the designated position. The information reading module scans the tray label to read the tray information.

[0073] S200, Obtain the image of the material tray.

[0074] Specifically, the tray image refers to an image containing the tray captured by the image acquisition module. In this embodiment, the position of the tray during image acquisition is consistent with the position during information reading, meaning the tray is both read and captured at the same location. Since the tray image is captured based on the tray itself, the image content should include the entire tray; therefore, in this embodiment, the image content includes a target area and a background area, where the target area is the area containing the entire tray, and the background area is the area outside the tray within the field of view of the image acquisition module. The image acquisition module is preferably an industrial CCD.

[0075] In this embodiment of the application, during the process of acquiring the image of the material tray, the position of the material tray remains unchanged, and the shooting direction of the image acquisition module is towards the material tray and parallel to the radial direction of the material tray; when the material tray is driven to the designated position by the clamping module, the image acquisition module takes an image of the area with the material tray.

[0076] S300, tray image processing, separates the target area and background area of ​​the tray image.

[0077] Specifically, the target region is used to reflect the edge contour of the tray. That is, the tray image includes the target region containing the tray and the background region excluding the tray during the imaging process. The target region is the necessary region for obtaining the tray spacing, while the background region is an irrelevant region for obtaining the tray spacing. Therefore, by separating the target region and the background region, the amount of data in image processing can be effectively reduced and the efficiency of tray image processing can be improved.

[0078] Reference Figure 2 The specific methods for processing tray images include:

[0079] S301. Obtain comparison images based on tray information.

[0080] Specifically, the comparison image refers to a cropped background image in the trayless state. In this embodiment, the background image is a pre-stored image, specifically obtained by capturing a panoramic image using an image acquisition module in the trayless state. This panoramic image only includes the background area. The panoramic image is then cropped according to the tray information to obtain comparison images corresponding one-to-one with the tray information. The comparison image represents the background area during the tray imaging process. In this embodiment, the tray information used to crop the panoramic image includes the tray size and standard tray spacing. Different tray sizes have different diameters, resulting in different comparison image sizes. For example, the comparison image size of a 7-inch tray with the same standard tray spacing is smaller than that of a 13-inch tray; the comparison image size of an 8mm tray with the same tray size is smaller than that of a 24mm tray. The size of the comparison image is set to be larger than the tray diameter and standard tray spacing in the tray information; that is, the length of the comparison image is greater than the tray diameter, and the width of the comparison image is greater than the standard tray spacing, thereby ensuring that the tray outline is fully reflected within the size range of the comparison image.

[0081] The comparison images corresponding to different tray sizes are pre-stored in the system. When the tray information is read by the information reading module, the system matches the comparison image of the tray size in the tray information. For example, if the tray size in the tray information is 7 inches, the system matches the comparison image corresponding to the 7-inch tray as the comparison image in this recognition.

[0082] S302. Based on the size of the comparison image, cut the material tray image to obtain a preprocessed image.

[0083] Specifically, the preprocessed image refers to the cropped tray image. In this embodiment, since the shooting angle of the image acquisition module remains unchanged and the position of the tray gripping module moving to the shooting point remains unchanged, the size of the image captured by the image acquisition module remains unchanged. The tray image is cropped according to the cropping position size information of the comparison image to obtain that the preprocessed image and the comparison image are in the same position in the tray image.

[0084] S303. Convert the preprocessed image into a grayscale image and calculate the gradient field of the grayscale image; obtain the gradient field information of the comparison image, and complete the division of the target region and background region of the grayscale image based on the gradient field information.

[0085] Specifically, calculating the gradient field of the grayscale image and obtaining the gradient field information of the comparison image are done sequentially. The gradient field information of the comparison image is pre-stored information, meaning that the gradient field calculation has already been completed during the comparison image creation process. In this embodiment, to ensure a more accurate division between the target region and the background region, the gradient field calculation method of the comparison image is consistent with the gradient field calculation method of the restored image.

[0086] The gradient field calculation method for preprocessed images specifically includes the following steps: First, the preprocessed image is converted into a grayscale image. Then, the grayscale image is divided into blocks. Gradient field calculation is performed on multiple blocks simultaneously to accelerate the system's computation speed. The area of ​​the blocks after the grayscale image is divided is measured in pixels. In this embodiment, the viewing angle of the image acquisition module of the normal tray is rectangular, that is, the cross-section of the tray along its axis is rectangular. Therefore, the blocks are divided into rectangular blocks. To ensure detection accuracy, the area of ​​the rectangular blocks is less than 10*10 (unit: pixels*pixels), and the area of ​​the grayscale image blocks is consistent with the size of the blocks in the comparison image.

[0087] The formula for calculating the gradient field is as follows:

[0088] Where T(i,j) is the gradient field value of the block, (i,j) is the coordinate of the center point of the block; (u,v) is the coordinate of the pixel point traversed in the block; and w is the width of the block. These are the partial derivatives in the x and y directions, respectively.

[0089] The gradient field information of the comparison image includes the gradient field value T'(i,j) corresponding to each block of the grayscale image. Because the material tray occludes part of the image information in the comparison image during imaging, the gradient field value in the occluded state differs from the gradient field value in the unoccluded state. Therefore, after the gradient field value of each block of the grayscale image is calculated, the gradient field value T(i,j) of each block of the grayscale image is compared with the gradient field value T'(i,j) of the corresponding block in the comparison image. If T(i,j) = T'(i,j), it indicates that the block is a background region; if T(i,j) ≠ T'(i,j), it indicates that the block is a target region. Each block is compared one by one to distinguish the target region and background region of the grayscale image.

[0090] S400: Extract image feature information from the target area and obtain the maximum spacing between the edge contours of the material tray.

[0091] Specifically, refer to Figure 2 and Figure 3Image feature information is used to reflect the maximum spacing between the edge contours of the tray. The maximum spacing obtained through image feature information is the upper limit of the actual spacing after the tray deformation. The maximum spacing between the tray contours can intuitively determine whether the tray has outward warping or inward concavity. After the target region is divided, the target region is input into the trained feature extraction model to obtain the overall contour of the target region and map it to the preset planar coordinate system in the feature extraction model to obtain the maximum spacing between the edge contours of the tray. In this embodiment, the feature extraction model is a deep learning model, and its network structure is preferably a fully convolutional network structure; the planar coordinate system is in pixels, so the unit of the maximum spacing between the edge contours of the tray reflected by the image feature information is pixels. In this embodiment, the image feature information includes several pixels distributed at the endpoints on the upper and lower sides of the tray. The endpoints on the upper and lower sides of the tray are used to reflect the position of the maximum outward warping deformation of the tray.

[0092] In one embodiment, reference is made to Figure 4 The image feature information includes two pixels: one pixel is the pixel with the largest y-coordinate, which represents point A at the endpoint of the largest deformation on the upper side of the material tray; the other pixel is the pixel with the smallest y-coordinate, which represents the pixel at the endpoint of the largest deformation on the lower side of the material tray.

[0093] Construct an upper line and a lower line parallel to the x-axis. The upper line passes through point A with the largest y-coordinate, and the lower line passes through point B with the smallest y-coordinate. Obtain the vertical distance between the upper and lower lines as the maximum distance between the edge contours of the material tray (unit: pixels). That is, the difference in y-coordinate between two pixels is the maximum distance between the edge contours of the material tray.

[0094] In another embodiment, reference Figure 5 Since the position of the tray and the image acquisition module remain unchanged during tray image acquisition, the position of the tray outline mapped onto the planar coordinate system is also fixed. When the feature extraction model presets the planar coordinates, the y-coordinate of the planar coordinate system is set to pass through the center of the tray. That is, the left side of the y-coordinate corresponds to the left side of the tray in the tray image, and the right side of the y-coordinate corresponds to the right side of the tray in the tray image. Because a large deformation on the side of the tray that first engages with the clamp during storage will affect the tray's storage in the storage location, while deformation on the other side will not affect storage, the tray is divided into left and right sides using the y-coordinate, and the maximum distance between the edge contours of the left and right sides of the tray is obtained respectively.

[0095] The image feature information includes four pixels: one pixel is the pixel with the largest y-coordinate on the negative half of the x-axis, which represents point A at the endpoint of the largest deformation on the upper left side of the material tray; another pixel is the pixel with the smallest y-coordinate on the negative half of the x-axis, which represents point B at the endpoint of the largest deformation on the lower left side of the material tray; another pixel is the pixel with the largest y-coordinate on the positive half of the x-axis, which represents point C at the endpoint of the largest deformation on the upper right side of the material tray; and the third pixel is the pixel with the smallest y-coordinate on the positive half of the x-axis, which represents point D at the endpoint of the largest deformation on the lower left side of the material tray.

[0096] Construct four lines parallel to the x-axis: upper left, lower left, upper right, and lower right. The upper left line passes through point A, where the y-coordinate is largest on the negative x-axis, and the lower left line passes through point B, where the y-coordinate is smallest on the negative x-axis. The vertical distance between the upper left and lower left lines is taken as the maximum distance (in pixels) between the left edges of the tray. That is, the difference in y-coordinates between points A and B is the maximum distance between the left edges of the tray. Similarly, the upper right line passes through point C, where the y-coordinate is largest on the positive x-axis, and the lower right line passes through point D, where the y-coordinate is smallest on the positive x-axis. The vertical distance between the upper right and lower right lines is taken as the maximum distance (in pixels) between the right edges of the tray. That is, the difference in y-coordinates between points C and D is the maximum distance between the left edges of the tray.

[0097] To better eliminate the influence of the background on the feature points of the tray outline, in this embodiment, when setting the image acquisition module, the image acquisition module is directly facing the tray so that the shooting direction of the image acquisition module is directly facing the tray, and the projection of the image acquisition module in its shooting direction is located in the middle of the tray, so that the origin of the plane coordinate system coincides with the center of the tray. That is, when the outline of the standard tray without deformation is mapped into the plane coordinate system, the vertical distance between the upper and lower sides of the tray outline and the x-coordinate axis is equal.

[0098] S500 detection and analysis: Based on the tray information and image feature information, deformation analysis is performed to determine the tray status information.

[0099] Specifically, image feature information can reflect the maximum spacing between the edge contours of the tray, while tray information reflects the standard spacing of the tray in a non-deformed state. Therefore, based on tray information and image information, the difference between the maximum spacing and the standard spacing of the tray can be calculated, thereby comparing and determining whether the tray has deformed.

[0100] The tray status information includes normal status and deformed status. Normal tray deformation includes two situations: concave and convex. During tray storage, concave deformation does not affect the storage of the tray. Therefore, in this embodiment, deformed tray refers only to convex deformation. Meanwhile, since the spacing between storage locations is slightly larger than the standard spacing of the actual stored trays, a standard spacing threshold is set. When the maximum tray spacing is greater than the standard spacing threshold, the tray status information is in a deformed state; when the maximum tray spacing is less than the standard spacing threshold, the tray status information is in a normal state. That is, when the maximum tray spacing is greater than or less than the standard tray spacing, the tray has actually deformed, but since the maximum tray spacing is not greater than the standard spacing threshold, the tray is considered to be in a normal state in the system.

[0101] Reference Figure 6 The specific method for determining the status of the material tray includes the following steps:

[0102] S501. Obtain the standard spacing threshold based on the tray information.

[0103] Specifically, the tray information includes the standard tray spacing. The system has preset standard spacing thresholds that correspond one-to-one with the standard tray spacing. For ease of subsequent analysis, the unit of the standard spacing threshold is the same as the unit of the maximum spacing, both being pixels. The system standard spacing threshold unit for each inbound / outbound system is set according to the actual height of its storage compartment, i.e., the standard spacing threshold is determined based on the difference between the storage compartment height and the standard tray spacing. For example, for a storage compartment holding 8mm trays, if the storage compartment height is 13mm, the standard spacing threshold is set to 1000 pixels; if the storage compartment height is 18mm, the standard spacing threshold is set to 1500 pixels.

[0104] S502. Based on the standard spacing threshold, determine whether the maximum spacing is greater than the standard spacing threshold and output the determination result;

[0105] S503. Based on the judgment result, determine the status information of the material tray.

[0106] Specifically, the image feature information not only reflects the maximum spacing between the edges of the tray, but also the number of pixels at the end points of the tray. The image feature information includes two and four pixels respectively.

[0107] If the image feature information consists of two pixels, then the deformation analysis analyzes whether the entire tray has undergone deformation. The overall deformation of the tray is defined as a maximum spacing greater than the standard spacing threshold. In this embodiment, the tray is considered deformed when the entire tray is deformed. That is, it determines whether the maximum spacing is greater than the standard spacing threshold. If the determination result is yes, the tray is in a deformed state; if the determination result is no, the tray is in a normal state.

[0108] If the image feature information consists of four pixels, the deformation analysis includes deformation of the left side of the tray, deformation of the right side of the tray, overall deformation of the tray, and no deformation of the tray. Deformation of the left side of the tray means the left spacing of the tray is greater than the standard spacing threshold, and the right spacing is less than the standard spacing threshold. Deformation of the right side of the tray means the left spacing is less than the standard spacing threshold, and the right spacing is greater than the standard spacing threshold. Overall deformation of the tray means both the left and right spacing are greater than the standard spacing threshold. No deformation of the tray means both the left and right spacing are less than the standard spacing threshold. In this embodiment, when the tray is deformed overall, the tray state is considered deformed, meaning that deformation of the left side and right side of the tray are also considered normal states. Specifically, it checks whether the left spacing is greater than the standard spacing threshold and whether the right spacing is greater than the standard spacing threshold. If both the left and right spacings are yes, the tray is in a normal state. If the left spacing is yes and the right spacing is no, the tray is in a normal state. If the left spacing is no and the right spacing is yes, the tray is in a normal state. If both the left and right spacings are no, the tray is in a deformed state.

[0109] The premise that deformation on both the left and right sides of the pallet is considered normal is based on deformation analysis to determine whether to output rotation control information. This rotation control information is used to control the horizontal rotation of the pallet to achieve orientation change. During storage, if the side of the pallet that first contacts the clamp can be successfully placed into the storage location, the other side of the pallet can also be successfully placed into the storage location even if it deforms beyond the standard spacing threshold. Therefore, if the side of the pallet that first contacts the clamp deforms beyond the standard spacing threshold, while the other side does not, rotation control information can be output to swap the two sides of the pallet. This allows the pallet to be placed in the storage location corresponding to the standard pallet spacing even if one side deforms while the other side does not. Whether the tray deforms on the left or right side is determined by the orientation of the tray's position in the planar coordinate system when it is gripped by the storage module. For example, if the tray is gripped by the storage module on the left side of the planar coordinate system, rotation control information is output when the tray deforms on the right side; if the tray is gripped by the storage module on the right side of the planar coordinate system, rotation control information is output when the tray deforms on the left side.

[0110] S600 matches storage location information based on pallet status information and outputs pallet storage information.

[0111] Specifically, the required storage location height for the pallet is first determined based on its status information. Then, it is matched with available storage locations at the corresponding height in the current clamp to determine the appropriate storage location for the pallet. The storage location information and pallet information are then linked to output the pallet's storage information. When a pallet is in a deformed state, the storage location is matched with a location one level higher than the pallet's height. That is, when a pallet with a standard 8mm spacing deforms, the storage location in the clamp that originally matched an 8mm pallet cannot store it correctly. In this case, an available storage location for a 12mm pallet must be found. The pallet storage information includes the specific location of the storage location, the path information for the storage module to move to that location, and the pallet information itself.

[0112] Example 2:

[0113] In one embodiment, an SMT tray in / out warehousing system is provided, corresponding one-to-one with the SMT tray in / out warehousing identification method of the above embodiment. This system is applied to SMT tray in / out warehousing identification, enabling efficient and reliable placement of trays into their corresponding storage locations upon entry into the warehouse. The system includes an information reading module 11, an image acquisition module 12, an image processing module 13, a feature extraction module 14, an analysis module 15, and a matching module 16. (Refer to...) Figure 7 The detailed descriptions of each functional module are as follows:

[0114] Information reading module 11 is used to read tray information; tray information includes tray size and standard tray spacing;

[0115] Image acquisition module 12 is used to acquire images of the material tray; the imaging content of the material tray image includes the target area and the background area;

[0116] Image processing module 13 is used for tray image processing, separating the target area and background area of ​​the tray image; the target area is used to reflect the edge contour of the tray.

[0117] The feature extraction module 14 is used to extract image feature information in the target area and obtain the maximum spacing between the edge contours of the tray;

[0118] Analysis module 15 is used for detection and analysis. Based on the tray information and image feature information, it performs deformation analysis to determine the tray status information. The tray status information includes normal status and deformed status.

[0119] The matching module 16 is used to match the storage location information based on the material tray status information and output the storage information.

[0120] Specifically, when a material tray needs to be put into storage, the information reading module 11 and the image acquisition module 12 execute their preset function programs in no particular order. When the information reading module 11 reads the material tray information and the image acquisition module 12 acquires the corresponding material tray image, the image processing module 13 receives the material tray information and the material tray image, and first obtains a comparison image corresponding to the material tray based on the material tray information; then, it crops the material tray image according to the size of the comparison image to obtain a preprocessed image with the same position and size as the comparison image in the original panoramic image; then, it converts the preprocessed image into a grayscale image, then divides the grayscale image into blocks and calculates the gradient field value of each grayscale image block, and separates the target area and the background area based on the gradient field value. After the target area of ​​the tray image is separated, the feature extraction module 14 receives the target area image and performs feature point extraction on the processed target area image to obtain the tray outline image. Then, the tray outline is mapped to a planar coordinate system. Coordinates are assigned to each pixel of the tray outline based on its position in the planar coordinate system. The point with the largest and smallest y-coordinates are obtained from the coordinates of each pixel in the outline. Parallel lines parallel to the x-axis are constructed based on the pixels, and the maximum spacing between tray outlines is obtained through these parallel lines. The analysis module 15 obtains a standard spacing threshold based on the tray information and compares the maximum spacing with the standard spacing threshold based on the image feature information to obtain deformation analysis. The tray state is determined based on the deformation analysis. The matching module 16 determines the actual placement height of the tray based on the tray state and tray information. Then, based on the placement height, it searches for and matches the information of the empty trays in the current clamp, combining the placement information and tray information to generate tray storage information.

[0121] Reference Figure 7 and Figure 8 The material tray completes the loading and unloading operations. The system also includes a frame and a lifting module 21, a clamping module 22, a rotating module 23, and a horizontal and vertical movement module 24 mounted on the frame. The information reading module 11 and the image acquisition module 12 are also mounted on the frame.

[0122] Lifting module 21 is used to drive the material tray to move up and down;

[0123] The clamping module 22 is connected to the lifting module 21 and is used to clamp the tray so that the information reading module 11 can read the tray information and the image acquisition module 12 can acquire the tray image.

[0124] Rotating module 23 is used to drive the gripping module 22 to rotate;

[0125] The horizontal and vertical movement module 24 is used to drive the clamping module 22 to move horizontally and / or vertically, so as to move the material tray.

[0126] The storage module 25 transfers and stores the material tray according to the storage information output by the matching module 16.

[0127] Specifically, refer to Figure 8 and Figure 9 When material trays are put into or taken out of the warehouse, they are stacked on the material support. Since material trays are put into or taken out one by one, the lifting module 21 drives the material trays to rise and fall, so as to ensure that the clamping module 22 can clamp a material tray from the material support for putting into the warehouse each time, or can clamp a material tray and place it on the material support for taking out each time. When the material tray is taken out, the lifting module 21 starts to control the placement height of the material tray. In this embodiment of the application, the lifting module 21 includes a lifting drive motor (not shown in the figure) and a lifting screw 211 disposed at the output end of the lifting drive motor. The lifting screw 211 is threadedly connected to a lifting frame 212, which is used to abut against and support the material tray. The lifting frame 212 is slidably connected to the frame, so that when the lifting drive motor drives the lifting screw 211 to rotate, the lifting frame 212 moves along the axial direction of the lifting screw 211 to drive the material tray to move up and down.

[0128] When the material tray needs to be stored, the gripping module 22 engages with the locking hole in the center of the tray, causing the tray to detach from the material support, thereby transferring the tray from the support. In this embodiment, the gripping module 22 is preferably a gripper cylinder.

[0129] Reference Figure 9 and Figure 10 The horizontal and vertical movement module 24 is used to drive the material tray clamping module 22 to move. When the material tray needs to be put into storage, the horizontal and vertical movement module 24 drives the clamping module 22 to first move horizontally to directly above the material tray, and then drives the clamping module 22 to move vertically downward so that the clamping module 22 is inserted into the material tray. Then, it drives the clamping module 22 to move vertically upward so that the material tray is at a specified height. Then, the horizontal and vertical movement module 24 drives the clamping module 22 to move horizontally to directly below the information reading module 11. At this time, the material tray is at a specified position, that is, when the outline of the material tray is mapped into the plane coordinate system, the center of the material tray coincides with the origin of the plane coordinate system. In this embodiment, the horizontal movement module includes a synchronous belt 241 and a lead screw slide 242. The synchronous belt 241 is used to drive the clamping module 22 to move horizontally, and the lead screw slide 242 is used to drive the clamping module 22 to move vertically. At the same time, the lead screw slide 242 is connected to the synchronous belt 241 and is driven by the synchronous belt 241 to move horizontally.

[0130] After the material tray is read and its status is determined, the matching module 16 outputs the material tray storage information. After receiving the material tray storage information, the storage module 25 connects with the clamping module 22 to clamp the material tray, so that the material tray rotates from the clamping module 22 to the storage module 25. Then, the storage module 25 moves the material tray to the corresponding storage location for storage according to the material tray storage information.

[0131] When the image feature information consists of two pixels, the system does not need to apply the rotation module 23; when the image feature information consists of four pixels, the system needs to apply the rotation module 23. This embodiment is described in the case of the application requiring the application of the rotation module 23. The rotation module 23 is disposed between the gripping module 22 and the horizontal and vertical movement module 24. In this embodiment, the rotation module 23 is preferably a rotary cylinder, but other structures capable of rotation, such as an encoder motor, can also be used. The cylinder body of the rotation module 23 is fixedly connected to the lead screw slide 242 and moves vertically up and down under the drive of the lead screw slide 242. The rotating part of the rotation module 23 is fixedly connected to the gripping module 22 to drive the gripping module 22 to rotate.

[0132] After the left and right spacings are obtained by the feature extraction module 14, the analysis module 15 performs deformation analysis on the left and right spacings against the standard spacing thresholds. If the left spacing is greater than the standard spacing threshold and the right spacing is less than the standard spacing threshold, the left side of the tray is deformed; if the left spacing is less than the standard spacing threshold and the right spacing is greater than the standard spacing threshold, the right side of the tray is deformed; if both the left and right spacings are greater than the standard spacing threshold, the entire tray is deformed; if both the left and right spacings are less than the standard spacing threshold, the tray is not deformed. The tray state and whether the rotation module 23 needs to rotate are determined based on the deformation analysis and the position between the storage module 25 and the tray. For example, if the left side of the tray mapped in the planar coordinate system is gripped by the storage module 25, rotation control information is output when the right side of the tray deforms; if the right side of the tray mapped in the planar coordinate system is gripped by the storage module 25, rotation control information is output when the left side of the tray deforms. The rotation module 23 receives and responds to the rotation control information to drive the gripping module 22 to rotate.

[0133] The rack is also equipped with a bad tray rack 26. When the information of the tray cannot be read and recognized, the system will control the storage module 25 to pick up the tray and store it on the bad tray rack 26 for subsequent manual processing.

[0134] Example 3:

[0135] In one embodiment, a smart terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. The memory stores training data, algorithm formulas, and filtering mechanisms from a training model. The processor provides computational and control capabilities, and when executing the computer program, it performs the following steps:

[0136] S100, Read tray information.

[0137] S200, Obtain the image of the material tray.

[0138] S300, image processing of the material tray, separates the target area and background area of ​​the material tray image.

[0139] The specific method in step S300 includes:

[0140] S301. Obtain comparison images based on tray information.

[0141] S302. Based on the size of the comparison image, cut the material tray image to obtain a preprocessed image.

[0142] S303. Convert the preprocessed image into a grayscale image and calculate the gradient field of the grayscale image; obtain the gradient field information of the comparison image, and complete the division of the target region and background region of the grayscale image based on the gradient field information.

[0143] S400: Extract image feature information from the target area and obtain the maximum spacing between the edge contours of the material tray.

[0144] S500 detection and analysis: Based on the tray information and image feature information, deformation analysis is performed to determine the tray status information.

[0145] The specific method in step S500 includes:

[0146] S501. Obtain the standard spacing threshold based on the tray information.

[0147] S502. Based on the standard spacing threshold, determine whether the image feature information is greater than the standard spacing threshold and output the judgment result.

[0148] S503. Based on the judgment result, determine the status information of the material tray.

[0149] S600 matches storage location information based on pallet status information and outputs pallet storage information.

[0150] The smart terminal provided in this embodiment can achieve the same technical effect as the aforementioned embodiments because the computer program in its memory runs on the processor. The principle analysis can be found in the relevant description of the aforementioned method steps, which will not be repeated here.

[0151] Example 4:

[0152] In one embodiment, a computer-readable storage medium is provided, which stores a computer program capable of being loaded by a processor and executed by the above-described method for identifying the entry and exit of SMT component reels. When executed by the processor, the computer program performs the following steps:

[0153] S100, Read tray information.

[0154] S200, Obtain the image of the material tray.

[0155] S300, image processing of the material tray, separates the target area and background area of ​​the material tray image.

[0156] The specific method in step S300 includes:

[0157] S301. Obtain comparison images based on tray information.

[0158] S302. Based on the size of the comparison image, cut the material tray image to obtain a preprocessed image.

[0159] S303. Convert the preprocessed image into a grayscale image and calculate the gradient field of the grayscale image; obtain the gradient field information of the comparison image, and complete the division of the target region and background region of the grayscale image based on the gradient field information.

[0160] S400: Extract image feature information from the target area and obtain the maximum spacing between the edge contours of the material tray.

[0161] S500 detection and analysis: Based on the tray information and image feature information, deformation analysis is performed to determine the tray status information.

[0162] The specific method in step S500 includes:

[0163] S501. Obtain the standard spacing threshold based on the tray information.

[0164] S502. Based on the standard spacing threshold, determine whether the image feature information is greater than the standard spacing threshold and output the judgment result.

[0165] S503. Based on the judgment result, determine the status information of the material tray.

[0166] S600 matches storage location information based on pallet status information and outputs pallet storage information.

[0167] The readable storage medium provided in this embodiment can achieve the same technical effect as the aforementioned embodiments because the computer program therein, after being loaded and run on the processor, will implement the various steps of the aforementioned embodiments. For the principle analysis, please refer to the relevant description of the aforementioned method steps, which will not be repeated here.

[0168] The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0169] The embodiments described in the specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the methods and principles of this application should be covered within the scope of protection of this application.

Claims

1. A SMT patch device tray warehouse in-out identification method, characterized in that, The method comprises the following steps: reading tray information, the tray information comprising tray size and standard tray spacing; acquiring a tray image, the imaging content of the tray image comprising a target region and a background region; tray image processing, separating the target region and the background region of the tray image, the target region being used to reflect the edge profile of the tray; extracting image feature information in the target region, and acquiring the maximum spacing between the edge profiles of the tray, comprising: inputting the target region into a feature extraction model, acquiring the overall profile of the target region, and mapping it to a preset plane coordinate system in the feature extraction model, to acquire the maximum spacing between the edge profiles of the tray, the y-coordinate axis of the plane coordinate system passing through the center of the tray, the image feature information comprising left spacing and right spacing; the image feature information reflecting the maximum spacing between the edge profiles of the tray, comprising: in the specific method of the step of acquiring the maximum spacing between the edge profiles of the tray, acquiring the point with the maximum y-coordinate and the point with the minimum y-coordinate on the negative half of the x-coordinate axis, to determine the left spacing, the left spacing reflecting the maximum spacing between the edge profiles of the tray on the negative half of the x-coordinate axis; acquiring the point with the maximum y-coordinate and the point with the minimum y-coordinate on the positive half of the x-coordinate axis, to determine the right spacing, the right spacing reflecting the maximum spacing between the edge profiles of the tray on the positive half of the x-coordinate axis; detection analysis, performing deformation analysis based on the tray information and the image feature information, to determine tray state information, the tray state information comprising normal state and deformation state; matching the tray state information with storage location information, and outputting tray storage information.

2. The SMT patch device tray warehouse in and out identification method according to claim 1, characterized in that, In the specific method of the step of tray image processing, comprising: acquiring a contrast image based on the tray information, the contrast image being a background image without a tray after being cropped; cropping the tray image based on the size of the contrast image, to obtain a pretreatment image, the pretreatment image being consistent with the position of the contrast image in the tray image; converting the pretreatment image into a gray image, calculating the gradient field of the gray image; acquiring gradient field information of the contrast image, and completing the division of the target region and the background region of the gray image according to the gradient field information.

3. The SMT patch device tray warehouse in and out identification method according to claim 1, characterized in that: Before the step of matching the tray state information with storage location information, and outputting tray storage information, there is still a step of judging whether to output rotation control information based on the deformation analysis, the rotation control information being used to control the horizontal rotation of the tray to realize orientation conversion; the deformation analysis comprising tray left deformation, tray right deformation, tray overall deformation and tray no deformation; when the deformation analysis is tray left deformation or tray right deformation, the rotation control information is outputted.

4. The SMT patch device tray warehouse in and out identification method according to claim 1, characterized in that: In the specific method of the step of detection analysis, comprising: acquiring a standard spacing threshold based on the tray information; judging whether the maximum spacing is greater than the standard spacing threshold based on the standard spacing threshold, and outputting a judgment result; determining the tray state information based on the judgment result.

5. An SMT pick-and-place device tray in-and-out system, characterized in that: The method comprises the following steps: an information reading module (11) is used to read tray information; the tray information comprising tray size and standard tray spacing; an image acquiring module (12) is used to acquire a tray image; the imaging content of the tray image comprising a target region and a background region; An image processing module (13) is configured to process the tray image, separate a target region and a background region of the tray image, and the target region is used to reflect the edge profile of the tray. A feature extraction module (14) is configured to extract image feature information in the target region and obtain a maximum distance between the edge profiles of the tray, and the feature extraction module includes: inputting the target region into a feature extraction model, obtaining an overall profile of the target region, mapping the overall profile to a preset plane coordinate system in the feature extraction model, and obtaining the maximum distance between the edge profiles of the tray, wherein a y-coordinate axis of the plane coordinate system passes through the center of the tray, and the image feature information includes a left distance and a right distance; the image feature information reflects the maximum distance between the edge profiles of the tray, and the feature extraction module includes: In a specific method of the step of obtaining the maximum distance between the edge profiles of the tray, obtaining a point with a maximum y-coordinate and a point with a minimum y-coordinate on a negative half of an x-coordinate axis, and determining the left distance, wherein the left distance reflects the maximum distance between the edge profiles of the tray on a side of the negative half of the x-coordinate axis; obtaining a point with a maximum y-coordinate and a point with a minimum y-coordinate on a positive half of the x-coordinate axis, and determining the right distance, wherein the right distance reflects the maximum distance between the edge profiles of the tray on a side of the positive half of the x-coordinate axis; an analysis module (15) is configured to perform deformation analysis based on tray information and image feature information, and determine tray state information, wherein the tray state information includes a normal state and a deformation state; a matching module (16) is configured to match bin information according to the tray state information, and output storage information.

6. The SMT patch device tray warehouse access system according to claim 5, characterized in that: Further comprising: a clamping module (22) is configured to clamp the tray so that the information reading module reads the tray information and the image acquisition module acquires the tray image; a rotating module (23) is configured to drive the clamping module (22) to rotate; an access module (25) is configured to transfer and store the tray according to the storage information output by the matching module (16).

7. A smart terminal, characterized by The storage includes a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor, and the computer program is capable of executing any one of the methods in claims 1 to 6.

8. A computer readable storage medium, characterized in that, The storage includes a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor, and the computer program is capable of executing any one of the methods in claims 1 to 6.

Images