A visual inspection and production management method and system for destacking sheet metal status
By combining optical images and resonant response data, the reliability of sheet metal detection in existing technologies has been improved, enabling accurate determination of sheet metal condition and dynamic adjustment of mold formula, thereby enhancing the efficiency and control of multi-variety production.
Patent Information
- Application Number
- CN202611628311.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies make it difficult to reliably distinguish between sheet metals with similar surface appearances but different structures on automotive hot stamping production lines. Furthermore, the test data is not effectively recorded and utilized, resulting in low production control efficiency when switching between multiple product types frequently.
By acquiring the optical image set and resonant response data of the sheet metal, extracting the contour point sequence and aligning it with the mold formula parameters, generating position determination results and grasping pose compensation values, and combining optical image matching and vibration response spectrum comparison, generating type, orientation and patch status determination results, achieving accurate determination of sheet metal status, and updating mold formula parameters.
It enables quantitative determination and type identification of sheet material placement deviation, ensuring consistency between sheet material and mold, identifying structural defects, and improving the switching efficiency and production control level of multi-variety co-production.
Smart Images

Figure CN123118681A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of sheet metal inspection technology, and more specifically, to a visual inspection and production management method and system for the state of destacking sheet metal. Background Technology
[0002] On an automotive hot stamping production line, a destacking trolley transfers sheet metal to the inspection station. A robot places the sheet metal onto a conveyor, which then feeds it into the press for stamping. The accuracy of the sheet metal placement on the conveyor directly affects the subsequent stamping quality and mold safety. The production line faces the challenge of producing multiple types of sheet metal on the same line. Different products require different molds and sheet metal specifications. Sheet metal placement errors mainly include positional misalignment, incorrect model numbers, missing patches, and reversed orientation. If these errors are not detected in time, they will lead to mold damage and product scrap. Therefore, there is an urgent need for automated detection of the destacking sheet metal status and rapid switching between multiple product types.
[0003] One existing inspection solution involves placing an industrial camera above the destacking station to photograph the sheet metal. The captured images are then compared with pre-stored standard templates. By calculating the degree of matching between image features and template features, the positional deviation and type of the sheet metal are determined. However, this existing solution relies solely on optical image comparison in practical applications, making it difficult to reliably distinguish between sheets with similar surface appearances but different structures. Furthermore, the deviation data generated during the inspection process is not effectively recorded and utilized. Each product change still relies on manual confirmation of the correctness of the template change. The inspection process is relatively isolated from the production management process, making it difficult to support efficient production control in scenarios with frequent changes in multiple product types. Summary of the Invention
[0004] The main objective of this application is to provide a visual inspection and production management method and system for destacking sheet metal, in order to solve the technical problem that the existing technology is not reliable enough in distinguishing sheet metal with structural differences due to the isolation between single optical comparison and inspection data and mold formula management.
[0005] To achieve the above objectives, the first aspect of this application proposes a visual inspection and production management method for the destacking status of sheet metal, comprising: Acquire the optical image set of the destacking sheet material to be detected, and the resonance response data of the destacking sheet material to be detected under vibration excitation. The contour point sequence is extracted from the optical image set, and the standard contour point sequence in the mold formula parameters corresponding to the current destacking sheet material to be detected is aligned with the contour point sequence to generate the corresponding position determination result and grasping pose compensation value. The optical image sets multiple feature points are matched with standard feature points to generate the type and orientation determination results of the destacking material to be detected. Based on the resonance response data, the corresponding vibration response spectrum is generated. The vibration response spectrum is compared with the standard spectrum template to obtain the corresponding patch status determination result. Based on the position determination result, grab pose compensation value, type determination result, orientation determination result and patch status determination result of the current destabilized sheet material to be detected, a status determination result is generated. Based on the current status determination result of the destacking sheet material to be inspected, the robot component performs the destacking operation on the current destacking sheet material to be inspected. After all the destacking operations of the current batch of destacking sheet materials to be inspected are completed, the corresponding mold formula parameters are updated based on the status determination results of multiple destacking sheet materials to be inspected in the current batch and the pressure drop rate during the corresponding destacking process.
[0006] Secondly, this application provides a visual inspection and production management system for the destacking status of sheet metal, including: The acquisition module is used to acquire the optical image set of the currently detected destacking sheet material and the resonance response data of the currently detected destacking sheet material under vibration excitation. The extraction module is used to extract the contour point sequence from the optical image set, align the standard contour point sequence in the mold formula parameters corresponding to the destacking sheet material to be detected with the contour point sequence, and generate the corresponding position determination result and grasping pose compensation value. The matching module is used to match multiple feature points in the optical image set with standard feature points to generate the type and orientation determination results of the current unstacked sheet material to be detected. Based on the resonance response data, the corresponding vibration response spectrum is generated, and the vibration response spectrum is compared with the standard spectrum template to obtain the corresponding patch status determination result. The generation module is used to generate status determination results based on the position determination results, grasped pose compensation values, type determination results, orientation determination results, and patch status determination results of the current destacking sheet material to be detected. The processing module is used to perform destacking operations on the current destacking sheet material based on the current status determination result of the destacking sheet material to be detected through the robot component, and after all the destacking operations of the current batch of destacking sheet materials are completed, update the corresponding mold formula parameters based on the status determination results of multiple destacking sheet materials to be detected in the current batch and the pressure drop rate during the corresponding destacking process.
[0007] Thirdly, this application provides an electronic device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the steps of a visual inspection and production management method for destacking sheet metal status as described in the first aspect above.
[0008] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the steps of a visual inspection and production management method for destacking sheet metal status as described in the first aspect above.
[0009] The technical solutions provided by the embodiments of this application may include the following beneficial effects: In this application, by acquiring the optical image set and resonant response data of the current destacking sheet material to be inspected, the contour point sequence extracted from the optical image set is aligned with the standard contour point sequence in the mold formula parameters to achieve quantitative determination of sheet material placement deviation and generate grasping pose compensation value, providing a precise grasping adjustment basis for robot components; multiple feature points in the optical image set are matched with standard feature points to generate type determination results and orientation determination results, ensuring the consistency between the current destacking sheet material to be inspected and the sheet material type corresponding to the current mold and identifying the front and back inversion; by generating a vibration response spectrum and comparing it with a standard spectrum template, the patch status is obtained. The judgment results can detect structural defects such as missing patches, making up for the shortcomings of single optical inspection in identifying structural differences. Based on the position judgment results, grasping pose compensation values, type judgment results, orientation judgment results, and patch status judgment results, a status judgment result is generated, providing a reliable decision basis for the robot component to perform destacking operations. After the current batch of destacking is completed, the corresponding mold formula parameters are updated based on the status judgment results of multiple destacking sheets and the pressure drop rate during the destacking process. This allows the mold formula parameters to be dynamically adjusted according to the actual production batch, realizing a closed-loop linkage between detection data and mold formula management, and improving the switching efficiency and production control level when multiple varieties are produced on the same line. Attached Figure Description
[0010] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:
[0011] Figure 1 A flowchart of a visual inspection and production management method for the destacking status of sheet metal provided in this application;
[0012] Figure 2This is a schematic diagram of a visual inspection and production management system for the destacking status of sheet metal provided in this application. Detailed Implementation
[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0015] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.
[0016] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.
[0017] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0018] The core of this application is to provide a visual inspection and production management method for the destacking status of sheet metal, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:
[0019] Step 101: Obtain the optical image set of the current destacking sheet material to be detected, and the resonance response data of the current destacking sheet material to be detected under vibration excitation.
[0020] Step 102: Extract the contour point sequence from the optical image set, align the standard contour point sequence in the mold formula parameters corresponding to the current destacking sheet material to be detected with the contour point sequence, and generate the corresponding position determination result and grasping pose compensation value.
[0021] Step 103: Match multiple feature points in the optical image with standard feature points to generate the type and orientation determination results of the current unstacked sheet material to be detected. Based on the resonance response data, generate the corresponding vibration response spectrum. Compare the vibration response spectrum with the standard spectrum template to obtain the corresponding patch status determination results.
[0022] Step 104: Generate a status determination result based on the position determination result, grasping pose compensation value, type determination result, orientation determination result, and patch status determination result corresponding to the current unstacked sheet material to be detected.
[0023] Step 105: Based on the current status determination result of the destacking sheet to be inspected, the robot component performs the destacking operation on the current destacking sheet to be inspected. After all the destacking operations of the current batch of destacking sheet to be inspected are completed, the corresponding mold formula parameters are updated based on the status determination results of multiple destacking sheet to be inspected in the current batch and the pressure drop rate during the destacking process.
[0024] In step 101 above, the sheet material to be inspected and destacking refers to a single sheet of metal located at the inspection position of the destacking station, about to be picked up by the robot and sent into the heating furnace for hot stamping. This sheet material may be a sheet material of a single material or a composite sheet material with patched plates welded to its surface. For example, the sheet material to be inspected and destacking can be a patched high-strength steel plate that has been placed at the inspection position of the destacking trolley and is waiting for visual inspection and picking.
[0025] An optical image set refers to at least one digital image obtained by triggering the capture of one or more industrial cameras mounted above the destacking trolley after the current destacking sheet material to be detected reaches the detection position. Each digital image records the light intensity distribution information of the sheet material surface in the visible light band in the form of a pixel array, and each pixel position corresponds to a gray value.
[0026] The resonant response data under vibration excitation refers to the mechanical excitation force with continuously varying frequency applied to the sheet material to be tested by the exciter installed in the destacking station support assembly, while the waveform data of the vibration velocity of the sheet material surface in the excitation direction changing with time is collected by the laser Doppler vibration meter installed near the edge of the sheet material. The waveform data is recorded in the form of a discrete sequence of voltage amplitude changing with sampling time. Each sampling point contains the instantaneous velocity value of the particles on the sheet material surface along the measurement direction at a certain moment after the vibration excitation.
[0027] In this embodiment of the application, after the destacking trolley transports the current destacking material to be inspected to the inspection position and completes the positioning, the robot component sends a destacking command to the control unit. The control unit then triggers the vision camera to expose and take a picture of the current destacking material to be inspected located in the center of the field of view, and collects at least one optical image containing the complete outline and surface features of the material. Each optical image is associated with the current mold number and stored to form an optical image set.
[0028] The control unit generates a sweep drive signal with a frequency that changes continuously over time and outputs this sweep drive signal to the exciter. The exciter applies a vertical excitation force to the lower surface of the sheet metal according to the sweep drive signal. The frequency of this excitation force changes continuously from low to high, covering the first frequency range corresponding to the overall bending vibration of the sheet metal and the second frequency range corresponding to the local vibration of the patch. The destacking sheet metal to be tested undergoes forced vibration under the excitation force, causing vertical displacement at the edge points of the sheet metal. The laser Doppler vibrometer emits a laser beam towards the edge points of the sheet metal, collects the instantaneous velocity changes caused by the vibration at the edge points of the sheet metal, and obtains the resonant response data composed of discrete sampling points after analog-to-digital conversion. The amplitude distribution of different frequency components in the resonant response data reflects the vibration response intensity of the sheet metal under different frequency excitations. Among them, the frequency component corresponding to the overall bending natural frequency of the sheet metal has a larger amplitude. If there are patches on the surface of the sheet metal, the frequency component corresponding to the local natural frequency of the patch will also produce a response characteristic with increased amplitude in the resonant response data.
[0029] In step 102 above, the step "extracting the contour point sequence from the optical image set" specifically includes the following steps: Step 201: Perform edge pixel detection and filtering on each optical image in the optical image set to obtain the edge pixel set.
[0030] Step 202: Extract the set of edge pixels that meet the closure condition from the set of edge pixels, and take the set of edge pixels with the largest number of edge pixels as the contour point sequence.
[0031] In step 201 above, the edge pixel set refers to the set of all pixels detected from the optical image located at the boundary between the sheet metal image area and the background area, including multiple edge pixels; an edge pixel refers to a single pixel in the edge pixel set located at the boundary of the sheet metal contour, which has unique row and column coordinates in the image coordinate system, and at least one pixel in its eight neighborhood has a gray value difference exceeding a preset edge intensity judgment value. The image coordinate system refers to a system with the top-left pixel of the optical image as the origin, and the horizontal direction to the right as the positive direction of the column coordinate axis. A two-dimensional pixel position coordinate system is established with the vertical downward direction as the positive direction of the row coordinate axis. The position of each pixel in this coordinate system is uniquely determined by its row number and column number. Both row coordinates and column coordinates are in pixels. The preset edge strength judgment value refers to the gray value difference threshold used to distinguish edge pixels from non-edge pixels. When the absolute value of the difference between the gray value of a pixel and any pixel in its eight neighborhoods is greater than the judgment value, the pixel is initially identified as an edge candidate pixel. The judgment value can be set according to the degree of contrast between the gray value of the sheet metal surface and the gray value of the background table of the destacking trolley. This application does not limit this.
[0032] In this embodiment, for each optical image in the optical image set: each pixel in each optical image is scanned line by line, the difference in grayscale value between each pixel and its horizontal and vertical adjacent pixels is calculated, the sum of squares of the grayscale value differences in the two directions is calculated, and the square root of the sum of squares is then performed to obtain the edge strength value. Pixels with edge strength values exceeding a preset edge strength judgment value are selected as candidate edge pixels, and candidate edge pixels with edge strength values that are the maximum in their neighborhood are extracted as local maxima pixels. Then, a dual threshold screening is performed on the local maxima pixels: local maxima pixels with edge strength values exceeding the high threshold are classified as strong edge pixels, and local maxima pixels with edge strength values between the low and high thresholds are classified as weak edge pixels. Pixels in the weak edge pixels that are connected to the eight-neighborhood of the strong edge pixels are retained, and the strong edge pixels and the retained weak edge pixels are integrated into an edge pixel set.
[0033] Local maxima pixels refer to candidate edge pixels whose edge intensity value is greater than that of all other candidate edge pixels in their eight-neighborhood. These pixels are used to filter out the most prominent pixels in the edge response, thereby refining the edge position and reducing the edge width. The high and low thresholds can be set according to the complexity of the sheet material's surface texture; this application does not impose any limitations on this.
[0034] In step 202 above, the closure condition refers to the geometric constraint that a closed loop is formed by connecting several edge pixels end to end. That is, starting from any edge pixel in the set of edge pixels, the connection can be made sequentially along the adjacent edge pixels and eventually return to the starting point. There are no breaks in the connection path. If there is a break in the connection path, that is, if an edge pixel cannot find another edge pixel belonging to the same set as the next connection point in its eight neighborhood, then the set does not satisfy the closure condition.
[0035] A contour point sequence refers to an ordered list of pixels formed by arranging all pixels in the set of edge pixels that meet the closure condition and contain the largest number of edge pixels, according to their adjacency. Each point in the sequence has pixel coordinates in the image coordinate system, and adjacent points are connected in their eight-neighborhood in the image. This sequence corresponds to the outer contour shape of the sheet metal in the image.
[0036] In this embodiment, starting from any edge pixel in the edge pixel set, adjacent edge pixels are sequentially searched within the eight-neighborhood and marked as visited. If the search path can return to the starting point, all edge pixels on that path constitute a set of edge pixels that satisfy the closure condition. When multiple sets of edge pixels satisfy the closure condition exist, the number of edge pixels in each set is counted, and the edge pixels in the set with the largest number of edge pixels are arranged in the order of the search path to form a contour point sequence. If the sheet metal surface has a patch plate, the contour of the patch plate itself will also be extracted by edge detection to form an additional set of edge pixels. However, the perimeter of the outer contour of the main sheet metal is greater than the perimeter of the patch plate contour. Under the constraint of selecting the set with the largest number of edge pixels, the interference of the patch plate contour will be eliminated.
[0037] The step "Aligning the standard contour point sequence with the contour point sequence in the mold formula parameters corresponding to the current destacking sheet material to be detected, and generating the corresponding position determination result and grasping pose compensation value" specifically includes the following steps: Step 211: Use the point set rigid registration algorithm to align the contour point sequence with the standard contour point sequence in the mold formula parameters corresponding to the current destacking material to generate the target rotation amount and target translation amount of the current destacking material.
[0038] Step 212: Based on the target rotation and target translation, perform a spatial transformation on the contour point sequence to obtain the transformed contour point sequence. Compare the transformed contour point sequence with the standard contour point sequence to calculate the contour overlap ratio.
[0039] Step 213: Compare the target rotation amount, target translation amount, and contour overlap ratio of the current unstacked sheet material to the angle tolerance, position tolerance, and preset ratio threshold in the corresponding mold formula parameters to generate the corresponding position determination result.
[0040] Step 214: Based on the target rotation and target translation, perform spatial transformation on the coordinates of each standard gripping point in the corresponding mold formula parameters to obtain multiple transformed coordinates. Integrate the transformed coordinates to obtain the corresponding gripping pose compensation value.
[0041] Step 211 above specifically includes the following steps: Step 221: Calculate the Euclidean distance between the pixel coordinates of each contour point in the contour point sequence and the pixel coordinates of each standard contour point in the standard contour point sequence. Based on the Euclidean distance between each contour point and each standard contour point, match each contour point with each standard contour point to obtain multiple contour point pairs.
[0042] Step 222: Iteratively solve for the rotation and translation amounts that minimize the sum of squared Euclidean distances between each pair of contour points until the decrease in the sum of squared Euclidean distances is less than a preset convergence threshold. The rotation and translation amounts solved when the decrease in the sum of squared Euclidean distances is less than the preset convergence threshold are taken as the target rotation and target translation amounts, respectively.
[0043] In step 221 above, the mold formula parameters refer to the set of detection benchmark data corresponding to the production mold of the destacking material to be detected, which is pre-established and stored in the control unit. It includes at least the standard contour point sequence, standard feature point template, standard spectrum template, angle tolerance, position tolerance, preset proportion threshold and standard gripping point coordinate set. It is established by the operator through multiple sets of standard destacking material sampling photos and vibration detection, and is automatically loaded by the control unit when changing the production mold.
[0044] The standard contour point sequence refers to the ordered set of pixel coordinates of the closed contour obtained by image acquisition and edge extraction of standard destacking sheet material stored in the mold formula parameters. Each standard contour point has standard pixel coordinates in the image coordinate system, and adjacent points satisfy the eight-neighbor connectivity relationship. This standard contour point sequence can be used as a reference benchmark for calculating the position deviation of the current destacking sheet material to be detected.
[0045] A contour point pair is a pair of points formed by pairing any contour point in a contour point sequence with the standard contour point in a standard contour point sequence that has the smallest Euclidean distance to that contour point. Each contour point pair contains one contour point and one standard contour point, and is used to establish the spatial correspondence between two point sequences.
[0046] In this embodiment, a standard contour point sequence is read from the mold formula parameters. The Euclidean distance between the pixel coordinates of each contour point in the contour point sequence and the pixel coordinates of each standard contour point in the standard contour point sequence is calculated. The standard contour point with the smallest Euclidean distance to the contour point is selected, and the two are paired as a contour point pair. After traversing all contour points in the contour point sequence, multiple contour point pairs are obtained.
[0047] In step 222 above, the rotation amount refers to the angle value, expressed in radians, used to rotate the pixel coordinates of each contour point in the contour point sequence around the origin of the image coordinate system. After rotation, the orientation of the contour point sequence tends to be consistent with that of the standard contour point sequence. The translation amount refers to the pixel distance moved by the rotated contour point sequence in the horizontal and vertical directions, respectively. After translating the rotated contour point sequence based on the translation amount, the position of the translated contour point sequence tends to coincide with that of the standard contour point sequence.
[0048] The preset convergence threshold is a critical value used to determine whether the iterative solution process can be terminated. When the decrease in the sum of squared Euclidean distances between all contour point pairs between two adjacent iterations is lower than this threshold, it indicates that continuing the iteration will have very little effect on improving the alignment accuracy, and the iteration process terminates. This preset convergence threshold can be set according to the detection accuracy requirements, and this application does not limit the value of the preset convergence threshold.
[0049] The target rotation amount refers to the rotation amount obtained when the iteration terminates, representing the actual deflection angle of the current destabilized sheet material relative to the standard destabilized sheet material on the destabilizing table. The target translation amount refers to the translation amount obtained when the iteration terminates, representing the actual positional offset of the current destabilized sheet material relative to the standard destabilized sheet material on the destabilizing table.
[0050] In this embodiment, the squared Euclidean distances between the pixel coordinates of each contour point in multiple contour point pairs and the pixel coordinates of the corresponding standard contour point are superimposed to obtain the sum of squared Euclidean distances. The target rotation and target translation are solved using the iterative nearest-point algorithm. Specifically, in each iteration, based on the currently established contour point pairs, the coordinates of the contour point pairs are substituted into the closed-form formula to solve for the rotation and translation that minimizes the sum of squared Euclidean distances of all contour point pairs. This closed-form formula is derived by taking the partial derivative of the sum of squared Euclidean distances with respect to the rotation and translation and setting it to zero. It can calculate the optimal rotation and translation for the current contour point pair in one step without iterative approximation. The pixel coordinates of each contour point in the contour point sequence are then subjected to rotation and translation transformations using the rotation and translation, resulting in a transformed contour point sequence. Contour point pairs are then re-established based on the transformed contour point sequence and the standard contour point sequence. The new sum of squared Euclidean distances is calculated, and the decrease in the sum of squared Euclidean distances between the current iteration and the previous iteration is calculated. The iteration stops when the decrease is less than the preset convergence threshold. The rotation amount solved in the last iteration is taken as the target rotation amount, and the translation amount solved in the last iteration is taken as the target translation amount.
[0051] Regarding the above-mentioned "rotation transformation and translation transformation" steps, a specific implementation method of the transformation can be achieved as follows: First, based on: the row coordinates of each contour point after rotation transformation = the row coordinate in the pixel coordinates × the cosine of the target rotation amount - the column coordinate in the pixel coordinates × the sine of the target rotation amount; the column coordinates after rotation transformation = the row coordinates in the pixel coordinates × the sine of the target rotation amount + the column coordinates in the pixel coordinates × the cosine of the target rotation amount, perform rotation transformation on the pixel coordinates of each contour point, and the row coordinates and column coordinates after rotation transformation of each contour point constitute the sequence of contour points after rotation transformation.
[0052] Then, based on the following: after translation transformation, row coordinates = after rotation transformation, row coordinates + vertical component of target translation, and after translation transformation, column coordinates = after rotation transformation, column coordinates + horizontal component of target translation, a translation transformation is performed on the pixel coordinates of each contour point in the contour point sequence after rotation transformation. The translated row coordinates and translated column coordinates of each contour point are used as the final pixel coordinates after spatial transformation of each contour point. The corresponding final pixel coordinates are arranged in the original order of the contour points to form the transformed contour point sequence.
[0053] In step 212 above, the transformed contour point sequence refers to the ordered set of new pixel coordinates obtained by rotating and translating the pixel coordinates of each contour point in the contour point sequence according to the target rotation amount. The relative positional relationship between each contour point in the sequence remains unchanged, but the overall position and orientation have been adjusted to the posture closest to the standard contour point sequence.
[0054] The contour overlap ratio refers to the ratio obtained by dividing the area of the closed region enclosed by the transformed contour point sequence and the area of the closed region enclosed by the standard contour point sequence by the area of the closed region enclosed by the standard contour point sequence. It reflects the overall degree of conformity between the two shapes. The closer the ratio is to one, the more consistent the shape of the destacking material to be detected is with the shape of the standard destacking material.
[0055] In this embodiment, the same rotation and translation transformation method as in step 222 is used. Based on the target rotation and target translation, the contour point sequence is spatially transformed to obtain the transformed contour point sequence. Then, the contour points in the transformed contour point sequence are connected sequentially to form a first closed region, and the standard contour points in the standard contour point sequence are connected sequentially to form a second closed region. The number of pixels simultaneously located in these two closed regions is counted and divided by the total number of pixels in the second closed region to obtain the contour overlap ratio.
[0056] In step 213 above, angular tolerance refers to the maximum range of allowable sheet metal deflection angles set in the mold formula parameters, expressed in radians. Positional tolerance refers to the maximum range of allowable sheet metal positional offsets set in the mold formula parameters, expressed in pixels.
[0057] The preset percentage threshold refers to the minimum acceptable value for the outline overlap ratio set in the mold formula parameters. This preset percentage threshold can be set according to the complexity of the sheet material shape and the requirements for detection accuracy. In this application embodiment, the specific value of the preset percentage threshold is not limited.
[0058] The position determination result refers to the judgment conclusion generated by comprehensively considering the comparison results of the absolute value of the target rotation amount and the angle tolerance, the comparison results of the absolute value of the target translation amount and the position tolerance, and the comparison results of the contour overlap ratio and the preset ratio threshold, which indicates whether the plate placement position is qualified. When all three comparison results meet the corresponding requirements, a position determination result indicating that the position is qualified is generated; otherwise, a position determination result indicating that the position is unqualified is generated.
[0059] In this embodiment, the absolute value of the target rotation amount is compared with the angle tolerance in the mold formula parameters. If the absolute value of the target rotation amount does not exceed the angle tolerance, the rotation is deemed qualified, meaning that the actual deflection angle of the current destabilized sheet material on the destabilization table is within the allowable deflection range, and the sheet material has not undergone excessive rotation. The absolute value of the target translation amount is compared with the position tolerance in the mold formula parameters. If the absolute value of the target translation amount does not exceed the position tolerance, the translation is deemed qualified, meaning that the actual positional offset of the current destabilized sheet material in the horizontal and vertical directions is within the allowable offset range, and the sheet material has not been placed too far from the standard position.
[0060] The contour overlap ratio is compared with the preset ratio threshold in the mold formula parameters. If the contour overlap ratio is greater than or equal to the preset ratio threshold, the shape is judged to be qualified, that is, the degree of overlap between the closed area enclosed by the transformed contour point sequence and the closed area enclosed by the standard contour point sequence meets the requirements, and the shape of the destacking material to be tested matches the shape of the standard destacking material well. When the rotation qualification, translation qualification and shape qualification are met at the same time, a position judgment result indicating that the position is qualified is generated. When any one of them is not met, a position judgment result indicating that the position is unqualified is generated.
[0061] In step 214 above, the standard gripping point coordinates refer to the pixel coordinates of the target adsorption positions of each suction cup on the standard destacking sheet material surface, which are stored in advance in the mold formula parameters in the image coordinate system. Each suction cup corresponds to one standard gripping point coordinate, and the standard gripping point coordinates corresponding to multiple suction cups constitute a standard gripping point coordinate set.
[0062] The transformed coordinates refer to the new pixel coordinates obtained by rotating and translating the coordinates of each standard gripping point according to the target rotation and target translation. These coordinates correspond to the position of the adsorption point at the same relative position on the actual sheet material in the image coordinate system.
[0063] The grasping pose compensation value refers to a set of coordinate values formed by arranging and combining the transformed coordinates in the order of the suction cup numbers. It is used to directly guide the robot components to adjust the target alignment position of each suction cup when grasping the current destacking material to be inspected.
[0064] In this embodiment, a set of standard gripping point coordinates is extracted from the mold formula parameters. The same rotation and translation transformation method as in step 222 is used to perform spatial transformation on the coordinates of each standard gripping point to obtain the transformed standard gripping point coordinates. The transformed standard gripping point coordinates are used as the transformed coordinates of the adsorption points on the actual sheet material corresponding to each suction cup. The transformed coordinates are arranged and combined in the order of suction cup number to form gripping pose compensation values.
[0065] The position determination results generated in this application improve the accuracy of sheet placement position detection; the gripping pose compensation value enables the robot component to automatically adjust the gripping pose according to the actual position of the sheet, thereby improving the reliability of sheet gripping and positioning during multi-variety co-line production.
[0066] In step 103 above, firstly, based on matching multiple feature points in the optical image set with standard feature points, the type determination result and orientation determination result of the current unstacked sheet material to be detected are generated. Then, based on the resonance response data, the corresponding vibration response spectrum is generated. Finally, the vibration response spectrum is compared with the standard spectrum template to obtain the corresponding patch status determination result.
[0067] The step "matching multiple feature points in the optical image set with standard feature points to generate the type and orientation determination results of the current destacking material to be detected" specifically includes the following steps: Step 301: Select a target sheet image that meets the preset image quality conditions from the optical image set, perform feature detection on the target sheet image, and obtain multiple feature points.
[0068] Step 302: Extract the local image block corresponding to each feature point from the target sheet image, and calculate the grayscale difference between each local image block and the corresponding template image block to obtain the sum of the absolute values of the grayscale differences corresponding to each feature point. The template image block is the image block corresponding to each standard feature point in the standard feature point template.
[0069] Step 303: Mark the feature points whose sum of absolute grayscale differences is less than the preset difference threshold and their corresponding standard feature points as matching point pairs, count the total number of matching point pairs, and combine the preset number of matching pairs to generate the type determination result of the current unstacked sheet material to be detected.
[0070] Step 304: Based on the orientation of each standard feature point in each matching point pair, generate the orientation determination result of the current destacking sheet material to be detected.
[0071] In step 301 above, the preset image quality conditions refer to the basic requirements that must be met when selecting the target sheet image for feature point detection from the optical image set, including that the current unstacked sheet to be detected is completely located in the center of the image field of view, there is no motion blur in the sheet area in the image, and the surface features of the sheet are clearly distinguishable.
[0072] The target sheet image refers to an optical image selected from the optical image set that meets the preset image quality conditions, used to detect feature points in order to identify the sheet type and orientation.
[0073] Feature points refer to pixels detected in a target sheet image that have a significant difference in grayscale distribution from the surrounding pixel areas. These include corner points and spots. Corner points are pixels where two edge directions intersect in the image, while spots are the center pixels of local areas in the target sheet image where the grayscale value is significantly different from that of its neighboring pixels.
[0074] In this embodiment, an optical image of the sheet metal that is completely centered in the field of view and whose image clarity meets the feature detection requirements is selected from the optical image set as the target sheet metal image. Corner detection is performed on the target sheet metal image, and the gray-level gradient of each pixel in the target sheet metal image and its neighboring pixels in the horizontal and vertical directions is calculated. Pixels whose gradient changes in both directions exceed a preset corner response threshold are identified as corners. The preset corner response threshold is a critical value used to distinguish between corners and non-corner points. This critical value can be set according to the edge sharpness of the feature points on the sheet metal surface. Spot detection is performed on the target sheet metal image, and the central pixel of a local area in the target sheet metal image whose gray value differs from the average gray value of its surrounding neighboring pixels from a preset spot response threshold is identified as a spot. The preset spot response threshold is a critical value used to distinguish between spots and non-spots. This threshold can be set according to the gray-level contrast of solder joints or local markings on the sheet metal surface. This application does not limit this.
[0075] In step 302 above, a local image block refers to a square pixel region of a preset size selected around each feature point in the target sheet image, centered on that feature point. This region contains the grayscale distribution information of the feature point and its neighboring pixels. A template image block refers to a square pixel region of a preset size corresponding to each standard feature point in the standard feature point template. This region contains the grayscale distribution information of the standard feature point and its neighboring pixels.
[0076] The sum of the absolute values of grayscale differences refers to the value obtained by subtracting the grayscale value of each pixel in the local image block from the grayscale value of the corresponding pixel in the template image block, taking the absolute value, and then summing the absolute values of the grayscale differences of all pixel positions. This value is used to measure the similarity between the local image block and the template image block. The smaller the value, the more similar the two are.
[0077] In this embodiment, a standard feature point template is read from the mold formula parameters. Each standard feature point in the template has a corresponding template image block and orientation mark. A local image block of a preset size is extracted around each feature point in the target sheet image, with the preset size corresponding to the size of the local image block used when establishing the standard feature point template. The grayscale difference between the local image block of each feature point and the template image block corresponding to each standard feature point in the standard feature point template is calculated one by one. Then, the absolute values of the grayscale differences at all pixel positions are summed to obtain the sum of the absolute values of the grayscale differences between each feature point and each standard feature point.
[0078] In step 303 above, a matching point pair refers to a point pair formed by pairing a feature point whose sum of the absolute values of the grayscale differences is less than a preset difference threshold with a standard feature point corresponding to the preset difference threshold. This indicates that the feature point and the standard feature point are highly similar in local grayscale distribution, and the two correspond to the same position or the same type of local feature on the sheet metal.
[0079] The preset difference threshold refers to the critical value of grayscale difference used to determine whether a feature point matches a standard feature point. When the sum of the absolute values of the grayscale differences is less than the threshold, the two are considered to match. This threshold can be set according to the complexity of the surface texture of the sheet and the requirements of detection accuracy. This application does not limit this.
[0080] The preset matching quantity refers to the minimum number of matching point pairs set in the mold formula parameters to determine the correctness of the sheet material type. This quantity is set according to the total number of standard feature points in the standard feature point template and the difficulty of distinguishing sheet material types. This application does not limit this quantity.
[0081] The type determination result refers to the judgment conclusion generated based on the comparison between the total number of matching points and the preset matching number, indicating whether the type of the currently detected destacking sheet material is consistent with the type of sheet material corresponding to the current mold.
[0082] In this embodiment, the sum of the absolute values of the grayscale differences between each feature point and each standard feature point is compared with a preset difference threshold. If the sum of the absolute values of the grayscale differences is greater than or equal to the preset difference threshold, the feature point is determined to be mismatched and is not considered. Feature points whose sum of the absolute values of the grayscale differences is less than the preset difference threshold are marked as a matching point pair with their corresponding standard feature points. After traversing all feature points, the total number of matching point pairs is counted. The total number of matching point pairs is compared with the preset matching number in the mold formula parameters. When the total number of matching point pairs is greater than or equal to the preset matching number, a type determination result indicating that the type is correct is generated, that is, the type of the destacking material to be detected is consistent with the type of the material corresponding to the current mold. When the total number of matching point pairs is less than the preset matching number, a type determination result indicating that the type is incorrect is generated.
[0083] In step 304 above, the orientation index of the standard feature point is the pre-assigned sheet orientation attribute of each standard feature point in the feature point template. The attribute value includes front and back, which is used to determine the front and back status of the current unstacked sheet after a successful match.
[0084] The orientation determination result refers to the conclusion generated based on the orientation distribution statistics of the standard feature points of each matching point, indicating whether the currently detected destacking board material is on the front or back.
[0085] In this embodiment, all matching point pairs are traversed, and the number of standard feature points facing forward and facing backward in each matching point pair is counted. When the number of standard feature points facing forward is greater than the number of standard feature points facing backward, an orientation determination result indicating a forward orientation is generated; when the number of standard feature points facing backward is greater than the number of standard feature points facing forward, an orientation determination result indicating a backward orientation is generated.
[0086] The step "generating the corresponding vibration response spectrum based on the resonance response data" specifically includes the following steps: Step 311: Perform time-frequency transformation on the resonant response data to obtain the vibration acceleration spectrum.
[0087] Step 312: Based on the first frequency range and the second frequency range in the mold formula parameters corresponding to the destacking material to be tested, the vibration acceleration spectrum is divided into a first frequency band and a second frequency band. The first frequency range is determined based on the bending vibration characteristics of the destacking material, and the second frequency range is based on the local vibration characteristics of the patch.
[0088] Step 313: Extract the first target frequency corresponding to the peak value of acceleration amplitude and the first phase value corresponding to the first target frequency from the first frequency band, and generate patch vibration characteristic parameters based on the acceleration amplitude in the second frequency band.
[0089] Step 314: Integrate the first target frequency, the first phase value, and the patch vibration characteristic parameters to generate a vibration response spectrum.
[0090] In step 311 above, the vibration acceleration spectrum refers to the spectrum obtained by transforming the resonant response data through time-frequency conversion, with frequency as the abscissa and acceleration amplitude as the ordinate. Different frequency positions in the spectrum correspond to the vibration response intensity of different frequency components.
[0091] In this embodiment, a fast Fourier transform is performed on the instantaneous velocity values of each sampling point in the resonant response data to convert the instantaneous velocity values arranged in the sampling time order in the time domain into velocity amplitudes arranged in the frequency order in the frequency domain. Each frequency point corresponds to a velocity amplitude, thus obtaining the vibration velocity spectrum. Then, the velocity amplitude of each frequency point in the vibration velocity spectrum is multiplied by the angular frequency value corresponding to that frequency point to convert the vibration velocity spectrum into a vibration acceleration spectrum. The aforementioned angular frequency value is equal to the frequency value of that frequency point multiplied by twice pi.
[0092] In step 312 above, the first frequency range refers to the frequency range in which the natural frequency of the overall bending vibration of the sheet material is located, as set in the mold formula parameters. This frequency range is determined based on the geometric dimensions and material properties of the sheet material.
[0093] The second frequency range refers to the frequency range in which the natural frequency of the local vibration of the corresponding patch is located, as set in the mold formula parameters. This frequency range is determined based on the connection structure properties between the patch and the sheet metal.
[0094] The first frequency band refers to the sub-spectrum formed by dividing the portion of the vibration acceleration spectrum whose frequency values fall within a first frequency range. The second frequency band refers to the sub-spectrum formed by dividing the portion of the vibration acceleration spectrum whose frequency values fall within a second frequency range.
[0095] In this embodiment, a first frequency range and a second frequency range are read from the mold formula parameters, and each frequency point in the vibration acceleration spectrum is divided according to its frequency value and its association with the first frequency range and the second frequency range: the frequency points with frequency values in the first frequency range and their acceleration amplitudes are divided into the first frequency band, and the frequency points with frequency values in the second frequency range and their acceleration amplitudes are divided into the second frequency band.
[0096] In step 313 above, firstly, the acceleration amplitude at each frequency point is compared point by point along the frequency-increasing direction within the first frequency band. The frequency value corresponding to the frequency point where the acceleration amplitude reaches its maximum value is taken as the first target frequency, and the phase angle of the acceleration spectrum at that frequency point is taken as the first phase value. The first target frequency refers to the frequency value corresponding to the frequency point where the acceleration amplitude reaches its peak value in the first frequency band, which corresponds to the natural frequency of the overall bending vibration of the sheet metal. The first phase value refers to the phase angle of the vibration acceleration signal at the first target frequency, reflecting the time relationship between the vibration response of the overall bending vibration of the sheet metal at the first target frequency and the excitation force input.
[0097] Then, based on the acceleration amplitude in the second frequency band, patch vibration characteristic parameters are generated. This step specifically includes the following steps: Step 321: Extract the second target frequency corresponding to the peak value of acceleration amplitude and the second phase value corresponding to the second target frequency from the second frequency band.
[0098] Step 322: Calculate the ratio of the sum of acceleration amplitudes in the second frequency band to the sum of acceleration amplitudes in the first frequency band, and count the number of target spectral peaks in the second frequency band whose corresponding acceleration amplitudes exceed the preset amplitude threshold.
[0099] Step 323: Match the frequency corresponding to each target spectral peak with the standard frequency point set in the mold formula parameters corresponding to the current destacking sheet material to be tested, so as to determine the target standard frequency corresponding to each target spectral peak.
[0100] Step 324: Based on the frequency values of each target spectral peak, determine the frequency distribution range and calculate the frequency offset between the frequency value of each target spectral peak and the corresponding target standard frequency.
[0101] Step 325: Integrate the second target frequency, second phase value, ratio, number of target spectral peaks, frequency distribution range, and frequency offset to obtain the patch vibration characteristic parameters.
[0102] In step 321 above, the second target frequency refers to the frequency value corresponding to the frequency point where the acceleration amplitude reaches its peak in the second frequency band, which corresponds to the natural frequency of the patch's local vibration. The second phase value refers to the phase angle of the vibration acceleration signal at the second target frequency, reflecting the time relationship between the vibration response of the patch's local vibration at the second target frequency and the excitation force input.
[0103] In this embodiment of the application, the acceleration amplitude of each frequency point is compared point by point along the frequency increasing direction in the second frequency band. The frequency value corresponding to the frequency point where the acceleration amplitude reaches the maximum value is taken as the second target frequency, and the phase angle of the acceleration spectrum at that frequency point is taken as the second phase value.
[0104] In step 322 above, the preset amplitude threshold refers to the minimum acceleration amplitude critical value used to screen spectral peaks with significant vibration response in the second frequency band. Frequency points with acceleration amplitude exceeding the critical value are considered to constitute a valid spectral peak. The critical value can be set according to the noise level in the second frequency band, and this application does not limit it.
[0105] The target spectral peak refers to the local maximum frequency point in the second frequency band where the acceleration amplitude exceeds the preset amplitude threshold. Each target spectral peak corresponds to a resonance response of the local vibration of the patch.
[0106] In this embodiment, the sum of acceleration amplitudes at all frequency points within the second frequency band is divided by the sum of acceleration amplitudes at all frequency points within the first frequency band, and the resulting ratio is the patch vibration energy ratio. The acceleration amplitude at each frequency point is scanned point-by-point along the frequency increasing direction of the second frequency band. Frequency points whose acceleration amplitude exceeds a preset amplitude threshold and is a local maximum among their adjacent frequency points are identified as target spectral peaks. The total number of target spectral peaks is then counted to obtain the number of spectral peaks.
[0107] In step 323 above, the standard frequency point set refers to the ordered set of multiple standard spectral peak frequency values that are pre-stored in the mold formula parameters and established after vibration testing of the standard patch on the intact standard sheet. Each standard frequency value corresponds to a standard resonance frequency of the patch's local vibration.
[0108] The target standard frequency refers to the frequency value in the standard frequency point set that is closest to the frequency corresponding to the target spectral peak after matching the frequency corresponding to the target spectral peak with the standard frequency point set.
[0109] In this embodiment of the application, a standard frequency point set is read from the mold formula parameters. For each target spectral peak, the standard frequency value with the smallest absolute difference from the target frequency value is searched in the standard frequency point set, and the standard frequency value is taken as the target standard frequency corresponding to the target spectral peak.
[0110] In step 324 above, the frequency distribution range refers to the frequency range between the highest and lowest frequency values corresponding to each target spectral peak, reflecting the distribution width of the target spectral peak within the second frequency band. The frequency offset refers to the difference between the frequency value corresponding to each target spectral peak and the corresponding target standard frequency, reflecting the degree and direction of the offset of the target spectral peak relative to the standard spectral peak on the frequency axis.
[0111] In this embodiment, the frequency distribution range is determined based on the numerical value of the frequency corresponding to each target spectral peak. The difference between the frequency value of each target spectral peak and the corresponding target standard frequency is calculated, and this difference is used as the frequency offset of the target spectral peak. A positive difference indicates that the target spectral peak frequency is higher than the standard spectral peak frequency, and a negative difference indicates that the target spectral peak frequency is lower than the standard spectral peak frequency.
[0112] In step 325 above, the patch vibration characteristic parameters refer to the set of parameters formed by combining the second target frequency, the second phase value, the patch vibration energy ratio, the number of spectral peaks, the frequency distribution range, and the frequency offsets. These parameters are used to describe the local vibration characteristics of the patch and are compared with a standard spectrum template to determine the patch status.
[0113] In this embodiment, the second target frequency, the second phase value, the patch vibration energy ratio, the number of spectral peaks, and the frequency distribution range are arranged sequentially. Then, the frequency offset of each target spectral peak is arranged sequentially after the frequency distribution range in order of the frequency value of the corresponding target spectral peak from low to high, thus forming the patch vibration characteristic parameters.
[0114] In step 314 above, the vibration response spectrum refers to the comprehensive set of spectrum parameters formed by integrating the first target frequency, the first phase value, and the patch vibration characteristic parameters, which also includes the overall bending vibration characteristics of the sheet and the local vibration characteristics of the patch.
[0115] In this embodiment, the first target frequency, the first phase value, and the patch vibration characteristic parameters are arranged in sequence to form a vibration response spectrum.
[0116] Finally, the vibration response spectrum is compared with the standard spectrum template to obtain the patch status judgment result. The patch status judgment result refers to the judgment conclusion generated after comparing the vibration response spectrum with the standard spectrum template, which indicates the existence and structural integrity of the patch on the currently inspected destacking plate. The patch status judgment result is that the patch is qualified, the patch is missing, or the patch is abnormal.
[0117] A specific implementation method for generating patch status determination results may be as follows: read a standard spectrum template from the mold formula parameters. The standard spectrum template includes the standard patch vibration energy ratio, the number of standard spectral peaks, the standard frequency distribution range, the standard frequency point set, and the corresponding allowable frequency offset range. Among them, the standard patch vibration energy ratio and the number of standard spectral peaks refer to the ratio of the sum of the acceleration amplitudes of the second frequency band to the sum of the acceleration amplitudes of the first frequency band, determined by the same method as steps 322-323, after vibration detection of the standard patch on the intact standard sheet, and the total number of target spectral peaks in the second frequency band, respectively. The standard frequency distribution range refers to the frequency range between the highest and lowest frequencies among the frequency values of each standard spectral peak, and the allowable frequency offset range refers to the maximum allowable offset of the frequency value of each target spectral peak relative to the corresponding standard frequency value.
[0118] The vibration energy ratio of the patch in the vibration response spectrum is compared with the vibration energy ratio of the standard patch. If the patch vibration energy ratio is lower than the difference between the standard patch vibration energy ratio and the preset energy ratio deviation range, it is determined that the patch is missing, that is, the patch has fallen off the sheet metal surface or the patch has not been welded. The vibration energy of the second frequency band is significantly reduced due to the missing patch. The preset energy ratio deviation range refers to the maximum allowable downward deviation of the measured patch vibration energy ratio relative to the standard patch vibration energy ratio set in the mold formula parameters. When the measured patch vibration energy ratio is lower than the difference obtained by subtracting this deviation range from the standard patch vibration energy ratio, it indicates that the vibration energy of the second frequency band has dropped to the energy level corresponding to the missing patch. This deviation range can be set according to the sheet metal material, the proportion of the patch area to the total sheet metal area, and the noise level of the vibration detection environment. This application does not limit this setting.
[0119] The number of spectral peaks is compared with the number of standard spectral peaks, the frequency distribution range is compared with the standard frequency distribution range, and each frequency offset is compared with the allowable frequency offset range. When the number of spectral peaks is equal to the number of standard spectral peaks, the frequency distribution range is within the difference between the standard frequency distribution range and the preset distribution range, and the absolute value of each frequency offset is within the allowable frequency offset range, the patch is deemed qualified, meaning the patch exists and is well connected to the board. Otherwise, the patch is deemed abnormal, indicating that the connection between the patch and the board has changed, and there may be poor bonding problems such as poor soldering, debonding, or decreased connection stiffness.
[0120] This application embodiment achieves reliable identification of sheet material type and front and back sides, vibration detection of patch presence and structural integrity; by performing visual inspection and vibration detection in parallel, it comprehensively judges the current unstacked sheet material to be inspected from two dimensions: appearance features and internal structure, making up for the inadequacy of single optical inspection in identifying structural differences.
[0121] In step 104 above, the state determination result refers to the structured data set generated based on the current position determination result of the destabilized sheet material to be detected, the gripping pose compensation value, the type determination result, the orientation determination result, and the patch state determination result, which is used to guide the robot components to perform destabilization operations. It includes the sheet material pickable mark, the target alignment position of each suction cup of the robot, and the corresponding suction cup pressure value.
[0122] In this embodiment, the position determination result, type determination result, orientation determination result, and patch status determination result are comprehensively determined: when all four determination results indicate that the sheet material is acceptable, a preliminary status determination result is generated, the gripping pose compensation value is used as the target alignment position of each suction cup of the robot component, the preset standard suction cup pressure value in the mold formula parameters is used as the suction cup pressure value, and the target alignment position and suction cup pressure value are added to the preliminary status determination result to obtain the status determination result.
[0123] When the patch status determination result indicates that the patch is abnormal, but the position determination result, type determination result, and orientation determination result are all qualified, a preliminary status determination result indicating that the sheet material is acceptable is generated. The gripping pose compensation value is used as the target alignment position of each suction cup of the robot component. The impact level of the patch abnormality is determined according to the degree of deviation between the frequency offset, number of spectral peaks, and patch vibration energy ratio in the patch vibration characteristic parameters and the corresponding parameters in the standard spectrum template. The higher the impact level, the worse the patch adhesion. Based on the impact level, the suction cup pressure value is reduced by a preset ratio on the basis of the standard suction cup pressure value to obtain the optimized suction cup pressure value. The preset ratio refers to the correspondence between the impact level and the reduction of suction cup pressure, which can be set according to the proportion of the patch area to the total area of the sheet material and the sheet material thickness. This application does not limit this. The target alignment position and the optimized suction cup pressure value are added to the preliminary status determination result to obtain the status determination result.
[0124] When any of the position, type, or orientation determination results indicates a failure, a preliminary status determination result indicating that the sheet material is unacceptable is generated. The recorded failure determination type is added to the preliminary status determination result to obtain the final status determination result. The status determination result is sent to the robot component, which determines whether to perform a gripping operation or stop destacking and issue an exception prompt based on the sheet material acceptability flag in the status determination result. The exception prompt includes the failure determination item.
[0125] For example, a specific embodiment for determining the impact level of patch anomalies may be as follows: when the number of spectral peaks is equal to the number of standard spectral peaks, the patch vibration energy ratio is greater than or equal to a preset ratio of the standard patch vibration energy ratio, and the absolute value of each frequency offset is within a preset multiple of the allowable frequency offset range, it is determined to be at the first impact level, corresponding to a patch with slight adhesion defects; when the number of spectral peaks is not equal to the number of standard spectral peaks, or the patch vibration energy ratio is less than a preset ratio of the standard patch vibration energy ratio but higher than the difference between the standard patch vibration energy ratio and the preset energy ratio deviation range, or the absolute value of any frequency offset exceeds a preset multiple of the allowable frequency offset range but does not exceed the allowable frequency offset range, it is determined to be at the second impact level, corresponding to a patch with moderate adhesion defects; when the following conditions are met simultaneously: the number of spectral peaks is not equal to the number of standard spectral peaks, the patch vibration energy ratio is lower than a preset ratio of the standard patch vibration energy ratio, and the absolute value of at least one frequency offset exceeds a preset multiple of the allowable frequency offset range, it is determined to be at the third impact level, corresponding to a patch with severe adhesion defects.
[0126] This application embodiment realizes a multi-dimensional comprehensive determination of the current position, type, orientation and patch status of the destacking board to be detected, enabling the robot component to automatically adjust the gripping position or trigger abnormal handling according to the actual state of the board, thereby improving the connection efficiency between detection results and execution actions.
[0127] In step 105 above, the current batch refers to a group of destacking sheets produced continuously under the same mold and using the same set of mold formula parameters, from the first sheet to be tested being destacking to the last sheet to be tested completing the complete production cycle.
[0128] The pressure drop rate during the destacking process refers to the rate at which the vacuum pressure in the end effector suction cup circuit decreases over time during the process of the robot component gripping the sheet material. It is expressed as the ratio of the change in vacuum pressure to the change time. This rate reflects the sealing state between the suction cup and the sheet material surface and the wear degree of the suction cup itself.
[0129] In this embodiment of the application, after all the destacking operations of the current batch of destacking sheets are completed, the status judgment results of all the destacking sheets in the current batch and the pressure drop rate during the corresponding destacking process are summarized, and the summarized data is used as the basis for updating the mold formula parameters.
[0130] The specific process for updating the corresponding mold formula parameters is as follows: Based on the target rotation and translation amounts from multiple position determination results in the current batch, the angular and positional tolerances in the mold formula parameters are corrected; based on multiple type and orientation determination results, the matching reliability of the standard feature point template is verified; based on multiple patch status determination results, the preset energy ratio deviation range and allowable frequency offset range of the patch vibration energy ratio used to determine patch status in the standard spectrum template are adjusted; based on the pressure drop rate during multiple destacking processes, the baseline curve of the standard suction cup pressure value and the pressure drop rate warning threshold in the mold formula parameters are optimized. Through the above updates, the mold formula parameters are dynamically adjusted according to the actual production batch performance, realizing closed-loop management of detection data and mold formula parameter updates. This replaces the repetitive labor of manual confirmation and adjustment during each product switch, improving the switchover efficiency and production control level when multiple products are produced on the same line.
[0131] Figure 2 This is a schematic diagram illustrating a specific implementation of a visual inspection and production management system for destacking sheet metal status provided in this application embodiment. (Refer to...) Figure 2 The system may include: The acquisition module 21 is used to acquire the optical image set of the currently detected destacking sheet material and the resonance response data of the currently detected destacking sheet material under vibration excitation. Extraction module 22 is used to extract the contour point sequence from the optical image set, align the standard contour point sequence in the mold formula parameters corresponding to the current destacking sheet material to be detected with the contour point sequence, and generate the corresponding position determination result and grasping pose compensation value. Matching module 23 is used to match multiple feature points in the optical image set with standard feature points to generate the type determination result and orientation determination result of the current unstacked sheet material to be detected, generate the corresponding vibration response spectrum based on the resonance response data, and compare the vibration response spectrum with the standard spectrum template to obtain the corresponding patch status determination result. The generation module 24 is used to generate a status determination result based on the position determination result, the captured pose compensation value, the type determination result, the orientation determination result, and the patch status determination result corresponding to the current destacking sheet material to be detected. The processing module 25 is used to perform destacking operations on the current destacking sheet material based on the current state determination result of the destacking sheet material to be detected through the robot component, and after all the destacking operations of the current batch of destacking sheet materials are completed, update the corresponding mold formula parameters based on the state determination results of multiple destacking sheet materials to be detected in the current batch and the pressure drop rate during the corresponding destacking process.
[0132] The visual inspection and production management system for destacking the status of sheet metal in this application is used to implement the aforementioned visual inspection and production management method for destacking the status of sheet metal. Therefore, the specific implementation of the visual inspection and production management system for destacking the status of sheet metal can be found in the embodiment section of the visual inspection and production management method for destacking the status of sheet metal described above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.
[0133] This application also provides an electronic device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are to be invoked and executed by the processing component to implement the steps of the visual inspection and production management method for destacking sheet metal status as described above.
[0134] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the steps of the visual inspection and production management method for destacking sheet metal status described above.
[0135] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.
[0136] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the visual inspection and production management method for destacking sheet metal status as described above.
[0137] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0138] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.
[0139] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A visual inspection and production management method for the destacking status of sheet metal, characterized in that, include: Acquire the optical image set of the destacking sheet material to be detected, and the resonance response data of the destacking sheet material to be detected under vibration excitation. The contour point sequence is extracted from the optical image set, and the standard contour point sequence in the mold formula parameters corresponding to the current destacking sheet material to be detected is aligned with the contour point sequence to generate the corresponding position determination result and grasping pose compensation value. The optical image sets multiple feature points are matched with standard feature points to generate the type and orientation determination results of the destacking material to be detected. Based on the resonance response data, the corresponding vibration response spectrum is generated. The vibration response spectrum is compared with the standard spectrum template to obtain the corresponding patch status determination result. Based on the position determination result, grab pose compensation value, type determination result, orientation determination result and patch status determination result of the current destabilized sheet material to be detected, a status determination result is generated. Based on the current status determination result of the destacking sheet material to be inspected, the robot component performs the destacking operation on the current destacking sheet material to be inspected. After all the destacking operations of the current batch of destacking sheet materials to be inspected are completed, the corresponding mold formula parameters are updated based on the status determination results of multiple destacking sheet materials to be inspected in the current batch and the pressure drop rate during the corresponding destacking process.
2. The method according to claim 1, characterized in that, Align the standard contour point sequence in the mold formula parameters corresponding to the currently detected destacking sheet material with the contour point sequence to generate the corresponding position determination result and grasping pose compensation value, including: The point set rigid registration algorithm is used to align the contour point sequence with the standard contour point sequence in the mold formula parameters corresponding to the current destacking material to be detected, so as to generate the target rotation amount and target translation amount of the current destacking material to be detected. Based on the target rotation and target translation, the contour point sequence is spatially transformed to obtain the transformed contour point sequence. The transformed contour point sequence is then compared with the standard contour point sequence to calculate the contour overlap ratio. The target rotation amount, target translation amount, and contour overlap ratio of the current destacking sheet material to be detected are compared with the angle tolerance, position tolerance, and preset ratio threshold in the corresponding mold formula parameters to generate the corresponding position determination result. Based on the target rotation and target translation, the coordinates of each standard gripping point in the corresponding mold formula parameters are spatially transformed to obtain multiple transformed coordinates. The transformed coordinates are then integrated to obtain the corresponding gripping pose compensation value.
3. The method according to claim 2, characterized in that, The point set rigid registration algorithm is used to align the contour point sequence with the standard contour point sequence in the mold formula parameters corresponding to the current destacking material to be detected, so as to generate the target rotation and target translation of the current destacking material to be detected, including: Calculate the Euclidean distance between the pixel coordinates of each contour point in the contour point sequence and the pixel coordinates of each standard contour point in the standard contour point sequence. Based on the Euclidean distance between each contour point and each standard contour point, match each contour point with each standard contour point to obtain multiple contour point pairs. The rotation and translation amounts that minimize the sum of squared Euclidean distances between each pair of contour points are iteratively solved until the decrease in the sum of squared Euclidean distances is less than a preset convergence threshold. The rotation and translation amounts solved when the decrease in the sum of squared Euclidean distances is less than the preset convergence threshold are respectively taken as the target rotation amount and the target translation amount.
4. The method according to claim 1, characterized in that, The optical image set is matched with standard feature points to generate a type determination result and orientation determination result for the current unstacked sheet material to be detected, including: Select a target sheet image that meets the preset image quality conditions from the optical image set, and perform feature detection on the target sheet image to obtain multiple feature points; Extract the local image block corresponding to each feature point from the target sheet image, and calculate the gray level difference between each local image block and the corresponding template image block to obtain the sum of the absolute values of the gray level difference corresponding to each feature point. The template image block is the image block corresponding to each standard feature point in the standard feature point template. Feature points whose sum of absolute grayscale differences is less than a preset difference threshold are marked as matching point pairs with their corresponding standard feature points. The total number of matching point pairs is counted, and combined with the preset number of matching pairs, the type determination result of the current unstacked sheet material to be detected is generated. Based on the orientation of each standard feature point in each matching point pair, the orientation determination result of the current destacking material to be detected is generated.
5. The method according to claim 1, characterized in that, Based on the resonant response data, a corresponding vibration response spectrum is generated, including: The vibration acceleration spectrum is obtained by performing a time-frequency transformation on the resonant response data; Based on the first frequency range and the second frequency range in the mold formula parameters corresponding to the destacking material to be tested, the vibration acceleration spectrum is divided into a first frequency band and a second frequency band. The first frequency range is determined based on the bending vibration characteristics of the destacking material, and the second frequency range is based on the local vibration characteristics of the patch. Extract the first target frequency corresponding to the peak acceleration amplitude and the first phase value corresponding to the first target frequency from the first frequency band, and generate patch vibration characteristic parameters based on the acceleration amplitude in the second frequency band; The first target frequency, the first phase value, and the patch vibration characteristic parameters are integrated to generate a vibration response spectrum.
6. The method according to claim 5, characterized in that, Based on the acceleration amplitude in the second frequency band, patch vibration characteristic parameters are generated, including: Extract the second target frequency corresponding to the peak acceleration amplitude and the second phase value corresponding to the second target frequency from the second frequency band; Calculate the ratio of the sum of acceleration amplitudes in the second frequency band to the sum of acceleration amplitudes in the first frequency band, and count the number of target spectral peaks in the second frequency band whose corresponding acceleration amplitudes exceed a preset amplitude threshold. The frequency corresponding to each target spectral peak is matched with the standard frequency point set in the mold formula parameters corresponding to the current destacking sheet material to be tested, so as to determine the target standard frequency corresponding to each target spectral peak. Based on the frequency values of each target spectral peak, the frequency distribution range is determined, and the frequency offset between the frequency value of each target spectral peak and the corresponding target standard frequency is calculated. The second target frequency, the second phase value, the ratio, the number of spectral peaks of the target spectral peak, the frequency distribution range, and each frequency offset are integrated to obtain the patch vibration characteristic parameters.
7. The method according to claim 1, characterized in that, Extracting a sequence of contour points from the optical image set includes: Edge pixel detection and filtering are performed on each optical image in the optical image set to obtain an edge pixel set; Extract the set of edge pixels that satisfy the closure condition from the set of edge pixels, and take the set of edge pixels with the largest number of edge pixels as the contour point sequence.
8. A visual inspection and production management system for the destacking status of sheet metal, characterized in that, include: The acquisition module is used to acquire the optical image set of the currently detected destacking sheet material and the resonance response data of the currently detected destacking sheet material under vibration excitation. The extraction module is used to extract the contour point sequence from the optical image set, align the standard contour point sequence in the mold formula parameters corresponding to the destacking material to be detected with the contour point sequence, and generate the corresponding position determination result and grasping pose compensation value. The matching module is used to match multiple feature points in the optical image set with standard feature points to generate the type determination result and orientation determination result of the current unstacked sheet material to be detected. Based on the resonance response data, the corresponding vibration response spectrum is generated. The vibration response spectrum is compared with the standard spectrum template to obtain the corresponding patch status determination result. The generation module is used to generate status determination results based on the position determination results, grasped pose compensation values, type determination results, orientation determination results, and patch status determination results of the current destacking sheet material to be detected. The processing module is used to perform destacking operations on the current destacking sheet material based on the current status determination result of the destacking sheet material to be detected through the robot component, and after all the destacking operations of the current batch of destacking sheet materials are completed, update the corresponding mold formula parameters based on the status determination results of multiple destacking sheet materials to be detected in the current batch and the pressure drop rate during the corresponding destacking process.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute a visual inspection and production management method for destacking sheet metal as described in any one of claims 1-7.
10. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform a visual inspection and production management method for destacking sheet metal status as described in any one of claims 1-7.
