Welding spot detection method and system based on cooperation of double manipulators

Abstract

The invention provides a welding spot detection method and system based on cooperation of double manipulators, and relates to the technical field of welding spot detection, and the method comprises the steps: determining second welding spot parameter information according to a to-be-detected piece mathematical model and first welding spot parameter information, and determining welding spot feature information; according to the welding spot feature information, adjacent welding spots with similar geometric features are combined to determine a detection area and isolated welding spots, the mass center of the detection area is determined, and the mass center and the isolated welding spots are defined as path control points; determining a node value and a primary function corresponding to the node value according to the path control point, and determining a path point coordinate according to the primary function, the path control point and the control point weight; optimizing the path point coordinates to determine a first track, and correcting the first track to determine a second track; and controlling one manipulator to clamp the to-be-detected piece and enabling the other manipulator to move around the to-be-detected piece along a second track so as to detect the welding spot. The problem that a welding spot detection method high in welding detection precision and efficiency is lacked in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of solder joint detection, and particularly to a solder joint detection method and system based on the cooperation of dual manipulators. Background Art

[0002] In some occasions of sheet metal part processing, it is necessary to fix the sheet metal part to other parts, and the fixing method mostly adopts welding. When the welding is completed, it is necessary to detect the size and shape of the solder joints to ensure the welding quality.

[0003] In the prior art, to achieve automated production and manufacturing, a manipulator is usually used to carry a detection device to detect the solder joints on the sheet metal part. Usually, a corresponding detection path is generated based on the theoretical model of the sheet metal part, and detection devices such as infrared, ray or ultrasonic with a fixed step size are used to perform scanning detection. However, when the distance between solder joints is large, scanning is performed in sequence along a fixed path, resulting in many idle strokes and affecting the detection efficiency. When the distance between solder joints is small, the manipulator needs to move frequently, and the jitter during the movement of the manipulator affects the scanning detection accuracy, thereby affecting the solder joint detection accuracy. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a solder joint detection method based on the cooperation of dual manipulators, aiming to solve the problem that there is a lack of a solder joint detection method with high welding detection accuracy and efficiency in the prior art.

[0005] According to an embodiment of the present invention, a solder joint detection method based on the cooperation of dual manipulators, the method includes: Receiving the digital model of the workpiece to be measured and the first solder joint parameter information, to determine the second solder joint parameter information according to the digital model of the workpiece to be measured and the first solder joint parameter information, and combining the first solder joint parameter information and the second solder joint parameter information to determine the solder joint feature information; Determining the detection area and isolated solder joints by combining adjacent solder joints with similar geometric features according to the solder joint feature information, and determining the centroid of the detection area and defining the centroid and the isolated solder joints as path control points; Determining the node value and the basis function corresponding to the node value according to the path control points, to determine the path point coordinates according to the basis function, the path control points and the control point weights; Optimizing the path point coordinates to determine the first trajectory to optimize the time target and energy target of the first trajectory, and correcting the first trajectory to determine the second trajectory to ensure that the second trajectory does not collide with the workpiece to be measured or the environment; Controlling one manipulator to clamp the workpiece to be measured, and controlling the other manipulator to move around the workpiece to be measured according to the second trajectory, so that the detection device on the manipulator detects the solder joints on the workpiece to be measured.

[0006] In addition, a solder joint detection method based on the cooperation of two manipulators according to the above embodiments of the present invention may further have the following additional technical features: Further, the first solder joint parameter information includes a solder joint coordinate matrix, and the step of determining the second solder joint parameter information according to the digital model of the workpiece to be measured and the first solder joint parameter information includes: Converting the digital model of the workpiece to be measured into a triangular mesh; Calculating the local curvature radius of the surface where each solder joint is located through the change rate of the normal vectors of adjacent triangular patches; Determining the material thickness corresponding to each solder joint area according to the digital model of the workpiece to be measured and the solder joint coordinate matrix, and the local curvature radius and the material thickness are the second solder joint parameters.

[0007] Further, the step of determining the detection area and the isolated solder joints by merging adjacent solder joints with similar geometric features according to the solder joint feature information includes: Determining the target distance between each solder joint and other solder joints according to a preset distance formula, and regarding the solder joints with the target distance less than the first preset value as neighbor points; Setting the solder joints with the number of neighbor points greater than the second preset value as core points, and merging the neighbor points corresponding to the core points starting from the core points to form the detection area, and the solder joints located outside the detection area are the isolated solder joints; The preset distance formula is:

[0008] where is the difference between the coordinates of two solder joints, is the difference between the corresponding curvature radii of two solder joints, is the difference between the corresponding thicknesses of two solder joints, is the weight parameter.

[0009] Further, the step of determining the centroid of the detection area and defining the centroid and the isolated solder joints as path control points to determine the path point coordinates according to the basis function, the path control points and the control point weights includes: Calculating the centroid of the detection area through a preset centroid formula, and defining the centroid and the isolated solder joints as path control points; Determining the knot vector and the corresponding knot values according to the path control points through a preset parameter formula; Recursively calculating the third-order NURBS basis function corresponding to the knot values; Determining the path point coordinates according to the basis function, the path control points and the control point weights through a preset trajectory point formula; The preset centroid formula is:

[0010] Among them, is the coordinate of the path control point; The preset parameter formula is:

[0011] is the value of the th node, is the coordinate of the path control point corresponding to the th node, is the Euclidean distance between the th node and the th node of adjacent control points, is the sum of the Euclidean distances between all adjacent control points; The preset trajectory point formula is:

[0012] Among them, is the basis function corresponding to the node value, is the weight parameter, is the control point coordinate.

[0013] Furthermore, the steps of optimizing the path point coordinates to determine the first trajectory to optimize the time target and energy target of the first trajectory include: Randomly sorting the path point coordinates to determine a first preset number of candidate paths and the energy target and time target corresponding to the candidate paths; Performing non-dominated sorting on the candidate paths according to the time target and the energy target to determine the levels corresponding to the candidate paths; Determining high-quality paths according to the levels and crowding degrees corresponding to the candidate paths, and then performing path crossover on the high-quality paths through a preset crossover rate to determine offspring paths; Fine-tuning the high-quality paths and the offspring paths according to a preset mutation rate and screening according to the levels and crowding degrees to determine a second preset number of first trajectories.

[0014] Furthermore, the steps of correcting the first trajectory to determine the second trajectory include: Performing hierarchical bounding and detection on the first trajectory to determine whether the distance between the trajectory points in the first trajectory and the digital mock-up of the part to be measured is less than a preset distance; If so, determining a local obstacle avoidance trajectory according to the first trajectory and the digital mock-up of the part to be measured to adjust the first trajectory, and screening the adjusted first trajectory to determine the second trajectory.

[0015] Further, the steps for the detection device on the robot arm to detect the solder joints on the workpiece to be measured include: Judge the type of the target to be measured. If the target to be measured is an isolated solder joint, determine a bow-shaped area with a preset radius centered on the isolated solder joint, and determine the corresponding real-time step length according to the real-time curvature of the bow-shaped area according to the preset step length calculation formula, so as to perform scanning detection on the bow-shaped area according to the real-time step length and the preset speed; If the target to be measured is the detection area, perform scanning detection on the detection area according to the real-time step length and the preset speed; The preset step length calculation formula is:

[0016] where R is the real-time curvature, is the real-time step length.

[0017] Another object of the embodiments of the present invention is to provide a solder joint detection system based on the cooperation of two robot arms, and the system includes: A solder joint feature information determination module, configured to receive the digital model of the workpiece to be measured and the first solder joint parameter information, so as to determine the second solder joint parameter information according to the digital model of the workpiece to be measured and the first solder joint parameter information, and merge the first solder joint parameter information and the second solder joint parameter information to determine the solder joint feature information; A path control point determination module, configured to determine the detection area and the isolated solder joints by merging adjacent solder joints with similar geometric features according to the solder joint feature information, and determine the centroid of the detection area and define the centroid and the isolated solder joints as path control points; A path point coordinate determination module, configured to determine the node value and the basis function corresponding to the node value according to the path control points, so as to determine the path point coordinates according to the basis function, the path control points and the control point weights; A second trajectory determination module, configured to optimize the path point coordinates to determine a first trajectory to optimize the time target and the energy target of the first trajectory, and correct the first trajectory to determine a second trajectory, so that the second trajectory does not collide with the workpiece to be measured or the environment; A detection module, configured to control one robot arm to clamp the workpiece to be measured, and control the other robot arm to move around the workpiece to be measured according to the second trajectory, so that the detection device on the robot arm detects the solder joints on the workpiece to be measured.

[0018] Another object of the embodiments of the present invention is to provide a storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned solder joint detection method based on the cooperation of two robot arms are implemented.

[0019] Another object of the embodiment of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned solder joint detection method based on the cooperation of dual manipulators are implemented.

[0020] In the present invention, according to the distribution of solder joints on the digital model of the workpiece to be measured, the solder joints are classified and merged, so that adjacent and relatively close points to be measured form a detection area, and other solder joints become isolated solder joints. Then, the centroids of each detection area and the isolated solder joints, that is, the path control points, which are the points that the manipulator must pass through, are sorted to determine multiple paths, and the multiple paths are iterated to determine the path with the optimal time target and energy target, that is, the time and energy consumed by adopting this path are less. Then, the path is adjusted according to the actual state to avoid the situation of collision interference between the manipulator stroke and the workpiece to be measured or the environment. Furthermore, the manipulator is controlled to move through the finally determined second trajectory. When the manipulator travels to the isolated solder joint, the galvanometer on the manipulator scans the area centered on the isolated solder joint to ensure the detection accuracy. When the manipulator travels to the detection area, the galvanometer rotates to scan the detection area. Furthermore, by merging the points with a small distance, when the manipulator moves, the distance between the two points is large, so that the manipulator can move with a large stroke and acceleration to improve the detection efficiency. For the points with a small distance, the galvanometer is controlled to detect an area, thus avoiding the influence of the vibration generated by the frequent movement of the manipulator on the detection accuracy. And the movement path of the manipulator is adjusted and optimized according to the actual solder joint distribution, the workpiece to be measured and the environmental conditions, further ensuring the movement effect of the manipulator. Furthermore, through the large-displacement positioning of the manipulator and the small-displacement fine scanning of the galvanometer, the efficiency and effect of solder joint detection are significantly improved. Therefore, the present invention solves the problem in the prior art that there is a lack of a solder joint detection method with high welding detection accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of the solder joint detection method based on the cooperation of dual manipulators in the first embodiment of the present invention; Figure 2 It is a structural block diagram of the solder joint detection system based on the cooperation of dual manipulators in the second embodiment of the present invention; Figure 3 It is a structural schematic diagram of the electronic device in the current regional embodiment of the present invention; The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0022] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0024] Embodiment 1 Please refer to Figure 1 , which shows the solder joint detection method based on the cooperation of double manipulators in the first embodiment of the present invention. This method specifically includes S01 - S05.

[0025] S01, receive the digital model of the workpiece to be measured and the first solder joint parameter information, determine the second solder joint parameter information according to the digital model of the workpiece to be measured and the first solder joint parameter information, and merge the first solder joint parameter information and the second solder joint parameter information to determine the solder joint feature information.

[0026] Specifically, the first solder joint parameter information includes a solder joint coordinate matrix. The step of determining the second solder joint parameter information according to the digital model of the workpiece to be measured and the first solder joint parameter information includes: converting the digital model of the workpiece to be measured into a triangular mesh; calculating the local curvature radius of each solder joint surface through the normal vector change rate of adjacent triangular patches; determining the material thickness corresponding to each solder joint area according to the digital model of the workpiece to be measured and the solder joint coordinate matrix. The local curvature radius and the material thickness are the second solder joint parameters. According to the existing input data, process the solder joint data and the workpiece data to be measured to determine the required solder joint feature information.

[0027] S02, determine the detection area and isolated solder joints by merging adjacent solder joints with similar geometric features according to the solder joint feature information, determine the centroid of the detection area, and define the centroid and the isolated solder joints as path control points.

[0028] Specifically, determine the target distance between each solder joint and other solder joints according to a preset distance formula, and regard the solder joints with the target distance less than the first preset value as neighbor points; set the solder joints with the number of neighbor points greater than the second preset value as core points, and merge the neighbor points corresponding to the core points starting from the core points to form the detection area. The solder joints located outside the detection area are the isolated solder joints; The preset distance formula is as follows:

[0029] where is the difference between the coordinates of two solder joints, is the difference between the corresponding curvature radii of two solder joints, is the difference between the corresponding thicknesses of two solder joints, is the weight parameter.

[0030] Specifically, by calculating the distance to determine whether two solder joints are physically adjacent, and then by calculating the curvature and thickness differences between the two solder joints to determine whether the two solder joints are in the same area and whether the detection parameters are consistent. Thus, through multi-dimensional judgment, it is determined whether it is appropriate to merge two solder joints into the same detection area, and then multiple welds are merged to simplify the movement path of the manipulator. Moreover, after the merger, when detecting the solder joints in the merged area, the parameters to be detected for the solder joints in the area are consistent or close, ensuring the efficiency and effect of detecting the solder joints in the area.

[0031] S03. Determine the node values and the basis functions corresponding to the node values according to the path control points, so as to determine the path point coordinates according to the basis functions, the path control points and the control point weights.

[0032] Specifically, calculate the centroid of the detection area through the preset centroid formula, and define the centroid and the isolated solder joint as path control points; determine the node vector and the corresponding node values according to the path control points through the preset parameter formula; recursively calculate the third-order NURBS basis functions corresponding to the node values; determine the path point coordinates according to the basis functions, the path control points and the control point weights through the preset trajectory point formula; The preset centroid formula is as follows:

[0033] where is the path control point coordinate; The preset parameter formula is as follows:

[0034] is the value of the th node, is the path control point coordinate corresponding to the th node, is the Euclidean distance between the th node and the th node of adjacent control points, is the sum of the Euclidean distances between all adjacent control points; The preset trajectory point formula is as follows:

[0035] Among them, is the basis function corresponding to the node value, is the weight parameter, is the coordinate of the control point.

[0036] Specifically, the best point for scanning and detecting the detection area is determined by calculating the centroid of the detection area, that is, it is easier to scan and detect other solder joints at the centroid. Furthermore, each centroid and isolated solder joint are the necessary points on the path. Then, by determining the node value and the basis function, the influence weight of each necessary point on the path shape is determined. Furthermore, by combining the basis function and the coordinates of the necessary points on the path, the corresponding path point coordinates are generated to determine a smooth and efficient path. Additionally, it is necessary to optimize and verify the path formed by the path points. By calculating the curvature of the path points to determine whether it is less than the allowable value of the manipulator curvature to ensure that the motion constraints of the manipulator are satisfied. And by detecting whether the derivatives of adjacent path points are connected to ensure the smoothness of the trajectory.

[0037] S04, optimize the path point coordinates to determine the first trajectory to optimize the time target and energy target of the first trajectory, and correct the first trajectory to determine the second trajectory so that the second trajectory does not collide with the workpiece to be measured or the environment.

[0038] Specifically, randomly sort the path point coordinates to determine a first preset number of candidate paths and the corresponding energy targets and time targets of the candidate paths; perform non-dominated sorting on the candidate paths according to the time targets and the energy targets to determine the levels corresponding to the candidate paths; determine high-quality paths according to the levels and crowding degrees corresponding to the candidate paths, and then perform path crossover on the high-quality paths through a preset crossover rate to determine offspring paths; perform fine-tuning on the high-quality paths and the offspring paths according to a preset mutation rate and perform screening according to levels and crowding degrees to determine a second preset number of first trajectories. In specific implementation, randomly sort all the path point coordinates to determine a large number of candidate paths, and then perform non-dominated sorting on the candidate paths according to two indicators of time targets and energy targets. The domination relationship is: Path one is better than path two if and only if path one is not worse than path two in terms of both time consumption and energy consumption, and is better in at least one target. The hierarchical division is as follows: The first layer is the individuals not dominated by any other paths. The second layer is the individuals only dominated by the individuals in the first layer. And so on for the remaining layers. Then screen the paths according to the levels. When the levels are the same, determine the paths according to the crowding degree, that is, when path one and path two are in the same layer, select the individual with a sparser distribution in the target space. Then perform crossover on the selected high-quality paths, that is, randomly select two crossover points for segmentation and exchange the middle segments after segmentation to obtain a new path, namely the offspring path. Then randomly select a control point in the determined offspring paths and excellent paths to add random perturbations to fine-tune the paths, so as to avoid the algorithm falling into local optimality and increase diversity. Finally, screen the fine-tuned paths according to levels and crowding degrees to determine the first trajectories with the target quantity.

[0039] Further, perform hierarchical enclosing and detection on the first trajectory to determine whether the distance between the trajectory points in the first trajectory and the digital mock-up of the workpiece to be measured is less than a preset distance; if so, determine a local obstacle avoidance trajectory according to the first trajectory and the digital mock-up of the workpiece to be measured to adjust the first trajectory, and perform screening on the adjusted first trajectory to determine the second trajectory. To avoid collisions between the manipulator, the workpiece to be measured and the environment, it is also necessary to detect the path and then adjust the local path according to the detection result to achieve local obstacle avoidance and detouring. In specific implementation, the Bezier obstacle avoidance trajectory generation method can be used to generate the local obstacle avoidance trajectory.

[0040] S05, control a manipulator to clamp the workpiece to be measured, and control another manipulator to move around the workpiece to be measured according to the second trajectory, so that the detection device on the manipulator detects the solder joints on the workpiece to be measured.

[0041] Specifically, when the optimal motion trajectory is determined, clamp the workpiece to be measured through the clamping manipulator, and notify another detection manipulator to perform motion detection around the workpiece to be measured according to the motion trajectory.

[0042] Further, the steps of the detection device on the manipulator for detecting the solder joints on the workpiece to be measured include: Judging the type of the target to be measured. If the target to be measured is an isolated solder joint, a bow-shaped area with a preset radius is determined with the isolated solder joint as the center, and then the corresponding real-time step length is determined according to the real-time curvature and the preset step length calculation formula of the bow-shaped area, so as to perform scanning detection on the bow-shaped area according to the real-time step length and the preset speed; if the target to be measured is the detection area, perform scanning detection on the detection area according to the real-time step length and the preset speed; The preset step length calculation formula is:

[0043] where R is the real-time curvature, is the real-time step length.

[0044] Specifically, when the manipulator moves to the preset position according to the optimized path, the scanning path of the galvanometer is adjusted according to the target to be detected corresponding to the preset position, and then through the large displacement positioning of the manipulator and the small displacement fine scanning of the galvanometer, the efficiency and effect of solder joint detection are significantly improved. And considering the influence of the curvature of different regions on the scanning effect, the real-time step length of the galvanometer is adjusted according to the actual curvature.

[0045] By way of example and not limitation, in some optional embodiments, the first solder joint parameter information further includes the solder joint normal vector. In specific implementation, the condition of the solder joint is detected by irradiating the solder joint with a laser, and then the laser correction angle can be adjusted through the solder joint normal vector to ensure that the laser is vertically incident, that is, to ensure the detection accuracy.

[0046] In addition, after the detection data is collected, the collected data is evaluated by a curve recognition model, and the moving path is adjusted for encrypted scanning for the area with a relatively large defect confidence to ensure the accuracy of the detection result. And a corresponding quality judgment report and a visualization model are generated.

[0047] In summary, the welding spot detection method based on the cooperation of two manipulators in the above-mentioned embodiment of the present invention, by classifying and merging the welding spots according to the distribution of the welding spots on the digital model of the test piece, so that the adjacent and closer test points constitute the detection area, and the other welding spots become isolated welding spots, and then sort the centroids of each detection area and the isolated welding spots, that is, the path control points, that is, the points that the manipulator must pass through, to determine multiple paths, and iterate the multiple paths to determine the path with the best time target and energy target, that is, the time and energy consumed by adopting this path are less. Then adjust the path according to the actual state to avoid the situation where the manipulator stroke collides and interferes with the test piece or the environment, and then control the manipulator to move through the second trajectory finally determined, so that when the manipulator moves to the isolated welding spot, the galvanometer on the manipulator performs regional scanning with the isolated welding spot as the center, thereby ensuring the detection accuracy, and when the manipulator moves to the detection area, the galvanometer rotates to scan the detection area. Furthermore, by merging points with smaller spacing, the distance between two points is larger when the manipulator moves, and the manipulator can move with a larger stroke and acceleration to improve the detection efficiency. For points with smaller spacing, the galvanometer is controlled to detect an area, thereby avoiding the influence of vibration caused by frequent movement of the manipulator on the detection accuracy. In addition, the movement path of the manipulator is adjusted and optimized according to the actual distribution of solder joints, the test pieces and the environmental conditions, so as to further ensure the movement effect of the manipulator. Furthermore, through the large displacement positioning of the manipulator and the small displacement precision scanning of the galvanometer, the efficiency and effect of solder joint detection are significantly improved. Therefore, the present invention solves the problem of the lack of a solder joint detection method with high welding detection accuracy and efficiency in the prior art.

[0048] Embodiment 2 See also Figure 2 , which is a structural block diagram of a welding spot detection system based on dual manipulator collaboration proposed in the second embodiment of the present invention, the welding spot detection system 200 based on dual manipulator collaboration includes: a welding spot feature information determination module 21, a path control point determination module 22, a path point coordinate determination module 23, a second trajectory determination module 24 and a detection module 25, wherein: The solder joint characteristic information determination module 21 is used to receive the digital model of the test piece and the first solder joint parameter information, determine the second solder joint parameter information according to the digital model of the test piece and the first solder joint parameter information, and combine the first solder joint parameter information and the second solder joint parameter information to determine the solder joint characteristic information; A path control point determination module 22, configured to determine the detection area and isolated welds by merging adjacent welds with similar geometric features according to the weld feature information, and to determine the centroid of the detection area and define the centroid and the isolated welds as path control points; A path point coordinate determination module 23, configured to determine node values and basis functions corresponding to the node values according to the path control points, so as to determine path point coordinates according to the basis functions, the path control points, and the control point weights; A second trajectory determination module 24, configured to optimize the path point coordinates to determine a first trajectory so that the time target and the energy target of the first trajectory are optimal, and correct the first trajectory to determine a second trajectory so that the second trajectory does not collide with the workpiece to be measured or the environment; A detection module 25, configured to control a manipulator to clamp the workpiece to be measured, and control another manipulator to move around the workpiece to be measured according to the second trajectory, so that a detection device on the manipulator detects the solder joints on the workpiece to be measured.

[0049] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments, and will not be elaborated here.

[0050] Embodiment III On the other hand, the present invention further provides an electronic device. Please refer to Figure 3 , which shows a schematic diagram of the electronic device in the current area embodiment of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, the solder joint detection method based on dual manipulator collaboration as described above is implemented.

[0051] Among them, in some embodiments, the processor 10 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips, and is used to run the program code stored in the memory 20 or process data, such as executing an access restriction program.

[0052] Among them, the memory 20 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disc, etc. The memory 20 may be an internal storage unit of the electronic device in some embodiments, such as the hard disk of the electronic device. The memory 20 may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 20 may include both the internal storage unit and the external storage device of the electronic device. The memory 20 can not only be used to store application software and various data of the electronic device, but also be used to temporarily store data that has been output or will be output.

[0053] It should be noted that Figure 3 the structures shown do not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than shown in the figures, or combine certain components, or have a different component arrangement.

[0054] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the solder joint detection method based on the cooperation of dual manipulators as described above.

[0055] Those skilled in the art can understand that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device.

[0056] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0057] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0058] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0059] The above embodiments only represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

Claims

1. A solder joint detection method based on the cooperation of double manipulators, characterized in that The method includes: Receiving the digital-analog model of the workpiece to be measured and the first solder joint parameter information, determining the second solder joint parameter information according to the digital-analog model of the workpiece to be measured and the first solder joint parameter information, and combining the first solder joint parameter information and the second solder joint parameter information to determine the solder joint feature information; Determining the detection area and isolated solder joints by combining adjacent solder joints with similar geometric features according to the solder joint feature information, determining the centroid of the detection area, and defining the centroid and the isolated solder joints as path control points; Determining the node values and the basis functions corresponding to the node values according to the path control points, and determining the path point coordinates according to the basis functions, the path control points and the control point weights; Optimizing the path point coordinates to determine a first trajectory to optimize the time target and energy target of the first trajectory, and correcting the first trajectory to determine a second trajectory so that the second trajectory does not collide with the workpiece to be measured or the environment; Controlling a manipulator to clamp the workpiece to be measured, and controlling another manipulator to move around the workpiece to be measured according to the second trajectory, so that the detection device on the manipulator detects the solder joints on the workpiece to be measured.

2. The solder joint detection method based on the cooperation of dual manipulators according to claim 1, wherein The first solder joint parameter information includes a solder joint coordinate matrix. The step of determining the second solder joint parameter information according to the digital-analog model of the workpiece to be measured and the first solder joint parameter information includes: Converting the digital-analog model of the workpiece to be measured into a triangular mesh; Calculating the local curvature radius of the surface where each solder joint is located through the change rate of the normal vectors of adjacent triangular patches; Determining the material thickness corresponding to each solder joint area according to the digital-analog model of the workpiece to be measured and the solder joint coordinate matrix, and the local curvature radius and the material thickness are the second solder joint parameters.

3. The solder joint detection method based on the cooperation of dual manipulators according to claim 2, wherein The step of determining the detection area and isolated solder joints by combining adjacent solder joints with similar geometric features according to the solder joint feature information includes: Determining the target distance between each solder joint and other solder joints according to a preset distance formula, and regarding the solder joints with the target distance less than a first preset value as neighbor points; Setting the solder joints with the number of neighbor points greater than a second preset value as core points, and merging the neighbor points corresponding to the core points starting from the core points to form the detection area, and the solder joints located outside the detection area are the isolated solder joints; The preset distance formula is: Among them, is the difference between the coordinates of two solder joints, is the difference between the corresponding curvature radii of two solder joints, is the difference between the corresponding thicknesses of two solder joints, is the weight parameter.

4. The solder joint detection method based on the cooperation of double manipulators according to claim 1, characterized in that Determining the centroid of the detection area and defining the centroid and the isolated solder joints as path control points. The step of determining the path point coordinates according to the basis functions, the path control points and the control point weights includes: Calculating the centroid of the detection area through a preset centroid formula, and defining the centroid and the isolated solder joints as path control points; Determining the node vector and the corresponding node values according to the path control points through a preset parameter formula; Recursively calculating the third-order NURBS basis functions corresponding to the node values; Determining the path point coordinates according to the basis functions, the path control points and the control point weights through a preset trajectory point formula; The preset centroid formula is: Among them, are the coordinates of the path control points; The preset parameter formula is: is the value of the th node, is the coordinate of the path control point corresponding to the th node, is the Euclidean distance between the th node and the th node of adjacent control points, is the sum of the Euclidean distances between all adjacent control points; The preset trajectory point formula is: Among them, is the basis function corresponding to the node value, is the weight parameter, is the control point coordinate.

5. The solder joint detection method based on the cooperation of dual manipulators according to claim 4, wherein, The step of optimizing the path point coordinates to determine a first trajectory to optimize the time target and energy target of the first trajectory includes: Randomly sort the coordinates of the path points to determine a first preset number of candidate paths and the corresponding energy targets and time targets for the candidate paths; Perform non-dominated sorting on the candidate paths according to the time targets and the energy targets to determine the levels corresponding to the candidate paths; Determine high-quality paths according to the levels and crowding degrees corresponding to the candidate paths, and then perform path crossover on the high-quality paths according to a preset crossover rate to determine offspring paths; Fine-tune the high-quality paths and the offspring paths according to a preset mutation rate and perform screening according to the levels and crowding degrees to determine a second preset number of first trajectories.

6. The solder joint detection method based on the cooperation of double manipulators according to claim 5, wherein, The steps of correcting the first trajectory to determine the second trajectory include: Perform hierarchical bounding and detection on the first trajectory to determine whether the distance between the trajectory points in the first trajectory and the digital model of the workpiece to be measured is less than a preset distance; If so, determine a local obstacle avoidance trajectory according to the first trajectory and the digital model of the workpiece to be measured to adjust the first trajectory, and perform screening on the adjusted first trajectory to determine the second trajectory.

7. The solder joint detection method based on the cooperation of dual manipulators according to claim 2, wherein The steps of the detection device on the robot arm detecting the solder joints on the workpiece to be measured include: Judge the type of the measurement target. If the measurement target is an isolated solder joint, determine a bow-shaped area with a preset radius centered on the isolated solder joint, and determine the corresponding real-time step length according to the real-time curvature of the bow-shaped area according to a preset step length calculation formula, so as to perform scanning detection on the bow-shaped area according to the real-time step length and a preset speed; If the measurement target is the detection area, perform scanning detection on the detection area according to the real-time step length and a preset speed; The preset step length calculation formula is: where R is the real-time curvature, is the real-time step size.

8. A solder joint detection system based on the cooperation of dual manipulators, characterized in that, A system for implementing the solder joint detection method based on the cooperation of two robot arms according to any one of claims 1 to 7, the system includes: A solder joint feature information determination module, configured to receive the digital model of the workpiece to be measured and first solder joint parameter information, determine second solder joint parameter information according to the digital model of the workpiece to be measured and the first solder joint parameter information, and combine the first solder joint parameter information and the second solder joint parameter information to determine solder joint feature information; A path control point determination module, configured to determine a detection area and isolated solder joints by merging adjacent solder joints with similar geometric features according to the solder joint feature information, determine the centroid of the detection area, and define the centroid and the isolated solder joints as path control points; A path point coordinate determination module, configured to determine a node value and a basis function corresponding to the node value according to the path control points, and determine path point coordinates according to the basis function, the path control points, and the control point weights; A second trajectory determination module, configured to optimize the path point coordinates to determine a first trajectory to optimize the time target and energy target of the first trajectory, and correct the first trajectory to determine a second trajectory so that the second trajectory does not collide with the workpiece to be measured or the environment; A detection module, configured to control a robot arm to hold the workpiece to be measured, and control another robot arm to move around the workpiece to be measured according to the second trajectory, so that the detection device on the robot arm detects the solder joints on the workpiece to be measured.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the solder joint detection method based on the cooperation of two manipulators according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the solder joint detection method based on the cooperation of two manipulators according to any one of claims 1 - 7.

Images