Mechanical arm surface monitoring method, device and equipment and storage medium
By acquiring and analyzing surface images in real time based on the motion state signals of the robotic arm, the problem of low efficiency in surface particle monitoring of robotic arms in existing technologies has been solved, achieving efficient and accurate particle detection and reducing product defects.
Patent Information
- Application Number
- CN202511793607.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-01-16
AI Technical Summary
In existing technologies, the monitoring of particles on the surface of robotic arms relies on regular manual inspections or offline detection, which is inefficient and cannot provide real-time feedback on the status of the production process. It is difficult to detect micron-sized particles, which affects production yield and product reliability.
By responding to the motion status signal of the robotic arm, it is determined whether it is in a preset monitoring position, and the imaging module is activated at that position to acquire images. The high-resolution camera and light source are used to acquire images, and combined with particle detection models and algorithms, the particle information on the surface of the robotic arm is detected in real time.
This technology enables efficient and accurate monitoring of particle conditions on the surface of the robotic arm without interrupting the production process, reducing product defects caused by particle contamination and improving the accuracy and efficiency of monitoring during production.
Smart Images
Figure CN121340366A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for surface monitoring of a robotic arm. Background Technology
[0002] In fields with extremely high requirements for environmental cleanliness, such as semiconductor manufacturing, precision electronic assembly, and biomedicine, robotic arms, as core equipment for automated operations, directly impact product quality due to the cleanliness of their surfaces. If tiny particles adhere to the surface of the robotic arm, these particles may transfer to the workpiece surface during gripping and handling, leading to problems such as chip short circuits, electronic component failures, and biological sample contamination, severely affecting production yield and product reliability.
[0003] In some scenarios, cameras are installed in the module assembly machine for monitoring. However, these cameras are mainly used to monitor the vibration of the robotic arm and the current state of the workpiece being handled by the robotic arm, and cannot determine the condition of particles on the robotic arm. Current methods for monitoring particles on the surface of robotic arms mainly rely on periodic manual inspections or offline monitoring equipment. However, manual inspection is inefficient, highly subjective, and difficult to capture micron-sized particles; while offline inspection cannot provide feedback on the surface condition of the robotic arm during production.
[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention
[0005] The main objective of this application is to provide a method, apparatus, device, and storage medium for monitoring the surface of a robotic arm, aiming to improve the accuracy and efficiency of monitoring particles on the surface of the robotic arm during the production process.
[0006] To achieve the above objectives, this application proposes a method for surface monitoring of a robotic arm, the method comprising: In response to the motion status signal of the robotic arm, determine whether the robotic arm is in a preset monitoring position; When the robotic arm is in a preset monitoring position, a preset imaging module is activated to acquire the image to be monitored on the surface of the robotic arm; Particle detection is performed on the image to be monitored to obtain particle information on the surface of the robotic arm.
[0007] In one embodiment, the motion state signal includes a motion command feedback signal and / or a position feedback signal; The step of determining whether the robotic arm is in a preset monitoring position in response to the motion state signal of the robotic arm includes: The action command feedback signal is parsed to obtain the command parsing result; The position feedback signal is analyzed to obtain the current position of the robotic arm; Based on the instruction parsing result and / or the current position of the robotic arm, determine whether the robotic arm is in a preset monitoring position.
[0008] In one embodiment, the instruction parsing result includes instruction type and execution status. Determining whether the robotic arm is at a preset monitoring position based on the instruction parsing result includes: Based on the instruction type, the preset instruction type, and the execution state, it is determined whether the robotic arm is in a preset monitoring position. The preset instruction type includes a first instruction type, a second instruction type, and a third instruction type. The first instruction type refers to the instruction to control the robotic arm to move to a preset first position to grasp the target object to be processed. The second instruction type refers to the instruction to control the robotic arm to move to a preset second position to transfer the target object to the process equipment for processing. The third instruction type refers to the instruction to control the robotic arm to move to a preset third position to place the target object after the processing is completed.
[0009] In one embodiment, the instruction parsing result includes a destination location and an execution status. The destination location is a pre-set location that the robotic arm needs to reach. Determining whether the robotic arm is in a preset monitoring position based on the instruction parsing result includes: Based on the target location and the execution state, determine whether the robotic arm is in a preset monitoring position.
[0010] In one embodiment, activating a preset imaging module when the robotic arm is at a preset monitoring position includes: Determine the duration of the robotic arm's stationary state; When the robotic arm is in a preset monitoring position and the static duration exceeds a preset static duration threshold, the imaging module is activated.
[0011] In one embodiment, after performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm, the method further includes: Record monitoring data, wherein the monitoring data includes monitoring time and particle information; Based on multiple recorded monitoring data, the particle change trend on the surface of the robotic arm and the particle distribution on the surface of the robotic arm are analyzed in order to determine the particle source based on the particle distribution.
[0012] In one embodiment, the particle information includes particle size and particle quantity; After performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm, the method further includes: When the particle size is greater than a preset particle size threshold, or the number of particles is greater than a preset particle number threshold, it is determined that the robotic arm is contaminated with particles. When the particle size is less than or equal to the preset particle size threshold and the particle number is less than or equal to the preset particle number threshold, it is determined that the robotic arm is free of particle contamination.
[0013] In one embodiment, the step of performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm includes: The image to be monitored is binarized according to a preset segmentation threshold to obtain a binarized image; Morphological recognition is performed on the binarized image to obtain the particle information; And / or, The particle detection model is used to detect particles in the image to be monitored to obtain particle information. The particle detection model is obtained by iterative training based on pre-collected image samples and particle labels associated with the image samples.
[0014] Furthermore, to achieve the above objectives, this application also proposes a surface monitoring device for a robotic arm, the surface monitoring device comprising: The determination module is used to determine whether the robotic arm is in a preset monitoring position in response to the motion state signal of the robotic arm; The trigger module is used to activate the preset imaging module when the robotic arm is in the preset monitoring position, so as to acquire the image to be monitored on the surface of the robotic arm; The image processing module is used to perform particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm.
[0015] In addition, to achieve the above objectives, this application also proposes a robotic arm surface monitoring device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the robotic arm surface monitoring method described above.
[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the robotic arm surface monitoring method described above.
[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the robotic arm surface monitoring method described above.
[0018] This application provides a method, apparatus, device, and storage medium for monitoring the surface of a robotic arm. The method includes: responding to a motion state signal of the robotic arm to determine whether the robotic arm is in a preset monitoring position; when the robotic arm is in the preset monitoring position, activating a preset imaging module to acquire an image of the robotic arm surface to be monitored; and performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm. This application, when combined with the motion state signal of the robotic arm and detecting that the robotic arm is in the preset monitoring position, activates a preset imaging module to acquire an image of the robotic arm surface to be monitored, and then analyzes the image to determine whether there is particle contamination on the robotic arm. This monitoring can be performed without interrupting the production process and without manual inspection, improving the accuracy and efficiency of particle monitoring on the surface of the robotic arm during production. Attached Figure Description
[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart illustrating an embodiment of the robotic arm surface monitoring method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the robotic arm surface monitoring method of this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the robotic arm surface monitoring method of this application; Figure 4 This is a flowchart illustrating Embodiment 4 of the robotic arm surface monitoring method of this application; Figure 5 This is a flowchart illustrating Embodiment 5 of the robotic arm surface monitoring method of this application. Figure 6 This is a flowchart illustrating Embodiment Six of the robotic arm surface monitoring method of this application; Figure 7 This is a schematic diagram of the module structure of the robotic arm surface monitoring device according to an embodiment of this application; Figure 8 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the robotic arm surface monitoring method in this application embodiment.
[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0025] The main solution of this application embodiment is as follows: In response to the motion state signal of the robotic arm, determine whether the robotic arm is in a preset monitoring position; when the robotic arm is in the preset monitoring position, activate a preset imaging module to acquire an image to be monitored on the surface of the robotic arm; perform particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm. When the motion state signal of the robotic arm indicates that the robotic arm is in the preset monitoring position, activate the preset imaging module to acquire an image to be monitored on the surface of the robotic arm, and then analyze the image to determine whether there is particle contamination on the robotic arm. This monitoring can be performed without interrupting the production process and without manual inspection, improving the accuracy and efficiency of particle monitoring on the surface of the robotic arm during production.
[0026] Current technologies for monitoring surface particles on robotic arms primarily rely on periodic manual inspections (e.g., visual observation, laboratory analysis after wiping) or offline detection equipment, which have the following shortcomings: (1) Manual detection is inefficient and subjective, making it difficult to detect micron-sized particles.
[0027] (2) Offline detection cannot provide feedback on the surface condition of the robotic arm during the production process.
[0028] (3) A camera is installed inside the module machine, but the camera can only monitor the shaking of the robotic arm and the state of the workpiece carried by the robotic arm, and cannot determine the state of the particles on the robotic arm.
[0029] This application provides a solution that improves the accuracy and efficiency of particle monitoring on the robotic arm surface during production by performing real-time and precise particle monitoring on the robotic arm without interrupting the production process or requiring manual inspection. This enables timely early warning of particle contamination and reduces product defects caused by particle contamination.
[0030] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device, big data service platform, particle monitoring system, etc., capable of realizing the above functions. The following description uses a particle monitoring system as an example to illustrate this embodiment and the subsequent embodiments.
[0031] Based on this, embodiments of this application provide a method for surface monitoring of a robotic arm, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the robotic arm surface monitoring method of this application.
[0032] In this embodiment, the defect mode detection method includes steps S11 to S13: Step S11: In response to the motion state signal of the robotic arm, determine whether the robotic arm is in a preset monitoring position; It should be noted that the preset monitoring location can be set according to the actual process conditions, such as the location where the target object is placed, the location where the target object is processed, or the location where the target object is stored after processing.
[0033] In one embodiment, the motion state signal includes an action command feedback signal. It should be noted that during the operation of the robotic arm, the control system of the robotic arm sends control commands to the robotic arm, such as movement commands, grasping commands, and placement commands. After completing the relevant commands, the robotic arm will return a corresponding response signal. In this embodiment, the particle monitoring system is communicatively connected to the control system of the robotic arm, thereby receiving the relevant action command feedback signals of the robotic arm, and then determining whether the robotic arm is in a preset monitoring position based on the action command feedback signals.
[0034] In one embodiment, the motion status signal includes a position feedback signal. Optionally, the robotic arm is equipped with a position sensor, which collects the position of the robotic arm and uploads it to the particle monitoring system. Based on the position feedback signal of the robotic arm, the current position of the robotic arm is determined, thereby determining whether the robotic arm is at a preset monitoring position.
[0035] In one embodiment, to improve the accuracy of position monitoring, the motion command feedback signal and the position feedback signal of the robotic arm can be combined to determine whether the robotic arm is in a preset monitoring position.
[0036] Step S12: When the robotic arm is in a preset monitoring position, a preset imaging module is activated to acquire the image to be monitored on the surface of the robotic arm. It should be noted that the imaging module includes at least one set of image acquisition devices and a light source for acquiring images of the surface of the robotic arm to be monitored. Optionally, the image acquisition device is an industrial camera, for example, a CMOS complementary metal-oxide-semiconductor image sensor with a resolution of not less than 20 million pixels, coupled with a macro lens, capable of capturing particles as small as 0.5 micrometers. The light source is used to provide illumination for the image acquisition devices. Optionally, the light source is an LED light source, specifically a multi-angle ring LED light source, for example, a combination of oblique and vertically direct LED light sources. In other embodiments, the angle of the oblique illumination can be set according to actual conditions, for example, 45° oblique illumination. The vertically direct light shines vertically downwards from the center of the ring light source onto the surface of the robotic arm, while the 45° oblique light shines obliquely onto the surface of the robotic arm from the side of the ring light source, thereby reducing surface reflection interference, ensuring the clarity of particle imaging, and improving the accuracy of particle detection.
[0037] In one embodiment, the imaging module is installed at a preset monitoring point on the movement trajectory of the robotic arm and / or on the end of the robotic arm, so that the range of image acquisition can cover the key surfaces of the robotic arm, such as the gripping end, arm body, joints and other surface areas.
[0038] It should be noted that the areas requiring particle monitoring should be pre-defined based on the robotic arm model and working scenario, such as the surface of the silicone suction cup at the gripping end, the metal shell of the arm body, etc.
[0039] In this embodiment, when the robotic arm is detected to be in a preset monitoring position, a preset imaging module is activated to acquire the image to be monitored on the surface of the robotic arm through the imaging module.
[0040] To avoid image blurring caused by robotic arm movement, in another embodiment, a static duration threshold is preset. When the robotic arm is in a preset monitoring position and the static duration of the robotic arm exceeds the preset static duration threshold, the imaging module is activated to acquire the image to be monitored on the surface of the robotic arm.
[0041] Step S13: Perform particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm.
[0042] It should be noted that particle information includes particle size, particle location, and particle quantity. In one embodiment, a pre-built particle detection model can be used to detect particles in the image to be monitored to obtain particle information on the surface of the robotic arm. In other embodiments, threshold segmentation algorithms and morphological recognition algorithms can also be combined to detect particles in the image to be monitored to obtain particle information on the surface of the robotic arm.
[0043] In one feasible implementation, after performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm, the method further includes: When the particle size is greater than a preset particle size threshold, or the number of particles is greater than a preset particle number threshold, it is determined that the robotic arm is contaminated with particles. When the particle size is less than or equal to the preset particle size threshold and the particle number is less than or equal to the preset particle number threshold, it is determined that the robotic arm is free of particle contamination.
[0044] It should be noted that there are pre-set judgment rules to determine whether there is particulate contamination in the robotic arm. The judgment rules can be set according to the actual situation. Optionally, the judgment rules are set as follows: the maximum diameter of the particles is allowed to be 1 micrometer and the number of particles in a single area is less than or equal to 3.
[0045] In this embodiment, the particle size is compared with a preset particle size threshold, and the particle number is compared with a preset particle number threshold. The preset particle size threshold and preset particle number threshold are set according to actual conditions. The preset particle size threshold is set to a particle diameter of 1 micrometer, and the preset particle number threshold is set to 3. When the particle size is greater than the preset particle size threshold, or the particle number is greater than the preset particle number threshold, it indicates that the particles on the surface of the robotic arm exceed the standard, thus determining that the robotic arm is contaminated with particles. When the particle size is less than or equal to the preset particle size threshold, and the particle number is less than or equal to the preset particle number threshold, it indicates that there are few or no particles on the surface of the robotic arm, thus determining that the robotic arm is not contaminated with particles.
[0046] This embodiment employs the above-described scheme, including: responding to the motion state signal of the robotic arm, determining whether the robotic arm is in a preset monitoring position; when the robotic arm is in the preset monitoring position, activating a preset imaging module to acquire a monitoring image of the robotic arm surface; and performing particle detection on the monitoring image to obtain particle information on the robotic arm surface. In this embodiment, when the robotic arm's motion state signal is combined with the detection that the robotic arm is in the preset monitoring position, the preset imaging module is activated to acquire a monitoring image of the robotic arm surface, and then the monitoring image is analyzed to determine whether particle contamination exists on the robotic arm. This monitoring is performed without interrupting the production process and without manual inspection, improving the accuracy and efficiency of particle monitoring on the robotic arm surface during production, thereby reducing product defects caused by particle contamination.
[0047] In one feasible implementation, refer to Figure 2 , Figure 2This is a flowchart illustrating Embodiment 2 of the robotic arm surface monitoring method of this application. In response to the motion state signal of the robotic arm, determining whether the robotic arm is at a preset monitoring position includes: Step S21: Analyze the action command feedback signal to obtain the command analysis result; Step S22: Analyze the position feedback signal to obtain the current position of the robotic arm; Step S23: Determine whether the robotic arm is in a preset monitoring position based on the instruction parsing result and / or the current position of the robotic arm.
[0048] It should be noted that the action command feedback signal is parsed to obtain command parsing results. These results include information such as command type and execution status. The execution status indicates whether the robotic arm has successfully executed the command. The command refers to the control command issued by the robotic arm's control system. These control commands include commands to move the robotic arm to a preset first position to grasp the target object to be processed, commands to move the robotic arm to a preset second position to transfer the target object to the processing equipment for processing, and commands to move the robotic arm to a preset third position to place the processed target object. In one embodiment, the robotic arm is determined to be at a preset monitoring position based on the command type and execution status. For example, if the command type is to move to point A containing the target object to grasp the target object to be processed, and the robotic arm's execution status is successful, it proves that the robotic arm has reached point A. If point A is a preset monitoring position, then the robotic arm can be determined to be at the preset monitoring position.
[0049] In one embodiment, the instruction parsing result may further include information such as the destination location and execution status. The destination location is the position that the robotic arm needs to reach, which is preset in the instruction. It should be noted that the meaning of the instruction has been explained in detail in the above embodiments, and will not be repeated here. Optionally, it is determined whether the destination location and the preset monitoring location match, and whether the robotic arm is at the preset monitoring location is determined based on the location determination result and the execution status.
[0050] In one embodiment, the position feedback signal is parsed to obtain the current position of the robotic arm; the current position of the robotic arm is compared with a preset monitoring position; if the current position of the robotic arm matches the preset monitoring position, it is determined that the robotic arm is in the preset monitoring position; if the current position of the robotic arm does not match the preset monitoring position, it is determined that the robotic arm is not in the preset monitoring position.
[0051] In one embodiment, the judgment can be made by combining the motion command feedback signal and the position feedback signal simultaneously to improve the accuracy of position judgment. More specifically: the motion command feedback signal is parsed to obtain the command parsing result, and then the position that the robot arm needs to reach in the pre-set command is determined based on the command parsing result. For example, if the command type in the command parsing result is a command to move to point A containing the target object to grab the target object, it proves that the position that the robot arm needs to reach is point A. In addition, the position feedback signal is also parsed to obtain the current position of the robot arm. Further, if the execution state is a successful state, it is determined whether the current position of the robot arm matches the position that the robot arm needs to reach in the pre-set command. If the position feedback signal parsing shows that the current position of the robot arm does not match the position that the robot arm needs to reach in the pre-set command parsing result, it proves that the robot arm has not moved to the location indicated by the command, and it is directly determined that the robot arm is not in the preset monitoring position. If the position feedback signal analysis shows that the current position of the robotic arm matches the pre-set position the robotic arm needs to reach in the instruction analysis result, then it is further determined whether the current position of the robotic arm matches the preset monitoring position. If the current position of the robotic arm matches the preset monitoring position, it is determined that the robotic arm is at the preset monitoring position; if it does not match the preset monitoring position, it is determined that the robotic arm is not at the preset monitoring position. In other embodiments, if the execution state is a failure state, it is directly determined that the robotic arm is not at the preset monitoring position. It should be noted that the meaning of the instructions has been explained in detail in the above embodiments, and will not be repeated here.
[0052] This embodiment accurately determines whether the robotic arm is in a preset monitoring position based on the command parsing results and / or the current position of the robotic arm, thereby enabling monitoring of the robotic arm during its operation intervals and improving the accuracy and efficiency of particle monitoring on the surface of the robotic arm during the production process.
[0053] In one feasible implementation, refer to Figure 3 , Figure 3 This is a flowchart illustrating Embodiment 3 of the robotic arm surface monitoring method of this application. The instruction parsing result includes the instruction type and execution status. Based on the instruction parsing result, determining whether the robotic arm is in a preset monitoring position includes: Step S31: Determine whether the robotic arm is in a preset monitoring position based on the instruction type, the preset instruction type, and the execution state.
[0054] It should be noted that the preset instruction types include a first instruction type, a second instruction type, and a third instruction type. The first instruction type refers to the instruction to control the robotic arm to move to a preset first position to grasp the target object to be processed. The second instruction type refers to the instruction to control the robotic arm to move to a preset second position to transfer the target object to the processing equipment for processing. The third instruction type refers to the instruction to control the robotic arm to move to a preset third position to place the target object after processing. Understandably, the first position refers to the position containing the target object to be processed, the second position refers to the position where the target object is processed, and the third position refers to the storage position of the processed target object. For example, in the semiconductor field, the first position could be the position of a wafer cassette containing wafers, the second position could be the position of the target equipment for processing wafers, and the third position could be the storage position of the processed wafer.
[0055] In one embodiment, it is determined whether the instruction type belongs to a preset instruction type. If the instruction type belongs to a preset instruction type and the execution status is successful, it proves that the robotic arm has moved to a preset position, that is, the first position, the second position, or the third position in this embodiment, and thus it is determined that the robotic arm is in a preset monitoring position. If the instruction type does not belong to a preset instruction type, or the execution status is a failure state, it proves that the robotic arm has not moved to the preset position, and thus it is determined that the robotic arm is not in a preset monitoring position.
[0056] This embodiment determines whether the robotic arm is in a preset monitoring position based on the instruction type, the preset instruction type, and the execution state, thereby enabling monitoring of the robotic arm during its operation intervals.
[0057] In one feasible implementation, refer to Figure 4 , Figure 4 This is a flowchart illustrating Embodiment 4 of the robotic arm surface monitoring method of this application. The instruction parsing result includes the destination position and execution status. The destination position is a pre-set position that the robotic arm needs to reach. Based on the instruction parsing result, determining whether the robotic arm is in the preset monitoring position includes: Step S41: Determine whether the robotic arm is in a preset monitoring position based on the target position and the execution state.
[0058] In one embodiment, if the target location matches the preset monitoring location and the execution status is successful, it proves that the robotic arm has moved to the target location and that the target location matches the preset monitoring location, thus determining that the robotic arm is at the preset monitoring location. If the target location and the preset monitoring location do not match, or the execution status is failed, it proves that the target location does not belong to the preset monitoring location, or that the robotic arm has not moved to the target location, thus determining that the robotic arm is not at the preset monitoring location. For example, if the target location is point A containing the target object, and the preset monitoring location is set to grasp the target object at point A, then the target location matches the preset monitoring location, and the robotic arm's execution status is successful, proving that the robotic arm has reached point A, thus determining that the robotic arm is at the preset monitoring location.
[0059] This embodiment determines whether the robotic arm is in a preset monitoring position based on the target position and execution status, thereby enabling monitoring of the robotic arm during its operation intervals.
[0060] In one feasible implementation, refer to Figure 5 , Figure 5 This is a flowchart illustrating Embodiment 5 of the robotic arm surface monitoring method of this application; when the robotic arm is in a preset monitoring position, a preset imaging module is activated, including: Step S51: Determine the duration of the robotic arm's stationary state; Step S52: When the robotic arm is in a preset monitoring position and the static duration exceeds a preset static duration threshold, the imaging module is activated.
[0061] It should be noted that the preset static duration threshold can be set according to the actual situation. The preset static duration threshold is set to be greater than or equal to 3 seconds, for example, 3 seconds, 4 seconds, or 5 seconds. In order to ensure that the image to be monitored captured by the imaging module is clear and without blur, in this embodiment, it is necessary to determine the static duration of the robotic arm in a static state, and then compare the static duration with the preset static duration threshold. When the robotic arm is in a preset monitoring position and the static duration exceeds the preset static duration threshold, it is proven that the robotic arm is in a stable state, and then the imaging module is activated to acquire the image to be monitored on the surface of the robotic arm.
[0062] In this embodiment, when the robotic arm is in a preset monitoring position and the static time exceeds a preset static time threshold, the imaging module is activated to ensure that the image to be monitored captured by the imaging module is clear and unblurred, thereby improving the accuracy of particle feature recognition.
[0063] In one feasible implementation, refer to Figure 6 , Figure 6This is a flowchart illustrating Embodiment Six of the robotic arm surface monitoring method of this application; after performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm, the method further includes: Step S61: Record monitoring data, wherein the monitoring data includes monitoring time and particle information; Step S62: Based on the monitoring data recorded multiple times, analyze the particle change trend on the surface of the robotic arm and analyze the particle distribution on the surface of the robotic arm, so as to determine the particle source based on the particle distribution.
[0064] It should be noted that the purpose of analyzing particulate matter change trends is to predict future particulate pollution trends through monitoring data, so as to take measures in advance.
[0065] It should be noted that the sources of particles on the robotic arm surface include self-generation and transfer during operation. For example, wear and tear on moving parts of the robotic arm itself, and friction from joints, guide rails, bearings, etc., generating metal shavings or polymer dust. Surface material peeling, aging and flaking of coatings or platings, or aging debris from rubber or plastic parts. Particles adhering to the workpiece surface can also be transferred from the workpiece to the robotic arm's end effector or surface during gripping and handling.
[0066] In this embodiment, monitoring data is recorded after particle monitoring is completed. This data includes monitoring time, particle size, location, and quantity. Further, the particle trend on the robotic arm surface is analyzed by combining multiple recorded monitoring data. Optionally, based on the monitoring data from multiple monitoring sessions, statistical and analytical methods are used to identify trends in particulate contamination. For example, analyzing the trend of particle quantity changes over time; if the particle quantity shows an upward trend, it may indicate that cleaning measures need improvement. Analyzing the distribution changes of particles on the robotic arm surface; if the particle quantity in certain areas gradually increases, it may be necessary to strengthen cleaning measures in those areas. Further, based on the analysis results, future trends in particulate contamination are predicted, and corresponding decisions are made. For example, if an increase in particle quantity is predicted, cleaning measures can be taken in advance.
[0067] In addition, based on multiple recorded monitoring data, the distribution of particles on the surface of the robotic arm can be analyzed to infer the source of the particles. For example, analysis of the monitoring data revealed that the particles mainly appeared in the end gripper of the robotic arm. The main particles were found in the monitoring after the workpiece was gripped, which indicates that the source of contamination is mainly the transfer of particles from the workpiece surface to the robotic arm gripper.
[0068] This embodiment, through the above-described scheme, analyzes the particle change trend on the surface of the robotic arm based on multiple recorded monitoring data, enabling proactive measures to be taken based on the particle trend analysis results. Furthermore, it analyzes the particle distribution on the robotic arm surface to identify the core sources of particulate contamination, allowing for targeted remediation based on these sources and significantly reducing ineffective maintenance time.
[0069] In one feasible implementation, the step of performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm includes: Step S71: Binarize the image to be monitored according to a preset segmentation threshold to obtain a binarized image; In this embodiment, each pixel of the image to be monitored is traversed, and the grayscale value of each pixel is compared with a preset segmentation threshold. Based on the comparison result, the image to be monitored is binarized to divide the pixels in the image into background and foreground, resulting in a binarized image. It should be noted that the preset segmentation threshold can be a fixed threshold or dynamically adjusted according to local regions of the image.
[0070] Step S72: Perform morphological recognition on the binarized image to obtain the particle information.
[0071] It should be noted that the binarized image may contain noise. In this embodiment, morphological operations such as erosion, dilation, opening, and closing are used to process the binarized image. These morphological operations are based on existing mature algorithms and will not be elaborated upon here. Further, the particle positions are extracted from the morphologically processed image. Additionally, the particle size needs to be calculated; for example, the particle diameter is used as the particle size. Furthermore, the number of particles in the image is counted, thereby forming the particle information based on particle position, particle size, and particle quantity.
[0072] In one feasible implementation, the step of performing particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm includes: Step S81: Perform particle detection on the image to be monitored using a pre-built particle detection model to obtain the particle information. The particle detection model is obtained by iterative training based on pre-collected image samples and particle labels associated with the image samples.
[0073] It should be noted that the particle detection model can be a model such as YOLO or CNN, which pre-collects multiple image samples. Each image sample is associated with a pre-defined particle label, which characterizes information such as the size and location of particles in the image. The particle detection model is then iteratively trained based on the image samples and their associated particle labels. In this embodiment, the image to be monitored is input into the particle detection model to perform particle detection on each region of the image, obtaining particle information such as particle location, size, and number.
[0074] This embodiment obtains particle information such as particle location, size, and quantity in the image to be monitored by performing particle detection. This eliminates the need for manual particle monitoring, improving the accuracy and efficiency of particle monitoring. Subsequently, based on the particle information, it can accurately determine whether the robotic arm is contaminated with particles, reducing the occurrence of product defects caused by particle contamination.
[0075] In one feasible implementation, after determining that the robotic arm is contaminated with particulate matter, the method further includes: Step S91: Activate the alarm to allow staff to perform cleaning procedures based on the level of particulate contamination.
[0076] In this embodiment, when the robotic arm is contaminated with particulate matter, an alarm is activated. Optionally, an audible and visual alarm, such as a red warning light or buzzer, can be used to alert the operator. Alternatively, a system warning signal can be sent to the operator, for example, by sending a contamination warning command to the production control system. Upon receiving the warning, the operator can determine the degree of contamination based on the monitored image and then perform cleaning procedures, such as laser cleaning or wiping with a lint-free cloth. Optionally, a manual re-inspection can be triggered after cleaning is completed.
[0077] In this embodiment, when the robotic arm is contaminated with particles, an alarm is triggered so that staff can perform cleaning according to the degree of particle contamination, thereby achieving timely early warning of particle contamination and reducing product defects caused by particles.
[0078] It should be noted that the examples in the figure are only for understanding this application and do not constitute a limitation on the robotic arm surface monitoring method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.
[0079] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0080] This application also provides a surface monitoring device for a robotic arm; please refer to [reference needed]. Figure 7 , Figure 7 This is a schematic diagram of the module structure of the robotic arm surface monitoring device according to an embodiment of this application; the robotic arm surface monitoring device includes: The determination module 101 is used to determine whether the robotic arm is in a preset monitoring position in response to the motion state signal of the robotic arm; The trigger module 102 is used to activate the preset imaging module when the robotic arm is in the preset monitoring position, so as to acquire the image to be monitored on the surface of the robotic arm; The image processing module 103 is used to perform particle detection on the image to be monitored to obtain particle information on the surface of the robotic arm.
[0081] The robotic arm surface monitoring device provided in this application, employing the robotic arm surface monitoring method in the above embodiments, can solve the technical problems mentioned in the background art. Compared with the prior art, the beneficial effects of the robotic arm surface monitoring device provided in this application are the same as those of the robotic arm surface monitoring method provided in the above embodiments, and other technical features in the robotic arm surface monitoring device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.
[0082] This application provides a robotic arm surface monitoring device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the robotic arm surface monitoring method in the first embodiment described above.
[0083] The following is for reference. Figure 8 , Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the robotic arm surface monitoring method in this application embodiment. The robotic arm surface monitoring device in this application embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), vehicle terminals (such as vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The robotic arm surface monitoring device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.
[0084] like Figure 8As shown, the robotic arm surface monitoring device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the robotic arm surface monitoring device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the robotic arm surface monitoring device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows robotic arm surface monitoring devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0085] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0086] The robotic arm surface monitoring device provided in this application, employing the robotic arm surface monitoring method in the above embodiments, can solve the technical problems mentioned in the background art. Compared with the prior art, the beneficial effects of the robotic arm surface monitoring device provided in this application are the same as those of the robotic arm surface monitoring method provided in the above embodiments, and other technical features of the robotic arm surface monitoring device are the same as those disclosed in the previous embodiment method, and will not be repeated here.
[0087] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0089] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the robotic arm surface monitoring method in the above embodiments.
[0090] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.
[0091] The aforementioned computer-readable storage medium may be included in the robotic arm surface monitoring device; or it may exist independently and not assembled into the robotic arm surface monitoring device.
[0092] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the robotic arm surface monitoring device, the robotic arm surface monitoring device: responds to the robotic arm's motion state signal to determine whether the robotic arm is in a preset monitoring position; when the robotic arm is in the preset monitoring position, it activates a preset imaging module to acquire an image of the robotic arm surface to be monitored; and performs particle detection on the image to be monitored to obtain particle information on the robotic arm surface. When the robotic arm's motion state signal is combined with the detection that the robotic arm is in the preset monitoring position, the preset imaging module is activated to acquire an image of the robotic arm surface to be monitored, and then the image to be monitored is analyzed to determine whether there is particle contamination on the robotic arm. This monitoring can be performed without interrupting the production process and without manual inspection, improving the accuracy and efficiency of particle monitoring on the robotic arm surface during production.
[0093] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0095] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0096] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described robotic arm surface monitoring method, and is capable of solving the technical problems described in the background art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the robotic arm surface monitoring method provided in the above embodiments, and will not be repeated here.
[0097] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the robotic arm surface monitoring method described above.
[0098] The computer program product provided in this application can solve the technical problems described in the background section. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the robotic arm surface monitoring method provided in the above embodiments, and will not be repeated here.
[0099] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A method of robot arm surface monitoring, characterized by, The method comprises the following steps: determining whether the robot arm is in a preset monitoring position in response to a motion state signal of the robot arm; starting a preset imaging module to collect a to-be-monitored image of the surface of the robot arm when the robot arm is in the preset monitoring position; performing particle detection on the to-be-monitored image to obtain particle information of the surface of the robot arm.
2. The method of claim 1, wherein, The motion state signal comprises an action instruction feedback signal and / or a position feedback signal; The method comprises the following steps of: analyzing the action instruction feedback signal to obtain an instruction analysis result; analyzing the position feedback signal to obtain a current position of the robot arm; determining whether the robot arm is in the preset monitoring position according to the instruction analysis result and / or the current position of the robot arm.
3. The method of claim 2, wherein, The instruction analysis result comprises an instruction type and an execution state, and the method comprises the following steps of: determining whether the robot arm is in the preset monitoring position according to the instruction type, a preset instruction type, and the execution state, wherein the preset instruction type comprises a first instruction type, a second instruction type, and a third instruction type; the first instruction type is an instruction for controlling the robot arm to move to a preset first position to grasp a target object to be processed; the second instruction type is an instruction for controlling the robot arm to move to a preset second position to transfer the target object to a process equipment for process processing; and the third instruction type is an instruction for controlling the robot arm to move to a preset third position to place the target object after process processing.
4. The method of claim 2, wherein, The instruction analysis result comprises a target position and an execution state, and the target position is a position that the robot arm needs to reach in advance, and the method comprises the following steps of: determining whether the robot arm is in the preset monitoring position according to the target position and the execution state.
5. The robotic arm surface monitoring method of claim 1, wherein, The method comprises the following steps of: determining a static duration when the robot arm is in a static state; starting the imaging module when the robot arm is in the preset monitoring position and the static duration exceeds a preset static duration threshold.
6. The robotic arm surface monitoring method of claim 1, wherein, After the particle detection on the to-be-monitored image to obtain the particle information of the surface of the robot arm, the method further comprises the following steps of: recording monitoring data, wherein the monitoring data comprises a monitoring time and the particle information; analyzing a particle change trend of the surface of the robot arm and analyzing a particle distribution condition of the surface of the robot arm according to the monitoring data recorded multiple times, so as to determine a particle source according to the particle distribution condition.
7. The robotic arm surface monitoring method of claim 1, wherein, The particle information comprises a particle size and a particle quantity. After the particle detection on the to-be-monitored image to obtain the particle information of the surface of the robot arm, the method further comprises the following steps of: determining that the robot arm is contaminated by particles when the particle size is greater than a preset particle size threshold or the particle quantity is greater than a preset particle quantity threshold. When the particle size is less than or equal to the preset particle size threshold, and the particle quantity is less than or equal to the preset particle quantity threshold, it is determined that the robot arm is free of particle contamination.
8. The robotic arm surface monitoring method of claim 1, wherein, The particle detection on the image to be monitored obtains particle information of the surface of the robot arm, including: The binary image is obtained by performing binaryzation processing on the image to be monitored according to a preset segmentation threshold; The particle information is obtained by performing morphological recognition on the binary image; And / or, The particle information is obtained by performing particle detection on the image to be monitored by using a pre-constructed particle detection model, wherein the particle detection model is obtained by iterative training according to pre-collected image samples and particle labels associated with the image samples.
9. A robot arm surface monitoring device, characterized by Including: The determination module is configured to determine whether the robot arm is at a preset monitoring position in response to a motion state signal of the robot arm; The triggering module is configured to start a preset imaging module to collect an image to be monitored on the surface of the robot arm when the robot arm is at the preset monitoring position; The image processing module is configured to perform particle detection on the image to be monitored to obtain particle information of the surface of the robot arm.
10. A robot arm surface monitoring device, characterized by The robot arm surface monitoring device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the robot arm surface monitoring method according to any one of claims 1 to 8.
11. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the robot arm surface monitoring method according to any one of claims 1 to 8.