Control method for an industrial robot system

Abstract

The application provides a control method of an industrial robot system, which responds to a user's operation and sends a self-check instruction to at least one of a robot unit and a vision unit. In response to the self-check instruction, at least one of the robot unit and the vision unit performs self-checking and sends a self-check normal instruction. Before receiving the self-check normal instruction, transmission of a motion control instruction for controlling the robot unit is intercepted, and after receiving the self-check normal instruction, the robot unit enters an enabled state. Image information obtained by the vision unit is matched with pre-stored first template information, and when a matching result meets a matching condition, vision positioning information is output, and when the matching result does not meet the matching condition, the image information is matched with pre-stored second template information. Based on the vision positioning information, a motion control instruction is generated. The robot unit performs a task based on the motion control instruction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of industrial robot arm control, and particularly relates to a control method of an industrial robot arm system. BACKGROUND

[0002] The operation link of the industrial robot arm system in the prior art is relatively complex, and highly depends on the intervention of experienced engineers in the switching of working scenes or fault maintenance, so that the overall work efficiency is relatively low. SUMMARY

[0003] The present application aims to provide a control method of an industrial robot arm system to at least solve or alleviate some problems in the prior art.

[0004] The present application provides a control method of an industrial robot arm system, the industrial robot arm system comprising a robot arm unit, a vision unit and a control unit. The control method of the industrial robot arm system comprises a start triggering step, a self-checking step, an enabling switching step, a vision positioning step, a task determining step and a task executing step. In the start triggering step, a self-checking instruction is sent to at least one of the robot arm unit and the vision unit in response to an operation of a user. In the self-checking step, at least one of the robot arm unit and the vision unit performs self-checking in response to the self-checking instruction and issues a self-checking normal instruction. In the enabling switching step, the transmission of a motion control instruction for controlling the robot arm unit is intercepted before the self-checking normal instruction is received, and the robot arm unit enters an enabled state after the self-checking normal instruction is received. In the vision positioning step, image information obtained by the vision unit is matched with first template information stored in advance, and vision positioning information is output when the matching result meets a matching condition, and the image information is matched with second template information stored in advance when the matching result does not meet the matching condition. In the task determining step, a motion control instruction is generated based on the vision positioning information. In the task executing step, the robot arm unit executes a task based on the motion control instruction.

[0005] In the technical solution of the present application, the start triggering step, the self-checking step and the enabling switching step are used to avoid misoperation when the device is not ready. The at least two templates available for matching in the vision positioning step can improve the efficiency of vision positioning.

[0006] In the control method of the industrial robot arm system in the optional technical solution, in the start triggering step, a self-checking instruction is sent to the robot arm unit, the vision unit and the control unit in response to a single action operation of a user.

[0007] In the above optional technical solution, the self-checking of the robot arm, the vision and the control unit is triggered at the same time through a single action, which reduces the human operation link, shortens the preparation time, reduces the abnormal probability caused by step omission, and makes the system ready process more concise and consistent.

[0008] In the control method of the industrial robot system of the optional technical solution, in the visual positioning step, the first template information includes information of the first workpiece, and the second template information includes information of the second workpiece.

[0009] In the above optional technical solution, when the first template corresponds to the first workpiece and the second template corresponds to the second workpiece, the visual positioning can directly cover multiple varieties of materials, reduce the workload of template reconstruction and parameter resetting when changing types, realize quick switching and stable recognition at the same station, and thus improve the adaptability of multi-product production.

[0010] In the control method of the industrial robot system of the optional technical solution, in the visual positioning step, the first template information includes information of the first workpiece, and the second template information includes information of the second workpiece.

[0011] In the control method of the industrial robot system of the optional technical solution, in the visual positioning step, the first template information includes standard template information and historical field template information, the matching condition is that a standard similarity of the image information to the standard template information exceeds a first threshold value, or the standard similarity exceeds a second threshold value and a field similarity of the image information to the historical field template information is higher than the standard similarity, and the standard similarity indicated by the second threshold value is lower than the standard similarity indicated by the first threshold value. The historical field template information refers to template information generated according to one or more images successfully grabbed in a recent period of time, which is closer to the field situation than the standard template information.

[0012] In the above optional technical solution, under the working conditions of oil stains, color difference, slight wear, and mixed use of different batches, the matching success rate can be improved, and the parameter maintenance workload can be reduced.

[0013] In the above optional technical solution, when the first template and the second template correspond to different angles of view of the same workpiece, the system has higher tolerance to attitude changes, can maintain stable matching under conditions such as workpiece corner turning and overturning, reduces missed detection and false detection, and ensures the rhythm and accuracy of continuous operation.

[0014] In the control method of the industrial robot system of the optional technical solution, in the visual positioning step, when the matching result does not satisfy the matching condition, the light-related parameters of the image information are first changed and corrected image information is generated, the corrected image information is matched with the first template information, and when the matching result still does not satisfy the matching condition, the image information is matched with the pre-stored second template information.

[0015] In the optional technical solution, when the matching condition is not met, the light-related parameter correction is preferentially performed, and then the first template is tried to match again, so that the influence of light drift can be preferentially eliminated, and unnecessary template switching can be avoided; if the matching condition is still not met, the second template is switched, so that the matching success rate is improved from the process.

[0016] In the control method of the industrial robot system in the optional technical solution, the industrial robot system further comprises a vibrating disc unit, and when the matching results of the image information and the pre-stored at least two template information do not meet the matching condition in the visual positioning step, a vibration instruction is sent to the vibrating disc unit. The at least two template information refers to at least including the first template information and the second template information.

[0017] In the optional technical solution, when the at least two templates cannot meet the matching condition and the light vibration of the feeding mechanism is triggered, the overlapped or stacked materials can be scattered, the target feature is re-exposed, the success rate of subsequent identification and grabbing is improved, and the probability of manual intervention and line stop is reduced.

[0018] In the control method of the industrial robot system in the optional technical solution, in the visual positioning step, the local area image of the image information in which the workpiece is suspected to be stacked is secondarily matched with the first template information, and when the secondary matching result does not meet the secondary matching condition, a vibration instruction is sent to the vibrating disc unit.

[0019] In the optional technical solution, the number of times of grabbing failure of the industrial robot system caused by the workpiece stacking in the grabbing process is reduced, and the smoothness and efficiency of the work of the industrial robot system are improved.

[0020] In the control method of the industrial robot system in the optional technical solution, the control unit comprises a graphical user interface module, and the image information and the visual positioning information are displayed on the graphical user interface module.

[0021] In the optional technical solution, the original image and the positioning result are displayed on the graphical user interface, so that the debugging and monitoring are visualized, the problems such as identification abnormality and improper parameters can be quickly found, the on-site adjustment time is shortened, and the risk caused by configuration errors is reduced.

[0022] In the control method of the industrial robot system in the optional technical solution, the visual positioning step comprises: an interference removal step, in response to an instruction of setting a local shielding area in the image information through the graphical user interface module, the local shielding area does not participate in the step of matching the image information with the first template information.

[0023] In the optional technical scheme, the local shielding area is set on the interface, so that the area does not participate in the matching calculation, the influence of the interference sources such as the background high reflection and the tray edge on the similarity can be effectively excluded, the false matching is reduced, and the positioning accuracy and the result stability are improved.

[0024] In the control method of the industrial robot system in the optional technical scheme, the task determination step includes a test step, in response to a partial motion test instruction input through the graphical user interface module, the robot unit is controlled to execute the partial motion test instruction, and in response to a replacement instruction input through the graphical user interface module, the partial motion test instruction is used to replace the corresponding partial instruction in the motion control instruction.

[0025] In the optional technical scheme, the partial motion test and replacement are introduced in the task determination, the local action segment is allowed to be controlled to run and replaced in place after verification, the adjustment range and risk are limited locally, the cost of repeated modification and regression verification of the whole program is reduced, and the context continuity and operation safety are ensured.

[0026] In the control method of the industrial robot system in the optional technical scheme, the control method further includes a fault handling step, judging a fault type, when the fault type is a first fault type that can be automatically recovered, entering an automatic recovery process, and when the fault type is a second fault type that cannot be automatically recovered, outputting a prompt information.

[0027] In the optional technical scheme, by judging the automatic recovery / non-automatic recovery in the fault handling link, for the first fault type that can be automatically recovered, the system directly enters the preset automatic recovery process, manual waiting and repeated trial and error are avoided, the average repair time is shortened, and the downtime is reduced. For the second fault type that cannot be automatically recovered, the system immediately stops the related action and outputs the prompt information, reminds the personnel to intervene, prevents continuous attempts under unsafe or uncontrollable conditions, and suppresses secondary faults and chain damage.

[0028] In the control method of the industrial robot system in the optional technical scheme, the control unit includes an analysis module, and any one of normal operation data in the task execution step, fault data in the fault handling step, and technical documents of the industrial robot system is input to the analysis module.

[0029] In the optional technical scheme, the normal operation data, the fault data and the technical data are input to the analysis module to form a unified entrance of data and knowledge, facilitate model learning and rule optimization, improve the accuracy and coverage of diagnosis and strategy recommendation, and gradually reduce the dependence on artificial experience.

[0030] In the control method of the industrial robot system of the optional technical solution, the step of judging the fault type comprises: based on the first determination condition, the fault is divided into the second fault type and the pending fault type, the first determination condition is used to indicate whether the fault is related to hardware damage or safety; based on the second determination condition, the fault of the pending fault type is divided into the first fault type and the second fault type, the second determination condition is used to indicate whether the automatic recovery feasibility probability value of the fault of the pending fault type exceeds the threshold value according to the evaluation of the analysis module.

[0031] In the above optional technical solution, two-level determination is adopted: first, the first determination is made based on whether it involves hardware damage or safety risk, and then the second determination is made based on the feasibility probability of the analysis module and the threshold value, which can improve the disposal efficiency under the premise of ensuring safety.

[0032] In the control method of the industrial robot system of the optional technical solution, the maintenance problem answering step and the maintenance process recording step are further included. In the maintenance problem answering step, the analysis module generates answer information for the inquiry in response to the inquiry of the user. In the maintenance process recording step, maintenance data is generated and sent to the analysis module in response to the maintenance operation of the user.

[0033] In the above optional technical solution, the maintenance problem answering and the maintenance process recording are set: the former provides interpretation and suggestion based on data and materials for on-site personnel, reducing the professional threshold; the latter feeds back the manual disposal process to the analysis module for subsequent model and rule updating, so that maintenance experience is converted into reusable strategies, thereby forming a closed loop for continuous improvement. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A structural schematic diagram of an industrial robot system according to an embodiment of the present application is shown;

[0035] Figure 2 A schematic diagram of a control method of an industrial robot system according to an embodiment of the present application is shown;

[0036] Figure 3 A schematic diagram of a visual positioning step in a control method of an industrial robot system according to an embodiment of the present application is shown;

[0037] Figure 4 A schematic diagram of a fault handling step in a control method of an industrial robot system according to an embodiment of the present application is shown.

[0038] Reference signs: industrial robot system 100, robot unit 10, visual unit 20, control unit 30, vibration disc unit 40. DETAILED DESCRIPTION

[0039] It should be noted that the working principle, features and advantages of the control method of the industrial robot system according to the present application will be described below by way of example, but it should be understood that all the descriptions are given only for illustration and therefore should not be understood as forming any limitation on the present application.

[0040] Furthermore, for any single technical characteristic described or implied in the embodiments mentioned herein, or any single technical characteristic shown or implied in the various drawings, the present application still allows any combination or deletion to be made between these technical characteristics (or their equivalents) without any technical obstacles, thus obtaining more other embodiments of the present application which can not be directly mentioned herein.

[0041] <Start-Enable Switching>

[0042] The present embodiment provides a control method of an industrial robot system 100, as shown in Figure 1 The industrial robot system 100 includes a robot unit 10, a vision unit 20 and a control unit 30. As shown in Figure 2 The control method of the industrial robot system 100 includes a start trigger step, a self-checking step and an enable switching step.

[0043] In the start trigger step, the industrial robot system 100 responds to a single action of the user and sends a self-check instruction to the robot unit 10, the vision unit 20 and the control unit 30. The single action can be pressing a physical "one-key start" button, or touching a screen, or stepping on a switch, or clicking a remote control. In response to the single action, the control unit 30 no longer requires the user to separately "turn on power, zero, and enable", but sends a self-check instruction to the robot unit 10 and the vision unit 20 at one time, and enters the self-check step. In the self-check step, the industrial robot system 100 responds to the self-check instruction, and the robot unit 10 and the vision unit 20 perform self-checking and issue a self-check normal instruction. The self-checking can specifically include power supply and emergency stop circuit checking of the robot unit 10, servo power-on preparation, necessary zero calibration (for example, Z-axis first lifting and then finding the original point, R-axis preset zero), camera connection of the vision unit 20, test shooting exposure and contrast threshold checking, etc. In the enable switching step, before receiving the self-check normal instruction, the transmission of the motion control instruction for controlling the robot unit 10 is intercepted, and after receiving the self-check normal instruction, the robot unit 10 enters an enabled state. Specifically, before receiving all the necessary self-check normal instructions, the control unit 30 intercepts all motion control instructions (such as point, trajectory, speed / acceleration setting, etc.) directed to the robot unit 10, and only allows instructions without motion side effects such as state query and parameter reading to pass. After receiving the self-check normal instruction, the control unit 30 switches the robot unit 10 to the enabled state, and removes the interception of the motion instruction, so that the system enters a movable working state. When the robot unit 10 is switched to the enabled state, the user can be visually prompted by a prompt light. When the industrial robot system 100 finds an abnormality in the self-check step, the user can be prompted to intervene by a buzzer alarm.

[0044] In the embodiment, by a single action of the user, a self-check instruction is sent to multiple hardware devices, and after receiving the self-check normal instruction, the robot unit 10 is switched to the enabled state, which is beneficial to improve the operation convenience of the industrial robot system 100 and reduce the risk of false motion and machine collision caused by false operation.

[0045] In other embodiments, the self-check instruction can be sent to only one or two hardware units, or can be sent to more hardware units.

[0046] In other embodiments, the start trigger condition can also be preset, such as a fixed time, a work shift rotation condition, etc. When the specific start trigger condition is met, the industrial robot system 100 automatically performs the start trigger step.

[0047] <Visual positioning>

[0048] Reference Figure 2 and Figure 3 The control method of the industrial robot system 100 provided by the embodiment comprises a visual positioning step, in which image information acquired by the vision unit 20 is matched with pre-stored first template information, and when the matching result meets the matching condition, visual positioning information is output, and when the matching result does not meet the matching condition, the light-related parameters of the image information are first changed and corrected image information is generated, and the corrected image information is matched with the first template information, and when the matching result still does not meet the matching condition, the image information is matched with pre-stored second template information. The so-called "first template information and second template information" can correspond to the appearance of different workpieces, or can correspond to the template description of the same workpiece under different viewing angles, different postures or different lighting conditions. For example, the first template information includes the information of a gear (as an example of the information of a first workpiece), and the second template information includes the information of a gasket (as an example of the information of a second workpiece). For another example, the first template information includes the information of the front view angle of a flange (as an example of the information of a first view angle of a first workpiece), and the second template information includes the information of the back view angle of the flange (as an example of the information of a second view angle of the first workpiece). The so-called "matching condition" generally refers to the similarity score of the template and the current image reaching a threshold value. The so-called "changing the light-related parameters of the image information" refers to performing brightness, contrast, local contrast enhancement, channel selection and other processing on the collected image without changing the scene hardware optical structure, so as to obtain a corrected image that is more conducive to matching.

[0049] In the embodiment, a first set of light-related parameters and a second set of light-related parameters are pre-set. In the step of "changing the light-related parameters of the image information", the first set of light-related parameters and the second set of light-related parameters can be tried in sequence. According to actual needs, multiple sets of light-related parameters can be pre-set.

[0050] In the embodiment, the industrial robot system 100 further comprises a vibration disc unit 40, and in the visual positioning step, when the matching results of the image information and the pre-stored at least two template information do not meet the matching condition, a vibration instruction is sent to the vibration disc unit 40. Specifically, when the matching results of the image information and the pre-stored at least two template information do not meet the matching condition, it is likely that there is a problem of overlapping between workpieces, which leads to the inability to match with the template. At this time, a vibration instruction can be sent to the vibration disc unit 40 to make the overlapped workpieces spread out and then re-match with the template.

[0051] In this embodiment, by pre-storing multiple templates, the visual positioning can be quickly performed when the type of the workpiece to be grabbed changes, and the working efficiency of the industrial robot system 100 is improved. In addition, by changing the light-related parameters of the image information and generating corrected image information for matching, the obstacles caused by the actual environment (light, oil stains, dust impurities) of the device to the matching can be reduced, the efficiency of the visual positioning of the system is improved, the manual intervention is reduced, and the working efficiency of the industrial robot system 100 is improved.

[0052] The visual positioning step in some embodiments can also include a secondary image comparison step, that is, selecting a local area image from the image information obtained by the visual unit 20 to compare with the template information (first template information and / or second template information). The selected local area image is the image of the local area where the superposition is suspected to exist. If it is believed that there is a local superposition of the workpiece after the secondary image comparison, a vibration instruction can be sent to the vibration disc. Through the secondary image comparison step, the number of times of grabbing failure of the industrial robot system 100 in the grabbing process due to the superposition of the workpiece is reduced, and the working fluency and working efficiency of the industrial robot system 100 are improved.

[0053] In some embodiments, in the visual positioning step, the first template information includes standard template information and historical field template information, and the matching condition is that the standard similarity of the image information to the standard template information exceeds a first threshold, or the standard similarity exceeds a second threshold and the field similarity of the image information to the historical field template information is higher than the standard similarity, the standard similarity indicated by the second threshold being lower than the standard similarity indicated by the first threshold. The historical field template information refers to template information generated according to one or more images successfully captured in a recent period of time, which is closer to the field situation than the standard template information. In this way, the matching success rate can be improved and the parameter maintenance workload can be reduced in the working conditions of oil stains, color difference, slight wear, and mixed use of different batches. Specifically, a standard template is first made with a clean sample when going online, and clear contour information is retained in the standard template. Then, every time the closed loop is successfully captured (the mechanical arm confirms that the correct workpiece is taken) and the photo quality is qualified, the system automatically removes the background, compresses the reflection, and adjusts the uniform scale of the local image, and stores it in a small sample library, only retaining the most reliable 50 samples in the recent period of time. These samples are fused into one or more “field templates”, which are equivalent to learning the current light, color deviation, and slight wear into the system. When determining, if the standard similarity is already high (such as 0.86, exceeding the first threshold 0.8), it can be considered that the matching condition is met; if the standard template score is relatively low (such as 0.78, lower than the first threshold 0.8 but higher than the second threshold 0.7), and the field similarity is higher than the standard similarity (such as the standard similarity is 0.78 and the field similarity is 0.82), it can also be considered that the matching condition is met. The calculation of the standard similarity and the field similarity can be calculated by taking the average value of the similarity of multiple points. In some embodiments, some isolated points with extremely high similarity and extremely low similarity can be excluded to improve the accuracy of the similarity calculation.

[0054] In the present embodiment, the visual positioning step can further include an interference removal step. This step is used to mask the non-target area in the image according to the user's instruction before matching. Specifically, in response to the instruction of the graphical user interface module to set a local shielding area for the image information, the pixels corresponding to the local shielding area are excluded from the calculation in the subsequent matching process of the image information and the first template information. The user can select the interference part (such as oil stains, dust, light and shadow spots, etc.) of the non-identification object on the image information displayed by the graphical user interface module and set it as a local shielding area, so that this area does not participate in the matching. Through the above interference removal step, the influence of the actual environment (such as light changes, oil stains, dust impurities) of the device on the matching can be effectively reduced, the positioning accuracy and stability of the vision can be improved, and the working efficiency of the industrial robot system 100 can be improved.

[0055] In some other embodiments, the number of templates can be three or more. For example, when three templates are pre-stored, the light-related parameter of the image information is changed before switching to the next template, and when the matching results of the image information and the pre-stored three template information all do not satisfy the matching condition, a vibration instruction is sent to the vibration disc unit 40.

[0056] <Generating and executing motion control instructions>

[0057] Reference Figure 2 The control method of the industrial robot system 100 provided in the embodiment includes a task determination step and a task execution step. In the task determination step, a motion control instruction is generated based on visual positioning information. Specifically, after the control unit 30 receives the visual positioning information, an executable motion control instruction sequence can be generated according to a predetermined process template. The visual positioning information can include the pose of the target workpiece in the camera coordinate system, which is transformed into the coordinate system of the robot unit 10 based on the hand-eye calibration matrix, and the result is filled into the parameter slot in the process template, such as “approach height, normal angle, grabbing point, jaw opening, placement pose”, etc. Then, by instantiating the parameterized process steps into specific motion instructions, such as speed and acceleration configuration / tool coordinate and workpiece coordinate switching / jaw opening and closing, etc., an instruction chain is formed in the order of “approach, alignment, grabbing, lifting, avoidance, placement, retreat”, which serves as the motion control instruction for a single operation. In the task execution step, the robot unit 10 executes the task based on the motion control instruction.

[0058] In the task determination step in the embodiment, a teaching tray arrangement step is further included. Specifically, the end of the robot unit 10 is moved to the left upper corner reference position of the tray and clicked on “Set A”, and the pose of this position in the coordinate system of the robot unit 10 is recorded as point A; then it is moved to the right upper corner reference position along the horizontal direction of the tray, clicked on “Set B”, and recorded as point B; and then it is moved to the left lower corner reference position along the vertical direction, clicked on “Set C”, and recorded as point C. The graphical user interface module inputs parameters such as the number of rows and columns and saves them. The control unit 30 establishes the tray coordinate system at point A accordingly. Through this teaching tray arrangement step, the cumulative error introduced by point-by-point teaching can be avoided, and the changeover time is reduced from “point-by-point input” to “teaching three points + row and column setting”, significantly improving the efficiency of machine adjustment. Moreover, the two basis vectors of the matrix are directly obtained from AB and AC, which can naturally adapt to “non-fully rectangular, slightly skewed” actual trays, ensure that the spacing and direction between adjacent units are consistent with the actual tray, and improve the neatness and consistency of the arrangement.

[0059] In this embodiment, the control unit 30 includes a graphical user interface module, and the robot arm control method further includes a test step. In the test step, in response to a partial motion test instruction input through the graphical user interface module, the robot arm unit 10 is controlled to execute the partial motion test instruction, and in response to a replacement instruction input through the graphical user interface module, the corresponding partial instruction in the motion control instruction is replaced with the partial motion test instruction. By "the control unit 30 includes a graphical user interface module", it is meant that the control unit 30 is built-in with a visual human-machine interface, so that the operator can view, select and adjust the partial segments of the robot motion instruction without writing scripts. By "partial motion test instruction", it is meant that a number of continuous or discrete action segments are extracted from the complete motion control instruction chain as test objects, typically such as approach segment, alignment segment, grabbing segment, lifting segment or avoidance segment, and temporary changes are allowed to be made to the parameters of these segments, such as speed, acceleration, approach height, end pose, round corner transition radius, force / torque limit, etc. By "replacement instruction", it is meant that after the test result meets the preset acceptance conditions, a write-back action is issued through the graphical user interface module, and the verified segment content in the test is used to replace the corresponding position in the original instruction chain, so as to fix the trial running optimization result as the formal instruction for subsequent batch running.

[0060] In this embodiment, the local segment is subjected to controlled trial running through graphical interaction, and is replaced in situ after reaching the preset safety and quality indicators. In this way, the changeover and machine adjustment time can be shortened, the impact of local parameter adjustment can be reduced, and the collision probability in the trial running stage can be reduced.

[0061] <Failure handling and maintenance>

[0062] The industrial robot arm system 100 provided in this embodiment includes an analysis module, and the robot arm control method includes a failure handling step, a maintenance problem answering step and a maintenance process recording step. The analysis module can be a local analysis module or connected to an external large language model.

[0063] Reference Figure 4In the fault processing step, the fault type is determined, when the fault type is determined to belong to the first fault type that can be automatically recovered, the automatic recovery process is entered, and when the fault type belongs to the second fault type that cannot be automatically recovered, prompt information is output. Specifically, the industrial robot system 100 continuously collects various types of data representing the state of the industrial robot system 100 equipment during normal operation, which can include image quality scores, negative pressure or clamping force readings, load and speed of each motion joint, minimum distance from the environment or tooling, communication delay, end effector state, and safety-related inputs (emergency stop, access control, area protection, etc.) and other types of operation data. When a certain data exceeds the set threshold, or the control flow is abnormal, that is, the fault processing step is entered. Then the fault type is determined. The first fault type that can be automatically recovered refers to an abnormality that can be recovered within a short time through programmed operation without the need for personnel to enter the equipment work area or replace hardware, such as insufficient clamping force but intact pipeline, temporary insufficient image quality, slight touch warning, short-time communication jitter, etc. The second fault type that cannot be automatically recovered refers to an abnormality that requires human intervention or has obvious safety risks, such as emergency stop being triggered, safety door being opened, drive component hardware failure, image acquisition device being offline, end effector being damaged, and abnormality that is difficult to recover reliably through programmed means within a limited time (such as continuous path calculation failure and no alternative path).

[0064] The step of determining the fault type can include: based on a first determination condition, the fault is divided into a second fault type and a pending fault type, the first determination condition is used to indicate whether the fault is related to hardware damage or safety; based on a second determination condition, the fault of the pending fault type is divided into the first fault type and the second fault type, the second determination condition is used to indicate whether the automatic recovery feasibility probability value of the fault of the pending fault type exceeds a threshold value according to the evaluation of the analysis module. Specifically, first, according to the fixed rule, the fault related to hardware damage or safety is determined as the second fault type, and for other faults not related to hardware damage or safety (i.e. pending fault type), the analysis module evaluates the automatic recovery feasibility probability, when the probability is higher than the set threshold, it is determined as the first fault type, and when the probability is lower than the set threshold, it is determined as the second fault type.

[0065] When the first fault type is determined, the system enters an automatic recovery procedure, which attempts each item in a preset order, such as reacquiring images and automatically adjusting imaging parameters, switching to a backup identification template or identification strategy, expanding the detection area and performing a review positioning, applying a short and light vibration to the feeding mechanism to disperse the stack, reducing the approach speed and adjusting the approach direction, cleaning and rebuilding the clamping or suction circuit, etc. Each attempt sets a maximum number of attempts and a timeout period, and any successful attempt exits the recovery and returns to the work flow. If multiple consecutive failures or overall timeout occurs, it is automatically changed to the second fault type. When the second fault type is determined, a prompt message can be output through the graphical user interface module or other media.

[0066] Normal operation data in the task execution step and fault data in the fault handling step can be recorded as logs in their entirety and sent to the analysis module. Fault data can include: fault occurrence time, fault code and level, original sensor sequence and control instruction fragments for a period of time before and after the fault, key image and cropped area snapshots, attempted automatic recovery items and their results (success / failure and time consumption), safety input state, personnel operations and remarks in the graphical user interface module, etc. Data can be stored in structured fields and attachments, first entering local buffer, then asynchronously uploaded when communication conditions allow. In the case of power failure or network disconnection, it can be automatically supplemented after the next start. In addition to fault-related data, the industrial robot system 100 can also record regular operation data, i.e. normal operation data, according to a sampling strategy, for model baseline maintenance. In this embodiment, the analysis module can also receive device operation manuals, maintenance manuals, etc., i.e. technical documents, segment, index and associate them with historical logs.

[0067] In the maintenance problem answering step, diagnostic and decision support is provided to on-site personnel. Personnel can initiate questions in natural language or fixed questions in the graphical user interface module, such as "What is the reason for the recent increase in missed catches" "How to handle the suction alarm" "Can the fault be automatically recovered". The analysis module can retrieve relevant logs and similar cases in combination with the current device state, or query the corresponding entries in the operation manual and maintenance manual, generate answer information containing cause analysis and operation suggestions, and output in text or voice. To avoid misoperation, "directly executable" suggestions in the answer can only be triggered by one key when safety conditions are met and operator permissions are passed, otherwise only step prompts are given. When there is insufficient evidence or the answer confidence is low, the system automatically downgrades to conservative suggestions and prompts to contact maintenance personnel with higher permissions.

[0068] In the maintenance process recording step, maintenance data is generated in response to a user's maintenance operation and sent to the analysis module. Specifically, when a person successfully recovers a fault of a certain second fault type through a set of explicit operation steps, the system marks the sequence of steps as a "candidate strategy". In subsequent multiple independent occurrences of similar faults, if different people repeatedly use the sequence successfully and no potential risks are found after inspection, the sequence is upgraded to an official automatic recovery item for the first fault type after being approved by a person with engineering authority in the graphical user interface module. That is, the analysis module enables the system to have continuous improvement capability, allowing manual handling experience to be incorporated into automatic recovery when conditions are met.

[0069] In this embodiment, through the control method, a closed loop from abnormality discovery, automatic or manual handling, data sedimentation to knowledge-based answering can be formed without reducing safety. Automatic recovery is used to quickly eliminate programmable problems. Logs enable the occurrence, attempted measures, success or failure results to be completely saved and learned. Maintenance Q&A feeds the sedimented knowledge to the on-site personnel in a comprehensible and executable manner, and manual experience can be converted into automatic strategies after being verified multiple times. Thus, the average repair time is shortened, the downtime probability is reduced, and the handling results are more consistent and traceable.

[0070] The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A control method for an industrial robotic arm system, the industrial robotic arm system comprising a robotic arm unit, a vision unit, and a control unit, characterized in that: include: The triggering step is initiated in response to the user's operation and a self-test command is sent to at least one of the robotic arm unit and the vision unit; In response to the self-test command, at least one of the robotic arm unit and the vision unit performs a self-test and issues a self-test normal command. The enable switching step involves intercepting the transmission of motion control commands for controlling the robotic arm unit before receiving the self-test normal command, and the robotic arm unit entering the enabled state after receiving the self-test normal command. The visual positioning step involves matching the image information acquired by the visual unit with the pre-stored first template information. When the matching result meets the matching conditions, visual positioning information is output. When the matching result does not meet the matching conditions, the image information is matched with the pre-stored second template information. The task determination step involves generating motion control commands based on the visual positioning information. After receiving the visual positioning information, the control unit generates an executable sequence of motion control commands according to a predetermined process template. The task execution steps involve the robotic arm unit executing the task based on the motion control commands. In the visual positioning step, the first template information includes standard template information and historical scene template information. The matching condition is that the standard similarity between the image information and the standard template information exceeds a first threshold, or the standard similarity exceeds a second threshold and the scene similarity between the image information and the historical scene template information is higher than the standard similarity. The standard similarity indicated by the second threshold is lower than the standard similarity indicated by the first threshold. The industrial robotic arm system also includes a vibratory feeder unit. In the visual positioning step, when the matching results of the image information and at least two pre-stored template information do not meet the matching conditions, a vibration command is sent to the vibratory feeder unit.

2. The control method for the industrial robotic arm system according to claim 1, characterized in that, In the activation triggering step, in response to a single action by the user, a self-test command is sent to the robotic arm unit, the vision unit, and the control unit.

3. The control method for the industrial robotic arm system according to claim 1, characterized in that, In the visual positioning step, the first template information includes information about the first workpiece, and the second template information includes information about the second workpiece.

4. The control method for the industrial robotic arm system according to claim 1, characterized in that, In the visual positioning step, the first template information includes information about a first view of the first workpiece, and the second template information includes information about a second view of the first workpiece.

5. The control method for the industrial robotic arm system according to claim 1, characterized in that, In the visual positioning step, when the matching result does not meet the matching conditions, the light correlation parameters of the image information are first changed and corrected image information is generated. The corrected image information is then matched with the first template information. If the matching result still does not meet the matching conditions, the image information is then matched with the pre-stored second template information.

6. The control method for the industrial robotic arm system according to claim 1, characterized in that, In the visual positioning step, the image of the local area in the image information where there is suspected workpiece stacking is matched with the first template information. When the result of the second matching does not meet the second matching conditions, a vibration command is sent to the vibratory feeder unit.

7. The control method for the industrial robotic arm system according to claim 1, characterized in that, The control unit includes a graphical user interface module, on which both the image information and the visual positioning information are displayed.

8. The control method for the industrial robotic arm system according to claim 7, characterized in that, The visual positioning steps include: The interference removal step is in response to an instruction by the graphical user interface module to set a local masking region in the image information, so that the local masking region does not participate in the matching of the image information with the first template information.

9. The control method for the industrial robotic arm system according to claim 7, characterized in that, The task determination steps include: The testing steps involve controlling the robotic arm unit to execute the partial motion test command input through the graphical user interface module, and replacing the corresponding partial command in the motion control command with the partial motion test command in response to the replacement command input through the graphical user interface module.

10. The control method for the industrial robotic arm system according to claim 1, characterized in that, Also includes: The fault handling steps are as follows: first, determine the fault type; if the fault type is determined to be the first type that can be automatically recovered, enter the automatic recovery process; if the fault type is the second type that cannot be automatically recovered, output a prompt message.

11. The control method for the industrial robotic arm system according to claim 10, characterized in that, The control unit includes an analysis module. Input any of the normal operation data in the task execution steps, the fault data in the fault handling steps, and the technical documents of the industrial robotic arm system into the analysis module.

12. The control method for the industrial robotic arm system according to claim 11, characterized in that, The steps for determining the fault type include: Based on the first determination condition, the fault is divided into the second fault type and the pending fault type. The first determination condition is used to indicate whether the fault is related to hardware damage or safety. Based on the second determination condition, the faults of the pending fault types are divided into the first fault type and the second fault type. The second determination condition is used to indicate whether the probability value of automatic recovery of the faults of the pending fault types exceeds a threshold according to the evaluation of the analysis module.

13. The control method for the industrial robotic arm system according to claim 12, characterized in that, Also includes: The maintenance question-and-answer process involves responding to user inquiries by generating answer information for those inquiries. The maintenance process records steps, generates maintenance data in response to user maintenance operations, and sends it to the analysis module.

Images