Industrial vision technology application training device and system

Abstract

The application relates to the field of industrial robots, and discloses an industrial visual technology application practical training device and system, which comprises a first data acquisition module, a trajectory generation and control module, a second data acquisition module, a motion generation and control module, a local loop control module and a global loop control module.The first data acquisition module is used for acquiring first feature data of an i-th unassembled part; the trajectory generation and control module is used for inputting the first feature data into a path generation model to acquire an assembly movement trajectory of the i-th unassembled part; the second data acquisition module is used for acquiring second assembly feature data when reaching an r-th stagnation position; the motion generation and control module is used for inputting the second assembly feature data into an assembly motion generation model to acquire an assembly motion sequence; the local loop control module is used for repeating the second data acquisition module to the motion generation and control module; and the global loop control module is used for repeating the first data acquisition module to the local loop control module until i=N, and assembly demonstration of a machine to be assembled is completed; and the application is beneficial to training and improving the assembly capacity of practical training personnel.

Description

Technical Field

[0001] This invention relates to the field of industrial robots, and more specifically, to an industrial vision technology application training device and system. Background Technology

[0002] With the rapid development of the manufacturing industry and the continuous improvement of automation levels, industrial robots are increasingly widely used on production lines. However, the disassembly, assembly, and maintenance of industrial robots remains a complex and critical task, requiring high skills and experience from operators. Especially in the initial learning stage of industrial robot disassembly, assembly, and maintenance, it is often necessary to hire professional maintenance personnel for guidance, which has the limitations and disadvantages of high training costs and long training time. Therefore, designing an industrial vision technology application training device and system has become an urgent task.

[0003] Currently, there is a lack of training devices and systems for the disassembly, assembly, and maintenance of industrial robots. Although a few relevant documents exist, such as Chinese Patent No. CN113334408A which discloses a modular robot teaching and training platform control system, while the above systems and existing technologies can provide training methods, research and practical application of the above systems and existing technologies have revealed at least the following shortcomings:

[0004] (1) Lack of automated control makes it impossible to generate assembly movement trajectories for assembly robots based on the assembly parts sets of different models of industrial robots.

[0005] (2) It is impossible to generate assembly action sequences for the assembly robot based on the assembly movement trajectory, thus making it impossible to demonstrate the assembly of different models of industrial robots based on the assembly movement trajectory and assembly action sequences. Consequently, it is difficult to provide trainees with automated assembly training scenarios and make it difficult for trainees to have a more intuitive and clear understanding of industrial robots. Summary of the Invention

[0006] To overcome the above-mentioned deficiencies of the prior art, embodiments of the present invention provide an industrial vision technology application training device and system.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] An industrial vision technology application training system, used in the assembly demonstration phase, includes:

[0009] The first data acquisition module is used to acquire the first feature data of the i-th unassembled part in the unassembled parts set C according to the set assembly sequence number. Each unassembled part in the parts set C is located in the parts placement area, and i is an integer greater than zero.

[0010] The trajectory generation and control module is used to input the first feature data of the i-th unassembled part into the pre-configured path generation model to obtain the assembly movement trajectory of the i-th unassembled part, wherein the assembly movement trajectory includes R dwell positions.

[0011] The second data acquisition module is used to control the assembly robot to move according to the assembly movement trajectory of the i-th unassembled part, and to acquire the second assembly feature data when the assembly robot reaches the r-th dwell position during the movement.

[0012] The motion generation and control module is used to input the second assembly feature data into the pre-configured assembly motion generation model, obtain the assembly motion sequence, control the assembly robot to complete all motions according to the assembly motion sequence, and set r = r + 1 to trigger the second data acquisition module.

[0013] The local loop control module is used to repeat the process from the second data acquisition module to the action generation and control module until r = R, at which point the loop ends and i = i + 1 is set to trigger the first data acquisition module.

[0014] The global loop control module is used to repeat the process from the first data acquisition module to the local loop control module until i = N, at which point the loop ends and the assembly demonstration of the robot to be assembled is completed, where N is the total number of unassembled parts.

[0015] Furthermore, before obtaining the first feature data of the i-th unassembled part in the set of unassembled parts C, the process includes:

[0016] Obtain the assembly demonstration request from the trainee, wherein the assembly demonstration request contains at least the machine model of the robot to be assembled;

[0017] Based on the mapping relationship between machine model and part data, part data is obtained; the part data includes part set A and part annotation drawing set corresponding to part set A; part set A includes M assembly parts, and the part annotation drawing set includes part annotation drawing of each assembly part, and each part annotation drawing is marked with the corresponding assembly number of the corresponding part, where M is an integer greater than zero.

[0018] Furthermore, before obtaining the first feature data of the i-th unassembled part in the set of unassembled parts C, the process also includes:

[0019] a1: Obtain the global part drawing of the machine assembly area, and obtain W local part sub-drawings from the global part drawing according to the image processing method, where W is an integer greater than zero;

[0020] a2: Extract the v-th local part sub-image and the q-th part annotation image from the part annotation image set, where v and q are both integers greater than zero;

[0021] a3: Calculate the similarity between the v-th local part sub-image and the q-th part annotation image using the cosine similarity algorithm, and compare the similarity with the similarity threshold. If the similarity is greater than the similarity threshold, it is determined that the parts in the v-th local part sub-image and the q-th part annotation image are the same, and the assembly sequence number of the corresponding part is recorded. Then, v = v + 1 and return to step a2. If the similarity is less than or equal to the similarity threshold, it is determined that the parts in the v-th local part sub-image and the q-th part annotation image are not the same. Then, q = q + 1 and return to step a2.

[0022] a4: Repeat steps a2 to a3 until v = W, then end the loop. Record all parts corresponding to the assembly sequence number as assembled parts, resulting in W assembled parts. Add the W assembled parts to an empty set to obtain part set B.

[0023] Furthermore, the logic for obtaining the parts set C is as follows:

[0024] Extract part set B and part set A;

[0025] Based on all parts in part set B, identical parts in part set A are removed to obtain D different parts. These different parts are then treated as unassembled parts and added to an empty set to obtain part set C.

[0026] Furthermore, the first characteristic data of the i-th unassembled part includes the model of the robot to be assembled, the assembly sequence number of the assembled parts, the tool used for the i-th unassembled part, the order in which the tools are used, the coordinates of the machine assembly area, the coordinates of the part placement area, and the coordinates of the tool storage area.

[0027] Furthermore, the generation logic of the pre-configured path generation model is as follows:

[0028] Acquire historical assembly movement trajectory data and divide the historical assembly movement trajectory data into an assembly movement trajectory training set and an assembly movement trajectory test set; the historical assembly movement trajectory data includes the first feature data of the i-th unassembled part and its corresponding assembly movement trajectory.

[0029] Construct a first regression network, using the first feature data of the i-th unassembled part in the assembly movement trajectory training set as the input data of the first regression network, and the assembly movement trajectory in the assembly movement trajectory training set as the output data of the first regression network, and train the first regression network to obtain the initial path generation network.

[0030] The initial path generation network was validated using the assembly movement trajectory test set. The initial path generation network whose output is less than or equal to the test error was used as the pre-configured path generation model.

[0031] Furthermore, the second assembly feature data includes the assembly sequence number of the i-th unassembled part, the region to which the dwelling position belongs, and the dwelling sequence number of the dwelling position.

[0032] Furthermore, the logic of the pre-configured assembly action generation model is as follows:

[0033] Acquire historical assembly action data and divide the historical assembly action data into an assembly action training set and an assembly action test set; the historical assembly action data includes second assembly feature data and its corresponding assembly action sequence.

[0034] A second regression network is constructed. The second assembly feature data in the assembly action training set is used as the input data of the second regression network, and the assembly action sequence in the assembly action training set is used as the output data of the second regression network. The second regression network is trained to obtain the initial assembly action generation network.

[0035] The initial assembly action generation network is validated using an assembly action test set. The initial assembly action generation network whose output is less than or equal to the test error is used as the pre-configured assembly action generation model.

[0036] An industrial vision technology application training device, the device including an industrial robot training platform, wherein the industrial robot training platform is equipped with any of the modules or devices in the industrial vision technology application training system described above.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] This application discloses an industrial vision technology application training device and system, comprising: a first data acquisition module for acquiring first feature data of the i-th unassembled part; a trajectory generation and control module for inputting the first feature data into a path generation model to acquire the assembly movement trajectory of the i-th unassembled part; a second data acquisition module for acquiring second assembly feature data when the part reaches the r-th dwell position; an action generation and control module for inputting the second assembly feature data into an assembly action generation model to acquire an assembly action sequence; a local loop control module for repeating the above steps from the second data acquisition module to the action generation and control module; and a global loop control module for repeating the above steps from the first data acquisition module to the local loop control module until i = N, thus completing the process. Assembly robot assembly demonstration; this invention is beneficial for training trainees' assembly skills; based on the above modules, this invention can generate assembly movement trajectories for assembly robots according to the assembly parts sets of different models of industrial robots. Furthermore, this invention can generate assembly action sequences for assembly robots based on the assembly movement trajectories, thereby facilitating the assembly demonstration of different models of industrial robots based on the assembly movement trajectories and assembly action sequences. This, in turn, is beneficial for providing trainees with automated assembly training scenarios. Furthermore, it helps trainees gain a more intuitive and clear understanding of industrial robots and an intuitive understanding of the working principles and characteristics of the robot's core components, thereby training trainees' ability to read drawings, use tools, and adjust assembly processes. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the industrial vision technology application training system provided by the present invention;

[0040] Figure 2 The flowchart shows the industrial vision technology application training method provided by this invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1

[0043] Please see Figure 1 As shown, this embodiment discloses an industrial vision technology application training system, applied in the assembly demonstration stage. The system includes:

[0044] The first data acquisition module 110 is used to acquire the first feature data of the i-th unassembled part in the unassembled parts set C according to the set assembly sequence number. Each unassembled part in the parts set C is located in the parts placement area, and i is an integer greater than zero.

[0045] It should be noted that this embodiment relies on an industrial vision technology application training device equipped with an assembly robot. This device includes a training platform, which is pre-divided into a machine assembly area, a parts placement area, and a tool storage area. The parts placement area is used to place all the assembly parts for the robot to be assembled. The machine assembly area provides an assembly space for the robot. The tool storage area stores the assembly tools required for assembling the robot, including but not limited to wrenches, screwdrivers, screws, and nuts. The assembly robot reciprocates between the machine assembly area, the parts placement area, and the tool storage area to assemble the robot.

[0046] Specifically, before obtaining the first feature data of the i-th unassembled part in the set of unassembled parts C, the following steps are included:

[0047] Obtain the assembly demonstration request from the trainee, wherein the assembly demonstration request contains at least the machine model of the robot to be assembled;

[0048] Based on the mapping relationship between machine model and part data, part data is obtained; the part data includes part set A and part annotation drawing set corresponding to part set A; part set A includes M assembly parts, and the part annotation drawing set includes part annotation drawing of each assembly part, and each part annotation drawing is marked with the corresponding assembly number of the corresponding part, where M is an integer greater than zero;

[0049] It should be noted that the system database pre-stores several mapping relationships between machine models and part data, which means that there are several machine models of industrial robots, and each machine model of industrial robot is associated with a unique part data; the part data is obtained according to the pre-set, and the part data details the assembly part data and assembly sequence number of the corresponding robot model.

[0050] It should also be noted that each part annotation drawing corresponds to an assembly part, which is a standard photograph of the corresponding assembly part. The assembly number of the corresponding assembly part is marked on the part annotation drawing.

[0051] Specifically, before obtaining the first feature data of the i-th unassembled part in the set of unassembled parts C, the process also includes:

[0052] a1: Obtain the global part drawing of the machine assembly area, and obtain W local part sub-drawings from the global part drawing according to the image processing method, where W is an integer greater than zero;

[0053] It should be noted that the image processing method is specifically one of the following: edge detection algorithm, threshold segmentation, K-means clustering, or Hough transform. The edge detection algorithm is specifically one of the following: Sobe l operator, Canny operator, or Prewitt operator.

[0054] a2: Extract the v-th local part sub-image and the q-th part annotation image from the part annotation image set, where v and q are both integers greater than zero;

[0055] a3: Calculate the similarity between the v-th local part sub-image and the q-th part annotation image using the cosine similarity algorithm, and compare the similarity with the similarity threshold. If the similarity is greater than the similarity threshold, it is determined that the parts in the v-th local part sub-image and the q-th part annotation image are the same, and the assembly sequence number of the corresponding part is recorded. Then, v = v + 1 and return to step a2. If the similarity is less than or equal to the similarity threshold, it is determined that the parts in the v-th local part sub-image and the q-th part annotation image are not the same. Then, q = q + 1 and return to step a2.

[0056] It should be understood that when the parts in the vth local part sub-drawing and the qth part annotation drawing are the same, it means that the parts in the two drawings are the same. In other words, the assembly sequence numbers of the parts in the vth local part sub-drawing and the qth part annotation drawing are also the same.

[0057] a4: Repeat steps a2 to a3 until v = W, then end the loop. Record all parts corresponding to the assembly sequence number as assembled parts, and obtain W assembled parts. Add the W assembled parts to an empty set to obtain part set B.

[0058] In implementation, the logic for obtaining the parts set C is as follows:

[0059] Extract part set B and part set A;

[0060] Based on all parts in part set B, remove identical parts from part set A to obtain D different parts. Treat the different parts as unassembled parts and add them to the empty set to obtain part set C.

[0061] For example, suppose there are three types of industrial robots, A1, A2 and A3. If A1 is selected by the trainee as the industrial robot to be assembled, then A1 is the robot to be assembled. Let's assume that the set of parts A of A1 is {part 1, part 2, part 3, part 4}, and the set of parts B is {part 1, part 2}. Then, D different parts include part 3 and part 4, so the set of parts C is {part 3, part 4}. It should be noted that the subscripts 1, 2, 3 and 4 in the set represent the assembly sequence number of the parts.

[0062] It should be understood that the first characteristic data of the i-th unassembled part in the unassembled parts set C is obtained in the order of the set assembly sequence number. For example, following the above assumption, if the parts set C is {part 3, part 4}, then part 3 will be taken as the i-th unassembled part first, and part 4 will not be taken as the i-th unassembled part first.

[0063] Specifically, the first feature data of the i-th unassembled part includes the model of the robot to be assembled, the assembly sequence number of the assembled parts, the tool used for the i-th unassembled part, the order in which the tools are used, the coordinates of the machine assembly area, the coordinates of the part placement area, and the coordinates of the tool storage area.

[0064] It should be noted that the tools used for the i-th unassembled part and the order in which the tools are used are determined according to the preset relationship between the assembly sequence number and the usage data. To further explain, there are multiple preset relationships between assembly sequence numbers and usage data stored in the system database. Each preset relationship between assembly sequence number and usage data is set in advance through experiments. This means that each part with an assembly sequence number is associated with a set of usage data, which includes the usage data of the part with the corresponding assembly sequence number and the order in which the tools are used.

[0065] The trajectory generation and control module 120 is used to input the first feature data of the i-th unassembled part into the pre-configured path generation model to obtain the assembly movement trajectory of the i-th unassembled part, wherein the assembly movement trajectory includes R dwell positions.

[0066] Specifically, the generation logic of the pre-configured path generation model is as follows:

[0067] Acquire historical assembly movement trajectory data and divide the historical assembly movement trajectory data into an assembly movement trajectory training set and an assembly movement trajectory test set; the historical assembly movement trajectory data includes the first feature data of the i-th unassembled part and its corresponding assembly movement trajectory.

[0068] It should be noted that the assembly movement trajectory in the historical assembly movement trajectory data was obtained through experimental records, which will not be elaborated on further.

[0069] Construct a first regression network, using the first feature data of the i-th unassembled part in the assembly movement trajectory training set as the input data of the first regression network, and the assembly movement trajectory in the assembly movement trajectory training set as the output data of the first regression network, and train the first regression network to obtain the initial path generation network.

[0070] The initial path generation network was validated using the assembly movement trajectory test set. The initial path generation network whose output is less than or equal to the test error was used as the pre-configured path generation model.

[0071] It should be noted that the first regression network is specifically one of the following models: decision tree regression, support vector machine regression, random forest regression, recurrent neural network, and convolutional neural network.

[0072] It should be understood that the assembly movement trajectory includes R dwell positions, each of which belongs to only one of the machine assembly area, parts placement area, and tool storage area. Each dwell position is assigned a position number according to the operation sequence. Similar to the assembly movement trajectory, the R dwell positions are determined based on a prior experimental process. It can be understood that the robot will perform corresponding operations at each dwell position. To further explain, the robot will stop and complete the corresponding action sequence in the machine assembly area, parts placement area, and tool storage area.

[0073] An illustrative example is as follows: Continuing with the above assumptions, if A1, as the robot to be assembled, has already completed the assembly of part 1 and part 2, then the robotic arm will proceed with the assembly of part 3. When assembling part 3, the robotic arm will first move to the first dwell position (the part placement area) according to the assembly movement trajectory. Then, based on the action sequence of the first dwell position (the action sequence will be described later), it will complete the retrieval of part 3 in the part placement area. Afterward, it will continue to move part 3 to the second dwell position (the machine assembly area) according to the assembly movement trajectory. Then, based on the second... The action sequence of the dwell position of the dwell number completes the insertion operation of part 3 in the machine assembly area (that is, placing part 3 into the hole slot of the assembled part). Then, the robot will move to the dwell position of the third dwell number according to the assembly movement trajectory, which is the tool storage area. Then, according to the action sequence of the dwell position of the third dwell number, the robot will complete the picking action of the corresponding tool in the tool storage area. Finally, the robot will move to the dwell position of the fourth dwell number according to the assembly movement trajectory, which is still in the machine assembly area. Then, according to the action sequence of the dwell position of the fourth dwell number, the robot will complete the fastening action of part 3 in the machine assembly area by using the tool.

[0074] The second data acquisition module 130 is used to control the assembly robot to move according to the assembly movement trajectory of the i-th unassembled part, and to acquire the second assembly feature data when the assembly robot reaches the r-th dwell position during the movement.

[0075] Specifically, the second assembly feature data includes the assembly sequence number of the i-th unassembled part, the region to which the dwell position belongs, and the dwell sequence number of the dwell position;

[0076] It should be understood that the area to which the dwelling location belongs includes one of the machine assembly area, parts placement area, and tool storage area, and each area has its own predefined boundaries; the dwelling sequence number of the dwelling location is obtained through pre-setting.

[0077] The motion generation and control module 140 is used to input the second assembly feature data into the pre-configured assembly motion generation model, obtain the assembly motion sequence, control the assembly robot to complete all motions according to the assembly motion sequence, and set r = r + 1 to trigger the second data acquisition module 130.

[0078] Specifically, the logic for generating the pre-configured assembly action model is as follows:

[0079] Acquire historical assembly action data and divide the historical assembly action data into an assembly action training set and an assembly action test set; the historical assembly action data includes second assembly feature data and its corresponding assembly action sequence.

[0080] It should be noted that the assembly action sequence in the historical assembly action data was obtained through experimental records, which will not be elaborated on further; the assembly action sequence contains multiple assembly actions, including but not limited to grasping actions, moving actions, plugging and unplugging actions, and turning actions, etc.

[0081] A second regression network is constructed. The second assembly feature data in the assembly action training set is used as the input data of the second regression network, and the assembly action sequence in the assembly action training set is used as the output data of the second regression network. The second regression network is trained to obtain the initial assembly action generation network.

[0082] The initial assembly action generation network is validated using an assembly action test set. The initial assembly action generation network whose output is less than or equal to the test error is used as the pre-configured assembly action generation model.

[0083] It should be noted that, similar to the first regression network mentioned above, the second regression network is specifically one of the following models: decision tree regression, support vector machine regression, random forest regression, recurrent neural network, and convolutional neural network.

[0084] The local loop control module 150 is used to repeat the process from the second data acquisition module 130 to the action generation and control module 140 until r = R, at which point the loop ends and i = i + 1 is set to trigger the first data acquisition module 110.

[0085] It should be understood that: the second assembly feature data of the assembly robot when it reaches the r-th dwell position is repeatedly obtained in a loop, and the assembly robot is controlled to complete all actions according to the assembly action sequence until all actions of all dwell positions are executed, only then can the entire assembly process of a certain part of the robot be realized.

[0086] The global loop control module 160 is used to repeat the above-mentioned first data acquisition module 110 to the local loop control module 150 until i=N, then the loop ends and the assembly demonstration of the robot to be assembled is completed, where N is the total number of unassembled parts.

[0087] It should be understood that the assembly process of each part of the assembly robot is repeated in a loop until the last part is reached, at which point the entire assembly process of the assembly robot and the assembly demonstration process are completed.

[0088] Example 2

[0089] Please see Figure 2 As shown, based on the same inventive concept, this embodiment discloses a training method for industrial vision technology applications, the method comprising:

[0090] S201: Obtain the first feature data of the i-th unassembled part in the unassembled parts set C according to the set assembly sequence number, where each unassembled part in the parts set C is located in the parts placement area, and i is an integer greater than zero.

[0091] S202: Input the first feature data of the i-th unassembled part into the pre-configured path generation model to obtain the assembly movement trajectory of the i-th unassembled part, wherein the assembly movement trajectory includes R dwell positions.

[0092] S203: Control the assembly robot to move according to the assembly movement trajectory of the i-th unassembled part, and acquire the second assembly feature data when the assembly robot reaches the r-th dwell position during the movement.

[0093] S204: Input the second assembly feature data into the pre-configured assembly action generation model, obtain the assembly action sequence, control the assembly robot to complete all actions according to the assembly action sequence, and let r = r + 1, return to step S203.

[0094] S205: Repeat steps S203 to S204 until r = R, then end the loop and set i = i + 1, returning to step S201;

[0095] S206: Repeat steps S201 to S205 until i = N, then end the loop and complete the assembly demonstration of the robot to be assembled, where N is the total number of unassembled parts.

[0096] It should be understood that the assembly process of each part of the assembly robot is repeated in a loop until the last part is reached, at which point the entire assembly process of the assembly robot and the assembly demonstration process are completed.

[0097] Example 3

[0098] Based on the above embodiment 1, this embodiment discloses an industrial vision technology application training device, the device including an industrial robot training platform, and the industrial robot training platform is additionally equipped with any of the modules or devices in the industrial vision technology application training system described above.

[0099] It should be noted that the industrial robot training platform is an existing device used for demonstration and training of industrial robot assembly. For example, the industrial robot training platform is specifically one of HY-CZ-01, HY-CZ-02 or HY-CZ-03.

[0100] It should be understood that by integrating the hardware and software systems developed in this application into existing industrial robot training platforms, it is beneficial to realize the assembly demonstration of various types of industrial robots, thereby enabling trainees to have a more intuitive and clear understanding of industrial robots, and to have an intuitive understanding of the working principles and characteristics of the core components of the robot, and to train trainees' ability to read drawings, use tools and adjust processes.

[0101] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0102] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0103] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0104] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0107] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0108] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An industrial vision technology application training system, characterized in that, The system is used in the assembly demonstration phase and includes: The first data acquisition module is used to acquire the first feature data of the i-th unassembled part in the set of unassembled parts C according to the set assembly sequence number. Each unassembled part in the set of parts C is located in the part placement area, and i is an integer greater than zero. The first feature data of the i-th unassembled part includes the model of the robot to be assembled, the assembly sequence number of the assembled parts, the tool used for the i-th unassembled part, the order in which the tools are used, the coordinates of the machine assembly area, the coordinates of the part placement area, and the coordinates of the tool storage area. Before obtaining the first feature data of the i-th unassembled part in the set of unassembled parts C, the following is included: Obtain the assembly demonstration request from the trainee, which includes at least the model number of the robot to be assembled; obtain the part data according to the mapping relationship between the machine model and the part data; the part data includes a part set A and a part annotation drawing set corresponding to part set A; the part set A includes M assembly parts, and the part annotation drawing set includes a part annotation drawing for each assembly part, and each part annotation drawing is marked with the corresponding assembly sequence number of the corresponding part, where M is an integer greater than zero; Before obtaining the first feature data of the i-th unassembled part in the set of unassembled parts C, the following steps are also included: a1: Obtain the global part image of the machine assembly area, and obtain W local part sub-images from the global part image according to the image processing method, where W is a positive integer; a2: Extract the v-th local part sub-image and the q-th part annotation image from the part annotation image set, where v and q are both positive integers; a3: Calculate the similarity between the v-th local part sub-image and the q-th part annotation image according to the cosine similarity algorithm, and compare the similarity with a similarity threshold. If the similarity is greater than the similarity threshold, then the v-th local part sub-image and the q-th part annotation image are considered similar. If the parts in the part annotation diagram are consistent, record the assembly sequence number of the corresponding parts, set v=v+1, and return to step a2; if the similarity is less than or equal to the similarity threshold, it is determined that the parts in the v-th local part sub-graph and the q-th part annotation diagram are inconsistent, then set q=q+1, and return to step a2; a4: Repeat the above steps a2~a3 until v=W, then end the loop, take all the corresponding parts recorded with the assembly sequence number as assembled parts, get W assembled parts, add the W assembled parts to the empty set, and get the part set B; The logic for obtaining the parts set C is as follows: Extract part set B and part set A; based on all parts in part set B, remove identical parts from part set A to obtain D different parts. Treat the different parts as unassembled parts and add them to the empty set to obtain part set C. The trajectory generation and control module is used to input the first feature data of the i-th unassembled part into a pre-configured path generation model to obtain the assembly movement trajectory of the i-th unassembled part, wherein the assembly movement trajectory includes R dwell positions; the generation logic of the pre-configured path generation model is as follows: Historical assembly movement trajectory data is acquired and divided into an assembly movement trajectory training set and an assembly movement trajectory test set. The historical assembly movement trajectory data includes the first feature data of the i-th unassembled part and its corresponding assembly movement trajectory. A first regression network is constructed, using the first feature data of the i-th unassembled part in the assembly movement trajectory training set as input data and the assembly movement trajectory in the training set as output data. The first regression network is trained to obtain an initial path generation network. The initial path generation network is validated using the assembly movement trajectory test set, and the initial path generation network whose output error is less than or equal to the test error is used as a pre-configured path generation model. The second data acquisition module is used to control the assembly robot to move according to the assembly movement trajectory of the i-th unassembled part, and during the movement, acquire the second assembly feature data when the assembly robot reaches the r-th dwell position; the second assembly feature data includes the assembly sequence number of the i-th unassembled part, the region to which the dwell position belongs, and the dwell sequence number of the dwell position. The action generation and control module is used to input the second assembly feature data into a pre-configured assembly action generation model, obtain the assembly action sequence, control the assembly robot to complete all actions according to the assembly action sequence, and set r=r+1 to trigger the second data acquisition module; the logic of the pre-configured assembly action generation model is as follows: Historical assembly action data is acquired and divided into an assembly action training set and an assembly action test set. The historical assembly action data includes second assembly feature data and its corresponding assembly action sequences. A second regression network is constructed, using the second assembly feature data from the assembly action training set as input data and the assembly action sequences from the training set as output data. The second regression network is then trained to obtain an initial assembly action generation network. The initial assembly action generation network is validated using the assembly action test set, and the initial assembly action generation network whose output error is less than or equal to the test error is used as a pre-configured assembly action generation model. The local loop control module is used to repeat the process from the second data acquisition module to the action generation and control module until r=R, at which point the loop ends and i=i+1 is set to trigger the first data acquisition module. The global loop control module is used to repeat the process from the first data acquisition module to the local loop control module until i=N, at which point the loop ends, completing the assembly demonstration of the robot to be assembled, where N is the total number of unassembled parts.

2. An industrial vision technology application training device, the device comprising an industrial robot training platform, characterized in that, The industrial robot training platform is equipped with a module or device from the industrial vision technology application training system of claim 1.

Images