Method, device and electronic equipment for creating a model for material picking and material picking position determination

Abstract

The present application provides a method, device and electronic device for creating a material picking and material picking position determination model, wherein the method includes: inputting a workpiece image obtained at a theoretical workpiece position and a work pile image obtained at a theoretical work pile position into a trained material picking position determination model, obtaining the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical work pile position; determining the actual grasping position of the workpiece according to the deviation correction coordinate value of the theoretical workpiece position; determining the workpiece return position according to the deviation correction coordinate value of the theoretical work pile position and the actual grasping position of the workpiece, so that the material picking device takes the workpiece out of the work pile at the workpiece return position. The present application determines the actual grasping position and return position of the workpiece according to the workpiece image and the work pile image when picking materials, so as to ensure that the workpiece is taken out from a specific position of the work pile when picking materials, prevents the workpiece and the work pile from being scratched, and improves the safety of the workpiece when picking materials.

Description

Technical Field

[0001] The present application relates to the field of workpiece material picking, and in particular, to a method, device and electronic device for creating a material picking and material picking position determination model. Background Art

[0002] At present, many workpieces are loaded and unloaded by robots instead of humans. Fixed work piles are usually set on the trolley for placing workpieces, and each work pile is placed with a corresponding workpiece. In order to prevent the workpiece from moving on the work pile, buckles are usually set on the work pile to fix the workpiece on the work pile. However, when the robot removes the workpiece from the trolley, there is often a scratch between the workpiece and the buckle, causing damage to the workpiece. Summary of the invention

[0003] In view of this, the purpose of the embodiments of the present application is to provide a method, device and electronic device for creating a material picking and material picking position determination model, which can avoid scratches between the workpiece and the buckle and prevent the workpiece from being damaged.

[0004] In a first aspect, an embodiment of the present application provides a material picking method, comprising: inputting a workpiece image acquired at a theoretical workpiece position and a work pile image acquired at a theoretical work pile position into a trained material picking position determination model to obtain a correction coordinate value of the theoretical workpiece position and a correction coordinate value of the theoretical work pile position; determining the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position; determining the workpiece return position according to the correction coordinate value of the theoretical work pile position and the actual grasping position of the workpiece, so that a material picking device can pick up the workpiece from the work pile at the workpiece return position; wherein the material picking position determination model comprises a workpiece image sub-model and a work pile image sub-model; the workpiece image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical workpiece position; and the work pile image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical work pile position.

[0005] In the above implementation process, the workpiece image obtained at the theoretical workpiece position and the work pile image obtained at the theoretical work pile position are input into the trained material picking position determination model to obtain the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical work pile position. The theoretical work pile position of the workpiece is compensated by the deviation correction coordinate value of the theoretical workpiece position to determine the actual grasping position of the workpiece, and then the actual grasping position of the workpiece is compensated according to the deviation correction coordinate value of the theoretical work pile position to determine the return position of the workpiece, which is the position where the workpiece is taken out from the work pile without being scratched. By taking out the workpiece at the return position, it is ensured that the workpiece is not scratched when picking up the material, and the workpiece is prevented from being damaged.

[0006] In one embodiment, in the material picking method, the material picking device includes a photographing posture and a material picking posture, and the material picking device switches from a photographing posture to a material picking posture at the theoretical work pile position; the actual grasping position of the workpiece is determined according to the correction coordinate value of the theoretical workpiece position, including: determining the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position and the posture switching difference; wherein, the posture switching difference is the coordinate difference of the material picking device switching from the photographing posture to the material picking posture.

[0007] In the above implementation process, since the material picking device switches from the shooting posture to the material picking posture, the conversion between postures will cause the position of the device to change when picking the material. When the material picking device is grasping the workpiece, it is necessary to switch the posture from the shooting posture to the material picking posture. When calculating the actual grasping position of the workpiece, by adding compensation for the posture switching difference, the error generated by the material picking device during the posture conversion can be compensated, ensuring that the actual grasping position of the workpiece fits the actual situation and improving the accuracy of determining the actual grasping position of the workpiece.

[0008] In one embodiment, the formula for determining the workpiece return position is: F=E+ΔA-ΔB; wherein F is the workpiece return position, E is the actual grasping position of the workpiece, ΔA is the correction coordinate value of the theoretical workpiece position, and ΔB is the correction coordinate value of the theoretical workpiece position.

[0009] In the above implementation process, the photographing position and the working pile position of the workpiece are compensated respectively by the correction coordinate value of the theoretical workpiece position and the correction coordinate value of the theoretical working pile position, thereby eliminating the position error of the workpiece photographing position and the working pile position, and then the workpiece return position obtained based on the actual grasping position of the workpiece is more accurate, thereby increasing the accuracy of the workpiece return position.

[0010] In a second aspect, an embodiment of the present application further provides a method for creating a model for determining a material picking position, comprising: controlling a material picking device to acquire a workpiece image at a reference position at a first preset position, so as to establish a workpiece image sub-model through the workpiece image at the reference position; controlling the material picking device to acquire a work pile image at a second preset position, so as to establish a work pile image sub-model through the work pile image at the second preset position; wherein the second preset position is obtained by the material picking device being offset by a preset distance from the first preset position.

[0011] In the above implementation process, by acquiring the workpiece image at the reference position at the first preset position, the standard placement image of the workpiece can be obtained, and by acquiring the work pile image at the second preset position, the standard placement image of the work pile can be obtained, and then the workpiece image sub-model and the work pile image sub-model are respectively established through the standard placement image of the workpiece and the standard placement image of the work pile, so as to realize the automatic modeling of the workpiece image and the work pile image, thereby reducing the workload of modeling.

[0012] In one embodiment, in the creation of the material picking position determination model, the material picking device includes a photo taking posture and a material picking posture, and the material picking position determination model creation method also includes: controlling the material picking device to switch from a photo taking posture to a material picking posture at the target position to obtain the current coordinate value of the material picking device; determining the posture switching difference based on the current coordinate value and the target position.

[0013] In the above implementation process, the material picking device is converted from the photographing posture to the material picking posture at the same position, and the posture conversion value of the material picking device is determined by the different positions of the material picking device under the two postures. Since the material picking device only performs the posture conversion action at this position, the position change between the target position and the current coordinate value is caused by the posture conversion, and the accuracy of the posture conversion value calculation is achieved.

[0014] In one embodiment, determining the posture switching difference according to the current coordinate value and the target position includes: determining the posture switching difference by subtracting the target position from the current coordinate value, and the posture switching difference includes coordinate differences of at least three degrees of freedom.

[0015] In the above implementation process, the posture switching difference is determined by subtracting the target position from the current coordinate value, thereby realizing the posture switching difference calculation. Since the position change between the target position and the current coordinate value is caused by the posture conversion, the accuracy of the posture conversion value calculation is achieved.

[0016] In one embodiment, before the material picking device is controlled to acquire the workpiece image at the reference position at a first preset position so as to establish a workpiece image sub-model through the workpiece image at the reference position, the material picking position determination model creation method further includes: establishing a data relationship between the camera coordinates and the material picking device clamping device coordinates through a nine-point calibration method.

[0017] In the above implementation process, the nine-point calibration method is used to establish a connection between the coordinates of the clamping device and the image acquisition device, ensuring that the coordinates of the clamping device and the coordinate values ​​obtained by the image acquisition device can be converted to control the movement of the material picking device through the actual position of the workpiece, ensuring the information sharing between the image acquisition device and the clamping device, and realizing the automatic model creation of the material picking equipment.

[0018] In a third aspect, an embodiment of the present application further provides a material picking device, comprising: a first acquisition module, used to input a workpiece image acquired at a theoretical workpiece position and a work pile image acquired at a theoretical work pile position into a trained material picking position determination model to obtain the correction coordinate value of the theoretical workpiece position and the correction coordinate value of the theoretical work pile position; a first determination module, used to determine the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position; a second determination module, used to determine the workpiece return position according to the correction value of the theoretical work pile position and the actual grasping position of the workpiece, so that the material picking device can take the workpiece out of the work pile at the workpiece return position; wherein the material picking position determination model comprises a workpiece image sub-model and a work pile image sub-model; the workpiece image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical workpiece position; the work pile image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical work pile position.

[0019] In a fourth aspect, an embodiment of the present application further provides a device for creating a model for determining a material picking position, comprising: a second acquisition module, used to control a material picking device to acquire a workpiece image at a reference position at a first preset position, so as to establish a workpiece image sub-model through the workpiece image at the reference position; a third acquisition module, used to control a material picking device to acquire a work pile image at a second preset position, so as to establish a work pile image sub-model through the work pile image at the second preset position; wherein the second preset position is obtained by the material picking device being offset by a preset distance from the first preset position.

[0020] In a fifth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method in the above-mentioned first aspect, or any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0021] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above-mentioned first aspect, or any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect are executed.

[0022] In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, embodiments are given below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of the interaction between the controller and the local terminal provided in an embodiment of the present application;

[0025] Figure 2 A block diagram of a controller provided in an embodiment of the present application;

[0026] Figure 3 A flow chart of a material taking method provided in an embodiment of the present application;

[0027] Figure 4 A flow chart of a method for creating a material taking position determination model provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the functional modules of the material taking device provided in the embodiment of the present application;

[0029] Figure 6 A schematic diagram of the functional modules of the device for creating a material taking position determination model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0031] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0032] With the rapid development of industrial technology, artificial intelligence and other aspects, in automated production, robot loading and unloading has gradually replaced manual loading. On the one hand, robot loading and unloading can reduce manual configuration and save labor costs; on the other hand, robot loading and unloading is based on automated devices such as manipulators and robots for assembly, which increases the assembly time and improves assembly efficiency. Although the robot loading and unloading is widely used in various industries and workshops, the workpiece scratches caused by the robot loading and unloading process should not be underestimated. At the least, it affects the appearance of the workpiece, and at the worst, it damages the workpiece and seriously affects the quality of the workpiece.

[0033] In view of this, the inventor of the present application proposes a material picking method, which obtains the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical working pile position by inputting the workpiece image obtained at the theoretical workpiece position and the working pile image obtained at the theoretical working pile position into the trained material picking position determination model. Then, the workpiece return position is determined according to the deviation correction value of the theoretical working pile position and the actual grasping position of the workpiece, so that the material picking device can take the workpiece out of the working pile at the workpiece return position to avoid the workpiece being scratched, prevent the workpiece and the working pile from being scratched, and improve the safety of the workpiece when picking.

[0034] To facilitate understanding of this embodiment, the operating environment of a method for creating a material picking and material picking position determination model disclosed in an embodiment of the present application is first introduced in detail.

[0035] like Figure 1 , which is a schematic diagram of the interaction between the controller and the local terminal provided in an embodiment of the present application. The controller 100 is connected to one or more local terminals through a network for data communication or interaction. The controller 100 may be a network controller, a database controller, etc. The local terminal here may be a material collection device 200, and the material collection device 200 may include an image acquisition device 201 and a clamping device 202, etc. The controller 100 is connected to the image acquisition device 201 and the clamping device 202 through a network.

[0036] The above-mentioned image acquisition device 201 can be one or more devices such as a camera, a video camera, a laser acquisition device, etc. The image acquisition device 201 is used to acquire image information of a target workpiece, a mounting device, etc.

[0037] The above-mentioned clamping device 202 can be an industrial automation device for clamping a target workpiece. For example, the clamping device 202 can be a mechanical arm, a mechanical hand, a robot, etc. The clamping device 202 is used to install the target workpiece at a target position.

[0038] In some embodiments, the image acquisition device 201 can be arranged on the clamping device 202, and the material picking device 200 includes a material picking posture and a photo taking posture.

[0039] It can be understood that the workpiece is placed on the work pile of the transport trolley. When the material-retrieving device 200 needs to take the workpiece from the work pile, the material-retrieving device 200 obtains the position information sent by the controller 100, moves to the theoretical work pile position according to the position information, and obtains the work pile image through the image acquisition device 201 to send the work pile image to the controller 100. The controller 100 determines the correction coordinate value of the theoretical work pile position according to the work pile image. At the same time, after obtaining the work pile image, the image acquisition device 201 moves to the theoretical workpiece position to obtain the workpiece image, and sends the workpiece image to the controller 100. The controller 100 determines the actual grasping position of the workpiece according to the work pile image. The controller 100 determines the workpiece return position according to the correction coordinate value of the theoretical work pile position and the actual grasping position of the workpiece, so that the material-retrieving device can take the workpiece from the work pile at the workpiece return position.

[0040] The controller 100 mentioned above may be a PLC (Programmable Logic Controller, Chinese name: Programmable Logic Controller), a host computer, a single-chip microcomputer, etc.

[0041] Optionally, the controller 100 and the material retrieving device 200 may be the same device or two different devices. The controller 100 and the material retrieving device 200 may be configured according to actual conditions and this application does not impose any specific restrictions.

[0042] To facilitate understanding of this embodiment, a controller that executes the workpiece installation original position determination method and the workpiece installation method disclosed in the embodiment of the present application is introduced in detail below.

[0043] like Figure 2 , which is a block diagram of a controller. The controller 100 may include a memory 111, a storage controller 112, a processor 113, and a peripheral interface 114. A person skilled in the art may understand that Figure 2 The structure shown is only for illustration and does not limit the structure of the controller 100. For example, the controller 100 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations shown.

[0044] The memory 111, storage controller 112, processor 113 and peripheral interface 114 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The processor 113 is used to execute the executable module stored in the memory.

[0045] The memory 111 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 111 is used to store programs, and the processor 113 executes the program after receiving the execution instruction. The method executed by the controller 100 defined by the process disclosed in any embodiment of the present application can be applied to the processor 113, or implemented by the processor 113.

[0046] The processor 113 may be an integrated circuit chip with signal processing capability. The processor 113 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0047] The peripheral interface 114 couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113 and the memory controller 112 can be implemented in a single chip. In other embodiments, they can be implemented by separate chips.

[0048] To facilitate understanding of this embodiment, the implementation process of the material taking method is first described in detail through several embodiments.

[0049] See also Figure 3 , is a flow chart of the material taking method provided in the embodiment of the present application. Figure 1 The specific process shown is described in detail.

[0050] Step S201, inputting the workpiece image acquired at the theoretical workpiece position and the work pile image acquired at the theoretical work pile position into the trained material picking position determination model to obtain the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical work pile position.

[0051] Among them, the material picking position determination model includes a workpiece image sub-model and a work pile image sub-model; the workpiece image sub-model is used to determine the correction coordinate value of the material picking equipment at the theoretical workpiece position; the work pile image sub-model is used to determine the correction coordinate value of the material picking equipment at the theoretical work pile position.

[0052] The workpiece here is the workpiece to be removed from the work pile, for example, an iron core, a magnetic steel, etc.

[0053] The above-mentioned theoretical workpiece position is the theoretical workpiece placement position sent by the controller to the material reclaiming device, and the theoretical work pile position is the position of the material reclaiming device after it moves downward by the height of the workpiece from the theoretical workpiece placement position.

[0054] The deviation correction coordinate value of the theoretical workpiece position here is the offset between the current workpiece position and the theoretical workpiece position. The deviation correction coordinate value of the theoretical work pile position is the offset between the current work pile position and the theoretical work pile position.

[0055] In actual workpiece installation, after the workpiece is placed on the work pile, the position of the workpiece may shift when it is placed or during the movement of the trolley. Therefore, there is a certain offset between the theoretical workpiece position and the actual workpiece position.

[0056] In some embodiments, before step S201, the method further includes: sending the theoretical workpiece position to a material picking device, so that the material picking device moves to the theoretical workpiece position.

[0057] After the material-grabbing device moves to the theoretical workpiece position, it will acquire a workpiece image and send the workpiece image to the controller. After acquiring the workpiece image, the controller controls the material-grabbing device to deviate downward from the theoretical workpiece position by a preset distance (i.e., move to the theoretical work pile position) and acquire a work pile image at the theoretical work pile position.

[0058] The preset distance here can be the height of the workpiece or a set distance. For example, the preset distance can be 10 cm, 20 cm, 50 cm, etc. The preset distance can be in the height direction or in the displacement direction. The preset distance can be determined according to the characteristics of the image acquisition device, the workpiece, the clamping device, etc., and this application does not make specific limitations.

[0059] Step S202, determining the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position.

[0060] It can be understood that the theoretical work pile position is the theoretical grasping position of the workpiece. Since the workpiece may be offset in actual situations, there is a certain deviation between the actual grasping position of the workpiece and the theoretical grasping position. The deviation can be compensated by the offset position of the theoretical workpiece position at the theoretical grasping position of the workpiece to obtain the actual grasping position of the workpiece.

[0061] In some embodiments, the actual gripping position of the workpiece can be obtained by subtracting the correction coordinate value of the theoretical workpiece position from the theoretical workpiece position.

[0062] For example, if the theoretical pile position is (a 1 , b 1 , c 1 , w 1 , p 1 , r 1 ), the correction coordinate value of the theoretical workpiece position is (Δa, Δb, Δc, Δw, Δp, Δr), then the actual grasping position of the workpiece is (a 1 -Δa,b 1 -Δb,c 1 -Δc,w 1 -Δw,p 1 -Δp,r 1 -Δr).

[0063] Step S203, determining the workpiece return position according to the deviation correction value of the theoretical working pile position and the actual gripping position of the workpiece, so that the material taking device takes the workpiece out of the working pile at the workpiece return position.

[0064] The return position here is the position where the workpiece will not be scratched by the buckle when it is taken out from the work pile. At this position, the workpiece and the buckle are no longer engaged with each other, and the workpiece can be directly taken out from the work pile without being scratched. The return position can be the center position of the work pile or a set special position. The position can be set according to actual conditions, and this application does not impose specific restrictions.

[0065] It is understandable that, since the workpiece is offset on the work pile, the actual gripping position of the workpiece is determined based on the offset of the workpiece, so when the workpiece is corrected, the correction coordinate value of the theoretical workpiece position of the workpiece needs to be added based on the actual gripping position of the workpiece. At the same time, the theoretical work pile position is also offset, so it is also necessary to compensate the correction coordinate value of the theoretical work pile position based on the actual gripping position of the workpiece.

[0066] Specifically, the workpiece return position determination formula is:

[0067] F = E + ΔA - ΔB;

[0068] Among them, F is the workpiece return position, E is the actual grasping position of the workpiece, ΔA is the correction coordinate value of the theoretical workpiece position, and ΔB is the correction coordinate value of the theoretical work pile position.

[0069] In some embodiments, in the material retrieving method, the material retrieving device includes a photographing posture and a material retrieving posture, and the material retrieving device switches from the photographing posture to the material retrieving posture at the theoretical working pile position.

[0070] Optionally, the coordinates of the actual grasping position, the correction coordinate value of the theoretical workpiece position, the correction coordinate value of the theoretical work pile position, etc. in the embodiment of the present application can be six-degree-of-freedom coordinates or three-degree-of-freedom coordinates. For example, the coordinates in the present application can be six-degree-of-freedom coordinates (x, y, z, w, p, r) or three-degree-of-freedom coordinates (x, y, r).

[0071] In the above implementation process, the workpiece image obtained at the theoretical workpiece position and the work pile image obtained at the theoretical work pile position are input into the trained material picking position determination model to obtain the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical work pile position. The theoretical work pile position of the workpiece is compensated by the deviation correction coordinate value of the theoretical workpiece position to determine the actual grasping position of the workpiece, and then the actual grasping position of the workpiece is compensated according to the deviation correction coordinate value of the theoretical work pile position to determine the return position of the workpiece, which is the position where the workpiece is taken out from the work pile without being scratched. By taking out the workpiece at the return position, it is ensured that the workpiece is not scratched when picking up the material, and the workpiece is prevented from being damaged.

[0072] In a possible implementation, step S202 includes: determining the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position and the posture switching difference.

[0073] Among them, the posture switching difference is the coordinate difference of the material picking device switching from the shooting posture to the material picking posture.

[0074] When the material picking device is converted from the shooting posture to the material picking posture, the position of the material picking device changes, that is, when the material picking device is converted from the shooting posture to the material picking posture, a posture switching difference is generated. The posture switching difference can be pre-set and stored in the controller or the material picking device, or it can be obtained by determining the material picking position model. The acquisition of the posture switching difference can be set according to the actual situation, and this application does not make specific restrictions.

[0075] Specifically, the actual grasping position of the workpiece is determined by:

[0076] E = (B + D) + ΔA;

[0077] Among them, E is the actual grasping position of the workpiece, B is the theoretical work pile position, D is the posture switching difference, and ΔA is the correction coordinate value of the theoretical workpiece position.

[0078] In the above implementation process, since the material picking device switches from the shooting posture to the material picking posture, the conversion between postures will cause the position of the device to change when picking the material. When the material picking device is grasping the workpiece, it is necessary to switch the posture from the shooting posture to the material picking posture. When calculating the actual grasping position of the workpiece, by adding compensation for the posture switching difference, the error generated by the material picking device during the posture conversion can be compensated, ensuring that the actual grasping position of the workpiece fits the actual situation and improving the accuracy of determining the actual grasping position of the workpiece.

[0079] See also Figure 4 , is a flow chart of the method for creating a material taking position determination model provided in the embodiment of the present application. Figure 4 The specific process shown is described in detail.

[0080] Step S301 : controlling a material taking device to obtain a workpiece image at a reference position at a first preset position, so as to establish a workpiece image sub-model through the workpiece image at the reference position.

[0081] The first preset position here is a set workpiece placement position. The first preset position can be pre-set and stored in a controller or a material handling device, or it can be a position point acquired in real time. The acquisition of the first preset position can be set according to actual conditions, and this application does not make any specific restrictions.

[0082] The above-mentioned reference position is the position where the workpiece is taken out of the work pile without being scratched by the buckle. The reference position can be the center position of the work pile or a set special position. The position can be set according to actual conditions and this application does not make specific restrictions.

[0083] It can be understood that the first preset position can be understood as the placement position of the workpiece as a whole, and the reference position can be understood as the set position of the workpiece placed on the work pile. The first preset position is the overall position of the workpiece, and the material-retrieving device obtains the workpiece image of the workpiece at the first preset position. The reference position is a specific position point on the work pile, which is used to limit the placement position of the workpiece.

[0084] Since the device obtains the image of the standard position of the workpiece at the standard position during material picking, the image of the workpiece at the reference position is the standard image of the workpiece setting.

[0085] Optionally, the workpiece image sub-model may be the workpiece image at the reference position, or may be the coordinates of each feature point in the workpiece image at the reference position, etc.

[0086] Step S302 : controlling the material taking device to obtain a working pile image at a second preset position, so as to establish a working pile image sub-model through the working pile image at the second preset position.

[0087] The second preset position is obtained when the material taking device is offset from the first preset position by a preset distance.

[0088] The preset distance here can be the height of the workpiece or a set height. For example, the preset distance can be 10 cm, 20 cm, 50 cm, etc. The preset distance can be determined according to the characteristics of the image acquisition device, the workpiece, the clamping device, etc., and this application does not make any specific limitation.

[0089] The second preset position mentioned above may be a standard position where the working pile is set, and the second preset position is used to obtain an image of the working pile in a standard state.

[0090] In some embodiments, the material picking device includes a photographing gesture and a material picking gesture.

[0091] In the above implementation process, by acquiring the workpiece image at the reference position at the first preset position, the standard placement image of the workpiece can be obtained, and by acquiring the work pile image at the second preset position, the standard placement image of the work pile can be obtained, and then the workpiece image sub-model and the work pile image sub-model are respectively established through the standard placement image of the workpiece and the standard placement image of the work pile, so as to realize the automatic modeling of the workpiece image and the work pile image, thereby reducing the workload of modeling.

[0092] In a possible implementation, the method for creating a model for determining the material picking position also includes: controlling the material picking device to switch from a photographing posture to a material picking posture at the target position to obtain the current coordinate value of the material picking device; and determining the posture switching difference based on the current coordinate value and the target position.

[0093] The target position here can be the first preset position, the second preset position, or any other position. The target position can be set according to actual conditions, and this application does not make any specific restrictions.

[0094] The material-retrieving device is in a photographing posture at the target position. When the material-retrieving device changes from the photographing posture to the material-retrieving posture, the position will move to the current coordinate value. The current coordinate value is the actual material-retrieving position of the material-retrieving device.

[0095] It can be understood that the material picking device obtains the workpiece image at the target position. The workpiece image is obtained at the target position, and the material picking device will change its posture when picking up materials. At this time, the position of the material picking device will change. In order to ensure the accuracy of the actual grasping position of the workpiece, when the actual grasping position of the workpiece is performed, it is necessary to compensate for the position change of the material picking device when the posture is changed based on the position of the workpiece obtained through the workpiece image.

[0096] Between the target position and the current coordinate value, the material picking device only performs a posture conversion, and the posture switching difference can be determined by the target position and the current coordinate value.

[0097] In the above implementation process, the material picking device is converted from the photographing posture to the material picking posture at the same position, and the posture conversion value of the material picking device is determined by the different positions of the material picking device under the two postures. Since the material picking device only performs the posture conversion action at this position, the position change between the target position and the current coordinate value is caused by the posture conversion, and the accuracy of the posture conversion value calculation is achieved.

[0098] In a possible implementation, determining the posture switching difference according to the current coordinate value and the target position includes: determining the posture switching difference by subtracting the target position from the current coordinate value.

[0099] The attitude switching difference here includes at least the coordinate difference of three degrees of freedom.

[0100] Specifically, the formula for determining the attitude switching difference is:

[0101] D = CA;

[0102] Among them, C is the current coordinate value, A is the target position, and D is the attitude switching difference.

[0103] In the above implementation process, the posture switching difference is determined by subtracting the target position from the current coordinate value, thereby realizing the posture switching difference calculation. Since the position change between the target position and the current coordinate value is caused by the posture conversion, the accuracy of the posture conversion value calculation is achieved.

[0104] In a possible implementation, before step S301, the material picking position determination model creation method further includes: establishing a data relationship between the image acquisition device coordinates and the clamping device coordinates by a nine-point calibration method.

[0105] It can be understood that the position of the clamping device and the position of the image acquisition device can be calibrated by four-point calibration, five-point calibration, nine-point calibration and other calibration methods. The following takes nine-point calibration as an example to further introduce the calibration method: Nine-point calibration is the universal basis of machine vision. Its principle is to move the 9 positions of the matrix within the acquisition range of the image acquisition device through the conversion device, and transmit the coordinate values ​​of these 9 positions to the image acquisition device, so that 9 corresponding pixel coordinates can be obtained. In this way, the coordinate system transformation relationship between the image acquisition device and the clamping device is directly established, and its essence is to solve the affine transformation matrix of the two sets of coordinate systems.

[0106] Here are the nine principles:

[0107] Assume the affine transformation relationship between the image acquisition device and the clamping device:

[0108]

[0109] in, is the coordinate system of the gripping device, is the coordinate system of the image acquisition device, R is the transformation matrix, and M is the parallel matrix. Substituting the calibrated coordinate system into the above-mentioned radiation transformation relationship formula, we get:

[0110]

[0111]

[0112] Among them, (X′ 0 ∩Y′ 0 ), (X′ 1 ∩Y′ 1 ), (X′ 2 ∩Y′ 2 ) is the coordinate of the gripping device, (X 0 ∩Y 0 ), (X 1 ∩Y 1 ), (X 2 ∩Y 2 ) are the coordinates of the image acquisition device, (a, b, c) and (a′, b′, c′) are unknown parameters.

[0113] Furthermore, after (a, b, c) and (a′, b′, c′) are determined, R and M can be further determined according to the matrix calculation rule.

[0114] In the above implementation process, the nine-point calibration method is used to establish a connection between the coordinates of the clamping device and the image acquisition device, ensuring that the coordinates of the clamping device and the coordinate values ​​obtained by the image acquisition device can be converted to control the movement of the material picking device through the actual position of the workpiece, ensuring the information sharing between the image acquisition device and the clamping device, and realizing the automatic model creation of the material picking equipment.

[0115] Based on the same application concept, a material picking device corresponding to the material picking method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to that in the aforementioned material picking method embodiment, the implementation of the device in this embodiment can refer to the description in the embodiment of the above method, and the repeated parts will not be repeated.

[0116] See also Figure 5, is a functional module diagram of the material taking device provided in the embodiment of the present application. Each module in the material taking device in this embodiment is used to execute each step in the above method embodiment. The material taking device includes a first acquisition module 401, a first determination module 402, and a second determination module 403; wherein,

[0117] The first acquisition module 401 is used to input the workpiece image acquired at the theoretical workpiece position and the work pile image acquired at the theoretical work pile position into the trained material picking position determination model to obtain the correction coordinate value of the theoretical workpiece position and the correction coordinate value of the theoretical work pile position.

[0118] The first determination module 402 is used to determine the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position.

[0119] The second determination module 403 is used to determine the workpiece return position according to the correction coordinate value of the theoretical work pile position and the actual gripping position of the workpiece, so that the material taking device takes the workpiece out of the work pile at the workpiece return position.

[0120] In a possible implementation, the first determination module 402 is also used to determine the actual grasping position of the workpiece based on the correction coordinate value of the workpiece position and the posture switching difference; wherein the posture switching difference is the coordinate difference of the material picking device switching from the photographing posture to the material picking posture.

[0121] Based on the same application concept, the embodiment of the present application also provides a material picking position determination model creation device corresponding to the material picking position determination model creation method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the aforementioned material picking position determination model creation method embodiment, the implementation of the device in this embodiment can refer to the description in the embodiment of the above-mentioned method, and the repeated parts will not be repeated.

[0122] See also Figure 6 , is a functional module diagram of the device for creating a model for determining a material taking position provided in an embodiment of the present application. Each module in the device for creating a model for determining a material taking position in this embodiment is used to execute each step in the above method embodiment. The device for creating a model for determining a material taking position includes a second acquisition module 501 and a third acquisition module 502; wherein,

[0123] The second acquisition module 501 is used to control the material picking device to acquire the workpiece image at the reference position at the first preset position, so as to establish a workpiece image sub-model through the workpiece image at the reference position.

[0124] The third acquisition module 502 is used to control the material-grabbing device to acquire the work pile image at a second preset position, so as to establish a work pile image sub-model through the work pile image at the second preset position; wherein the second preset position is obtained by the material-grabbing device being offset from the first preset position by a preset distance.

[0125] In a possible implementation, the device for creating a model for determining a material picking position also includes a control module, which is used to: control the material picking device to switch from a photo taking posture to a material picking posture at a target position to obtain a current coordinate value of the material picking device; and determine a posture switching difference based on the current coordinate value and the target position.

[0126] In a possible implementation manner, the control module is further configured to determine the posture switching difference by subtracting the target position from the current coordinate value, and the posture switching difference includes at least coordinate differences of three degrees of freedom.

[0127] In a possible implementation manner, the device for creating a model for determining a material picking position further includes a coordinate establishment module for establishing a data relationship between camera coordinates and material picking device clamping device coordinates by a nine-point calibration method.

[0128] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the material collection method and / or the material collection position determination model creation method described in the above method embodiment are executed.

[0129] The computer program product of the material picking method and / or the material picking location determination model creation method provided in the embodiments of the present application includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the material picking method and / or the material picking location determination model creation method described in the above method embodiments. Please refer to the above method embodiments for details, which will not be repeated here.

[0130] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0132] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a controller, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variation thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the presence of other identical elements in the process, method, article or apparatus that includes the elements.

[0133] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0134] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A material taking method, characterized in that: include: Controlling the material taking device to obtain the workpiece image at the reference position at the first preset position, so as to establish the workpiece image sub-model through the workpiece image at the reference position; Controlling the material taking device to obtain the working pile image at a second preset position, so as to establish the working pile image sub-model through the working pile image at the second preset position; wherein the second preset position is obtained by the material taking device being offset from the first preset position by a preset distance; The workpiece image obtained at the theoretical workpiece position and the work pile image obtained at the theoretical work pile position are input into the trained material picking position determination model to obtain the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical work pile position; wherein the theoretical work pile position is the position of the material picking device after it moves downward by the height of the workpiece from the theoretical workpiece placement position; Determine the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position; The workpiece return position is determined according to the correction coordinate value of the theoretical working pile position and the actual gripping position of the workpiece, so that the material taking device takes the workpiece out of the working pile at the workpiece return position; the return position is a position where the workpiece will not be scratched by the buckle when being taken out of the working pile; Wherein, the material picking position determination model includes a workpiece image sub-model and a work pile image sub-model; the workpiece image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical workpiece position; the work pile image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical work pile position; In the material picking method, the material picking device includes a photographing posture and a material picking posture, and the material picking device switches from the photographing posture to the material picking posture at the theoretical work pile position; the actual grabbing position of the workpiece is determined according to the correction coordinate value of the theoretical workpiece position, including: Determine the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position and the posture switching difference; The posture switching difference is the coordinate difference of the material picking device switching from the photographing posture to the material picking posture.

2. The method according to claim 1, characterized in that The formula for determining the workpiece return position is: ； in, To return the workpiece to its original position, is the actual grasping position of the workpiece, is the correction coordinate value of the theoretical workpiece position, It is the correction coordinate value of the theoretical working pile position.

3. A method for creating a material taking position determination model, characterized in that: The material taking position determination model is applied to the material taking method as claimed in claim 1 or 2; The method for creating a material taking position determination model comprises: Controlling the material taking device to obtain the workpiece image at the reference position at the first preset position, so as to establish the workpiece image sub-model through the workpiece image at the reference position; Controlling the material taking device to acquire the working pile image at the second preset position, so as to establish the working pile image sub-model through the working pile image at the second preset position; The second preset position is obtained when the material taking device is offset from the first preset position by a preset distance.

4. The method according to claim 3, characterized in that In the creation of the material picking position determination model, the material picking device includes a photographing posture and a material picking posture, and the material picking position determination model creation method further includes: Controlling the material-retrieving device to switch from a photographing posture to a material-retrieving posture at a target position to obtain a current coordinate value of the material-retrieving device; A posture switching difference is determined according to the current coordinate value and the target position.

5. The method according to claim 4, characterized in that The step of determining the posture switching difference according to the current coordinate value and the target position includes: The posture switching difference is determined by subtracting the target position from the current coordinate value, and the posture switching difference includes at least coordinate differences of three degrees of freedom.

6. A material taking device, characterized in that: include: A second acquisition module is used to control the material taking device to acquire the workpiece image at the reference position at the first preset position, so as to establish a workpiece image sub-model through the workpiece image at the reference position; A third acquisition module is used to control the material taking device to acquire the image of the working pile at a second preset position, so as to establish a working pile image sub-model through the image of the working pile at the second preset position; wherein the second preset position is obtained by the material taking device being offset from the first preset position by a preset distance; The first acquisition module is used to input the workpiece image acquired at the theoretical workpiece position and the work pile image acquired at the theoretical work pile position into the trained material picking position determination model to obtain the deviation correction coordinate value of the theoretical workpiece position and the deviation correction coordinate value of the theoretical work pile position; wherein the theoretical work pile position is the position of the material picking device after moving downward by the height of the workpiece from the theoretical workpiece placement position; A first determination module is used to determine the actual grasping position of the workpiece according to the correction coordinate value of the theoretical workpiece position; The second determination module is used to determine the workpiece return position according to the correction coordinate value of the theoretical working pile position and the actual gripping position of the workpiece, so that the material taking device takes the workpiece out of the working pile at the workpiece return position; the return position is the position where the workpiece will not be scratched by the buckle when it is taken out of the working pile; Wherein, the material picking position determination model includes a workpiece image sub-model and a work pile image sub-model; the workpiece image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical workpiece position; the work pile image sub-model is used to determine the correction coordinate value of the material picking device at the theoretical work pile position; the material picking device includes a photographing posture and a material picking posture, and the material picking device switches from the photographing posture to the material picking posture at the theoretical work pile position; The first determination module is also used to determine the actual grasping position of the workpiece based on the correction coordinate value of the theoretical workpiece position and the posture switching difference; wherein the posture switching difference is the coordinate difference of the material picking device switching from the photographing posture to the material picking posture.

7. A device for creating a model for determining a material taking position, characterized in that: The material taking position determination model is applied to the material taking method as claimed in claim 1 or 2; The material taking position determination model creation device comprises: A second acquisition module is used to control the material taking device to acquire the workpiece image at the reference position at the first preset position, so as to establish a workpiece image sub-model through the workpiece image at the reference position; A third acquisition module is used to control the material taking device to acquire the working pile image at a second preset position, so as to establish a working pile image sub-model through the working pile image at the second preset position; The second preset position is obtained when the material taking device is offset from the first preset position by a preset distance.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of any method described in claims 1 to 2 or claims 3 to 5.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 or claims 3 to 5 are executed.

Images