Visual calibration method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a visual calibration method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of computers, and the method comprises the following steps: obtaining first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit in the plurality of material storage units; acquiring first position data of a second positioning point corresponding to a second material in the to-be-calibrated storage unit; according to the first position data and the first calibration data, second position data of a second material taking point corresponding to the second material and third position data of a second material placing point are determined; and according to the first position data, the second position data and the third position data, a second relative position relation among a second positioning point, a second material taking point and a second material placing point corresponding to a second material in the to-be-calibrated storage unit is calibrated. Therefore, the time required for debugging and calibration is greatly shortened, the visual calibration efficiency is effectively improved, and the multi-station visual calibration process is remarkably optimized.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a vision calibration method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] Intelligent manufacturing is a human-machine integrated intelligent system composed of robots and human experts, which conducts intelligent activities during the manufacturing process. In order for the robot to complete tasks more precisely, it is crucial to clarify its position, posture in space, and the relative relationship with surrounding objects, which depends on the vision calibration of the robot.

[0003] However, in the prior art, the vision calibration of robots mostly relies on manual operations. Manual calibration requires professionals to manually complete many cumbersome steps, and the process is extremely time-consuming and laborious. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a vision calibration method, apparatus, electronic device, storage medium, and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, a vision calibration method is provided, and the method includes: Obtain first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among multiple material storage units; the first calibration data is used to represent the first relative position relationship among a first positioning point, a first material taking point, and a first material placing point corresponding to a first material in the target storage unit; the positioning point is used to represent the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit; Obtain first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among multiple material storage units; Determine second position data of a second material taking point and third position data of a second material placing point corresponding to the second material according to the first position data and the first calibration data; Calibrate the second relative position relationship among the second positioning point, the second material taking point, and the second material placing point corresponding to the second material in the storage unit to be calibrated according to the first position data, the second position data, and the third position data.

[0006] Optionally, the obtaining first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated includes: Control the robotic arm to move so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of a first identification point corresponding to the second material in the storage unit to be calibrated; Take the position of the robotic arm when the center of the acquisition field of view of the image acquisition device coincides with the center of the first identification point as the second positioning point, and obtain the first position data corresponding to the second positioning point.

[0007] Optionally, controlling the movement of the robotic arm so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated includes: Obtain the preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when the second material is acquired. Based on the preset image acquisition distance, control the movement of the robotic arm so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0008] Optionally, the preset image acquisition distance is preset in the following manner: Control the robotic arm to move to the initial position, where the initial position is the position of the robotic arm when the acquisition field of view of the image acquisition device on the robotic arm includes the second material. Based on the specified direction corresponding to the initial position, control the robotic arm to move within the target movement range in the specified direction to obtain the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range. Determine the preset image acquisition distance according to the multiple material images.

[0009] Optionally, the determining the preset image acquisition distance according to the multiple material images includes: For each material image, determine the evaluation score corresponding to the material image; the higher the evaluation score, the higher the clarity of the material image. Take the material image with the highest evaluation score as the target image, and take the movement position corresponding to the target image as the target position. Determine the preset image acquisition distance according to the target position.

[0010] Optionally, the determining the evaluation score corresponding to the material image includes: Determine the gradient value corresponding to each pixel point in the material image. Determine the evaluation score corresponding to the material image according to the multiple gradient values.

[0011] Optionally, the obtaining the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range includes: When the residence time of the robotic arm at each moving position reaches a preset time threshold, control the image acquisition device to acquire the material image.

[0012] Optionally, the first calibration data further includes fourth position data corresponding to the first positioning point. According to the first position data and the first calibration data, determining the second position data of the second picking point and the third position data of the second placing point corresponding to the second material includes: Determine the relative position between the first position data and the fourth position data; According to the relative position and the first relative position relationship, determine the second position data corresponding to the second picking point and the third position data corresponding to the second placing point.

[0013] According to a second aspect of the embodiments of the present disclosure, there is provided a vision calibration device, the device includes: A first acquisition module configured to acquire first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among multiple material storage units; the first calibration data is used to characterize the first relative position relationship among the first positioning point, the first picking point, and the first placing point of the first material in the target storage unit; the positioning point is used to characterize the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit; A second acquisition module configured to acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among multiple material storage units; A determination module configured to determine second position data of a second picking point and third position data of a second placing point corresponding to the second material according to the first position data and the first calibration data; A calibration module configured to calibrate the second relative position relationship among the second positioning point, the second picking point, and the second placing point of the second material in the storage unit to be calibrated according to the first position data, the second position data, and the third position data.

[0014] Optionally, the second acquisition module is configured to control the robotic arm to move so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; use the position where the robotic arm is located when the center of the acquisition field of view of the image acquisition device coincides with the center of the first identification point as the second positioning point, and acquire the first position data corresponding to the second positioning point.

[0015] Optionally, the second acquisition module is configured to acquire a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when the image acquisition device acquires the second material; based on the preset image acquisition distance, control the robotic arm to move so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the to-be-calibrated storage unit.

[0016] Optionally, the preset image acquisition distance is preset in the following manner: Control the robotic arm to move to an initial position, where the initial position is the position of the robotic arm when the acquisition field of view of the image acquisition device on the robotic arm includes the second material; Based on the specified direction corresponding to the initial position, control the robotic arm to move within the target movement range in the specified direction to acquire the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range; Determine the preset image acquisition distance according to the multiple material images.

[0017] Optionally, the determining the preset image acquisition distance according to the multiple material images includes: For each material image, determine an evaluation score corresponding to the material image; the higher the evaluation score, the higher the clarity of the material image; Take the material image with the highest evaluation score as the target image, and take the movement position corresponding to the target image as the target position; Determine the preset image acquisition distance according to the target position.

[0018] Optionally, the determining the evaluation score corresponding to the material image includes: Determine the gradient value corresponding to each pixel point in the material image; Determine the evaluation score corresponding to the material image according to the multiple gradient values.

[0019] Optionally, the acquiring the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range includes: When the residence time of the robotic arm at each movement position reaches a preset time threshold, control the image acquisition device to acquire the material image.

[0020] Optionally, the first calibration data further includes fourth position data corresponding to a first positioning point, and the determining module is configured to determine a relative position between the first position data and the fourth position data; and determine second position data corresponding to the second material taking point and third position data corresponding to the second material placing point according to the relative position and the first relative position relationship.

[0021] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing executable instructions executable by the processor; wherein, the processor is configured to implement the steps of the visual calibration method provided in the first aspect of the present disclosure when calling the executable instructions stored on the memory.

[0022] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the visual calibration method provided in the first aspect of the present disclosure are implemented.

[0023] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the visual calibration method provided in the first aspect of the present disclosure are implemented.

[0024] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: First, obtain first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among a plurality of material storage units; the first calibration data is used to characterize the first relative position relationship among the first positioning point, the first material taking point, and the first material placing point of the first material in the target storage unit; the positioning point is used to characterize the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit. Second, obtain the first position data of the second positioning point corresponding to the second material in the storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among a plurality of material storage units. Then, according to the first position data and the first calibration data, determine the second position data of the second material taking point and the third position data of the second material placing point corresponding to the second material. Finally, calibrate the second relative position relationship among the second positioning point, the second material taking point, and the second material placing point of the second material in the storage unit to be calibrated according to the first position data, the second position data, and the third position data. Through the above method, in the scenario of visual calibration of multiple workstations, only the first calibration data of the target storage unit among a plurality of material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material therein can be automatically obtained. Subsequently, based on the first position data and the calibration result of the target storage unit (i.e., the first calibration data), automatic calibration of the storage unit to be calibrated can be achieved. In this way, the time required for debugging and calibration is significantly shortened, the efficiency of visual calibration is effectively improved, and the multi-station visual calibration process is significantly optimized.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Brief Description of the Drawings

[0026] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0027] Figure 1 is a flowchart of a visual calibration method shown according to an exemplary embodiment.

[0028] Figure 2 is a flowchart of another visual calibration method shown according to an exemplary embodiment.

[0029] Figure 3 is a flowchart of another visual calibration method shown according to an exemplary embodiment.

[0030] Figure 4It is a flowchart of a visual calibration method shown according to an exemplary embodiment.

[0031] Figure 5 It is a block diagram of a visual calibration device shown according to an exemplary embodiment.

[0032] Figure 6 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0033] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0034] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining authorization from the owner of the corresponding device.

[0035] The terms "first", "second", etc. in the description and claims of this application and the above-mentioned accompanying drawings are used to distinguish similar objects and do not have to be understood as a specific order or sequence. In addition, in the description with reference to the accompanying drawings, the same reference numerals in different drawings represent the same elements.

[0036] The term "including" and its variants used herein are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0037] In the description of the present disclosure, unless otherwise specified, "a plurality" means two or more, and other quantifiers are similar; "at least one", "one or more" or similar expressions refer to any combination of these items, including any combination of a single item or plural items. For example, at least one a can represent any number of a; for another example, one or more of a, b, and c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple; "and / or" is a relationship describing associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. The character " / " indicates that the associated objects before and after are in an "or" relationship.

[0038] In the embodiments of the present disclosure, although operations or steps are described in a specific order in the drawings, it should not be understood that these operations or steps are required to be performed in the specific order shown or in a serial order, or that all the operations or steps shown are required to be performed to obtain the desired result. In the embodiments of the present disclosure, these operations or steps can be performed serially; they can also be performed in parallel; or a part of these operations or steps can be performed.

[0039] Before introducing the visual calibration method, device, electronic device, storage medium, and program product provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. The present disclosure can be applied to the scenario where a robot performs intelligent activities in the field of intelligent manufacturing. In one application scenario, the robot needs to grasp and place materials from multiple material drawers. After receiving the instruction, the robotic arm flexibly rotates its joints, locates the position of the drawer, and accurately extends the gripper to open the drawer. Then, it accurately grasps the material, and then smoothly raises the arm, moves to the specified placement area (i.e., the material placement point) according to the planned path, slowly places the material, completes the grasping and placing actions, and then quickly returns to the initial position to prepare for the next task. In this scenario, in order to enable the robot to complete the task more accurately and clarify its position, posture, and relative relationship with surrounding objects in space, it is often necessary to perform visual calibration on the robot.

[0040] For a robot device of the type where a robotic arm corresponds to multiple material drawers for picking and placing materials, its camera is usually installed on the robotic arm. During equipment debugging, a one-to-one calibration operation of the robotic arm - material drawer needs to be carried out for each material drawer, and currently, the calibration of each material drawer is completed through the manual teaching method. First, for each material drawer, corresponding photo - recognition points need to be established to photograph and recognize the corresponding Mark points (also known as reference points. After being recognized by an image acquisition device (such as a camera), they provide accurate position and attitude (translation / rotation) information for the equipment and are the core reference for visual positioning). Secondly, corresponding material - picking points and material - placing points need to be established for each material drawer, and precise manual teaching is required. For example, if a robot device has 16 drawers, 16 photo - recognition points need to be set, 16 material - picking points and 16 material - placing points need to be manually taught, and 16 visual calibrations need to be carried out at the same time.

[0041] In long - term practice, the inventor found that the above - mentioned visual calibration method mainly has the following two problems. On the one hand, the efficiency of manual teaching is relatively low, very time - consuming, and its accuracy judged by the human eye is affected by the debugging experience of technicians. A large amount of repetitive manual labor makes the calibration efficiency extremely low, seriously restricting the production rhythm and overall efficiency improvement of intelligent factories and making it difficult to meet the needs of the rapid development of intelligent manufacturing. On the other hand, the perspective and field of view of the human eye are relatively limited. When an object within the human eye's field of view is blocked by other objects and light cannot enter the human eye, a situation of blocked line of sight will occur. For multi - drawer devices, the drawers are distributed in the upper and lower layers of the device. The drawers in the lower layer often have problems such as blocked vision and difficult teaching, and there is a high risk of collision.

[0042] To solve the above - mentioned technical problems, the present invention provides a visual calibration method, device, electronic device, storage medium, and program product. In the scenario of visual calibration for multiple workstations, only the first calibration data of the target storage unit among multiple material storage units needs to be pre - calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration result of the target storage unit (i.e., the first calibration data), the automatic calibration of the storage units to be calibrated can be achieved. In this way, the time required for debugging and calibration is significantly shortened, the efficiency of visual calibration is effectively improved, and the multi - workstation visual calibration process is significantly optimized.

[0043] The following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings.

[0044] Figure 1is a flowchart of a visual calibration method shown according to an exemplary embodiment. This method can be used in a terminal device, such as a robot. In some embodiments, the robot may include a robotic arm and an image acquisition device. Among them, the robotic arm serves as an execution unit for grasping and placing materials from multiple material storage units on a material rack. Exemplarily, the material rack may include multiple material storage units arranged in an up-and-down order, and each material storage unit contains a corresponding material. The material storage unit can be, for example, a material drawer. The robotic arm achieves high-precision motion through multi-degree-of-freedom joints and has capabilities such as flexible grasping, stable handling, and precise assembly. It is the "executing hand" that supports the robot to complete actual operations. The image acquisition device is arranged on the robotic arm and is used to acquire image information of the materials on the material storage unit. By real-time obtaining visual data such as the position, pose, and type of the materials, after processing, it provides the robot with environmental perception capabilities, assists the robotic arm in accurately positioning the materials, planning the motion path, and ensuring the accuracy of grasping and placing operations. The two cooperate to enable the robot to achieve intelligent interaction of "visual guidance + precise execution" in the material handling scenario. As Figure 1 shown, the visual calibration method may include the following steps.

[0045] In step S101, obtain the first calibration data corresponding to the target storage unit.

[0046] Among them, the target storage unit is a pre-calibrated material storage unit among multiple material storage units; the first calibration data is used to represent the first relative position relationship between the first positioning point, the first picking point, and the first placing point corresponding to the first material in the target storage unit; the positioning point is used to represent the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit.

[0047] In some embodiments, in order to improve the calibration speed of the terminal device, a material storage unit can be selected from multiple material storage units as the main storage unit, that is, the target storage unit, and the target storage unit is pre-calibrated.

[0048] Specifically, the first calibration data of the target storage unit can be pre-calibrated through the following steps: First, the first positioning point corresponding to the first material in the target storage unit can be determined.

[0049] Among them, the first positioning point can represent the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the first material on the target storage unit. The first positioning point can also be understood as the first photographing position point corresponding to the target storage unit, where photographing and positioning of the first material are achieved.

[0050] In order to more accurately achieve the grasping and placement of materials, identification points, also known as Mark points, can be pre-set on the materials. The Mark point can be a position point or a position area, and can be set at the edge or center position of the material according to actual needs. When the center of the acquisition field of the image acquisition device coincides with the identification point, it can be considered that the robotic arm has reached the positioning point. At this time, automatic focusing and photographing can be performed on the positioning point to identify the center coordinates of the current positioning point. For example, when the center of the acquisition field of the image acquisition device coincides with the second identification point of the first material, this position point is considered the first positioning point. Further, the fourth position data corresponding to the first positioning point can be determined. Among them, the fourth position data can include, for example, the center coordinates of the second identification point, that is, the center coordinates (x, y) of the Mark point.

[0051] Secondly, after completing the positioning of the first positioning point, the first material taking point and the first material placing point can be further determined.

[0052] To ensure the accuracy of subsequent calibration, the position calibration of the first material taking point and the first material placing point for the target storage unit can be carried out by manual teaching to sequentially determine the fifth position data corresponding to the first material taking point and the sixth position data corresponding to the first material placing point.

[0053] Finally, according to the fourth position data corresponding to the first positioning point, the fifth position data corresponding to the first material taking point, and the sixth position data corresponding to the first material placing point, the first relative position relationship among the first positioning point, the first material taking point, and the first material placing point is determined to achieve the calibration of the target storage unit.

[0054] In addition, to further improve the accuracy of the robotic arm in grasping and placing materials, a nine-point correction and rotation correction process can also be performed on the robotic arm based on the above calibration results. For example, in the nine-point correction, the reference plane and 9 sampling points can be determined first, the initial position data can be obtained, the deviation value and compensation amount can be calculated by comparison, and finally the compensation amount is input into the control system for real-time adjustment to complete the correction. During rotation correction, the angle sensor is used to detect the deviation, the correction strategy is determined according to the deviation and motion state, and the control system sends commands to the rotary joint motor accordingly to drive the joint to rotate to achieve the rotation angle correction. The specific implementation method can refer to the step process in the related technology and will not be specifically described here.

[0055] After completing the manual teaching, the first calibration data of the target storage unit can be obtained, thereby ensuring that the material taking position and the material placing position of the robotic arm to the target storage unit are both accurate.

[0056] In step S102, the first position data of the second positioning point corresponding to the second material in the storage unit to be calibrated is obtained.

[0057] Among them, the storage unit to be calibrated is the uncalibrated storage unit among multiple material storage units. It can be understood that in this embodiment, only the target storage unit needs to be pre-calibrated, and the subsequent storage units to be calibrated can all be automatically calibrated in the following manner, greatly saving labor costs and effectively improving the overall calibration efficiency.

[0058] In some embodiments, by controlling the movement of the robotic arm, the center of the acquisition field of the image acquisition device on the robotic arm can be made to coincide with the center of the first identification point corresponding to the second material in the storage unit to be calibrated, and the position point where the robotic arm is located when the center of the acquisition field of the image acquisition device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is obtained.

[0059] In order to improve the calibration efficiency and accuracy, the identification points corresponding to all the material storage units on the material rack can be set to be the same. In this way, when calibrating the first position data of the second positioning points of other storage units to be calibrated subsequently, the moving distance of the robotic arm can be shortened, and the calibration efficiency can be improved.

[0060] In step S103, according to the first position data and the first calibration data, the second position data of the second material taking point corresponding to the second material and the third position data of the second material placing point are determined.

[0061] Since the first relative position relationship among the first positioning point, the first material taking point, and the first material placing point corresponding to the first material is recorded in the first calibration data, after the first position data corresponding to the second positioning point of the second material is determined, the second position data of the second material taking point corresponding to the second material and the third position data corresponding to the second material placing point can be determined according to the first position data and the first calibration data.

[0062] According to the foregoing, the second positioning point is the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the second material on the storage unit to be calibrated. That is to say, when the robotic arm is at the second positioning point, the image acquisition device thereon can acquire the image of the second material. Therefore, in one implementation manner, the center position of the second material can also be determined based on the second positioning point, and the second material taking point can be determined according to the center position and the vertical distance between the robotic arm and the second material. In order to improve the calibration efficiency of the second material taking point, the first identification point can be set at the center position of the second material, which can shorten the time for the robotic arm to move to find the center position of the second material.

[0063] In step S104, according to the first position data, the second position data, and the third position data, the second relative position relationship among the second positioning point, the second material taking point, and the second material placing point corresponding to the second material in the storage unit to be calibrated is calibrated.

[0064] After obtaining the first position data, the second position data, and the third position data, that is, after knowing the positions of the second positioning point, the second material taking point, and the second material placing point, the second relative position relationship between the second positioning point, the second material taking point, and the second material placing point corresponding to the second material in the to-be-calibrated storage unit can be determined, thereby completing the calibration of the remaining to-be-calibrated storage units.

[0065] Using the above method, in the scenario of visual calibration for multiple workstations, only the first calibration data of the target storage unit among multiple material storage units needs to be pre-calibrated. For the remaining to-be-calibrated storage units, the first position data corresponding to the second positioning point of the second material therein can be automatically obtained. Subsequently, based on the first position data and the calibration result of the target storage unit (i.e., the first calibration data), the automatic calibration of the to-be-calibrated storage unit can be achieved. In this way, the time required for debugging and calibration is significantly shortened, the efficiency of visual calibration is effectively improved, and the visual calibration process for multiple workstations is significantly optimized.

[0066] Image clarity is an important indicator to measure image quality. If the industrial lens is not accurately focused, the image will become blurred and unclear, thus affecting the visual calibration effect. In the existing industrial lens focusing schemes, after the camera lens is installed, the debugging personnel first fix the focusing ring of the lens, and then manually perform the focusing work. Specifically, first, the working distance of the camera lens is adjusted in a fixed step, and at the same time, a photo is taken, and the debugging personnel manually judge the current image quality; if the current image quality meets the best focusing effect judged manually, it is considered that the focusing is completed at present, otherwise, continue to move the fixed step to adjust the working distance of the camera lens and make a judgment until the final focusing is completed.

[0067] However, the existing manual focusing scheme is greatly affected by the subjectivity of the debugging personnel and highly dependent on the debugging experience of the debugging personnel, and it is easy to have the situation of insufficiently clear focusing, thus affecting the subsequent positioning accuracy.

[0068] Based on the above situation, in order to improve the focusing accuracy and the subsequent positioning accuracy, the present disclosure proposes an automatic focusing method for an image acquisition device to focus on and photograph a material to determine a positioning point. By automatically calculating the optimal working distance of each corresponding material storage unit, the automatic focusing is completed with an objective standard, getting rid of the problem that the image imaging effect is affected by personal subjectivity and debugging experience when manually focusing the camera lens.

[0069] Taking the above step S102 as an example, the automatic focusing method provided by the present disclosure will be described in detail as follows. Figure 2 As shown, the obtaining of the first position data of the second positioning point corresponding to the second material in the to-be-calibrated storage unit in the above step S102 may include the following steps: In step S1021, control the robotic arm to move so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0070] First, a preset image acquisition distance corresponding to the second material can be obtained.

[0071] Wherein, the preset image acquisition distance is the distance between the image acquisition device and the second material when the image acquisition device acquires the second material. The preset image acquisition distance can be understood as the distance at which the image acquisition device acquires the second material at this position, and the clarity of the image is the highest. That is to say, the preset image acquisition distance is the optimal focusing distance for the image acquisition device to acquire the second material.

[0072] Wherein, the preset image acquisition distance can be preset in the following manner: (1) Control the robotic arm to move to the initial position.

[0073] Wherein, the initial position is the position of the robotic arm when the acquisition field of the image acquisition device on the robotic arm includes the second material. For example, the initial position can be determined according to the ROI (Region Of Interest, the area of interest, which refers to a specific area of interest extracted from an image or video. This area can be an object, a specific part of a scene, or an area with certain characteristics) position of the storage unit to be calibrated. That is, at the initial position, the image acquisition device can acquire the ROI position of the second material. Among them, the ROI position can be understood as the position area containing the second material. The initial position can be, for example, the position directly above the ROI position, and in order to avoid the risk of collision, the height corresponding to the initial position (that is, the distance between the image acquisition device and the material) can be slightly greater than the preset working distance of the robotic arm.

[0074] In some embodiments, the focusing ring of the image acquisition device installed on the robotic arm can be rotated to the first preset position (such as the middle) and fixed, the exposure ring of the lens can be rotated to the second preset position (such as the middle) and fixed, and the robotic arm equipped with the image acquisition device can be moved to the position above the second material, that is, the initial position. That is to say, at this time, the focusing distance and exposure parameters of the image acquisition device are fixed. Then, by moving the distance between the image acquisition device and the material, the image within the acquisition field of the image acquisition device can be made clearer.

[0075] (2) Based on the specified direction corresponding to the initial position, control the robotic arm to move within the target movement range in the specified direction to obtain the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range.

[0076] Among them, the specified direction may include vertically upward, so as to ensure that there is no risk of collision. The target movement range may be a preset range vertically upward, and the target movement range may cover the maximum variable range of the zoom ring of the image acquisition device, that is, the target movement range may be determined according to the maximum variable range.

[0077] In some embodiments, based on the specified direction corresponding to the initial position, the robotic arm can be controlled to move within the target movement range in the specified direction. Each time it moves a certain step distance (for example, 0.1 mm), it reaches a movement position. At this time, the image acquisition device can be controlled to acquire a material image containing the second material.

[0078] In other embodiments, considering that after the robotic arm moves, the image acquisition device may shake, resulting in image distortion of the material image. Therefore, in order to ensure the image quality of the material image, when the residence time of the robotic arm at each movement position reaches a preset time threshold, the image acquisition device is controlled to acquire the material image.

[0079] Exemplarily, if the maximum variable range that the zoom ring of the image acquisition device can rotate is 3 mm and the step distance is 0.1 mm, it should be set to move 30 times. In addition, after the robotic arm arrives at each position, a 200 ms delay can be performed before taking a picture to avoid image distortion caused by movement.

[0080] (3)Determine the preset image acquisition distance according to multiple such material images.

[0081] In some embodiments, the preset image acquisition distance can be determined through the following steps: a) For each material image, determine the evaluation score corresponding to the material image.

[0082] Among them, the higher the evaluation score, the higher the clarity of the material image.

[0083] Specifically, the evaluation score corresponding to each material image can be determined through a pre-set evaluation function. By way of example, the evaluation function can include, for example but not limited to, a variance function (constructing an evaluation function by calculating the variance or standard deviation of the image to evaluate the clarity of the image), a Laplacian function (obtaining a gradient matrix by performing convolution processing on the gray values of each pixel point in the image, and calculating the sum of squares of the matrix as the evaluation function), an energy gradient function (calculating the gradient of a point using the difference between adjacent points as the evaluation function), a Brenner function (calculating the gradient in one direction, calculating the difference in gray values of pixels separated by two units, and then squaring this difference as the evaluation function), and a Tenegrad function (evaluating the gradient values in the horizontal and vertical directions using the Sobel operator, amplifying the image edge gradient using the square operation, and defining the evaluation function as the sum of squares of the gradients). Any one of them can be used.

[0084] In a possible implementation, the gradient value corresponding to each pixel point in the material image can be determined first, and then, based on multiple gradient values, the evaluation score corresponding to the material image can be determined.

[0085] b) Use the material image with the highest evaluation score as the target image, and use the moving position corresponding to the target image as the target position.

[0086] The material image with the highest evaluation score is the image with the highest clarity. Therefore, the material image with the highest evaluation score can be used as the target image, and the moving position corresponding to the target image can be used as the target position.

[0087] c) Determine the preset image acquisition distance according to the target position.

[0088] Determine the preset image acquisition distance according to the position of the target position on the Z-axis.

[0089] Then, based on the preset image acquisition distance, control the movement of the robotic arm so that the acquisition field center of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0090] When the identification points corresponding to all the material storage units are the same, since the position of the first positioning point of the target storage unit has been pre-calibrated when the acquisition field center coincides exactly with the second identification point, controlling the movement of the robotic arm based on the preset image acquisition distance so that the acquisition field center of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated can also include: Based on the preset image acquisition distance, control the robotic arm to move so that when the deviation between the center of the acquisition field of the image acquisition device on the robotic arm and the center of the first identification point corresponding to the second material in the storage unit to be calibrated is less than the preset pixel unit, it can be considered that the two coincide.

[0091] That is to say, on the basis of ensuring the accuracy of the first positioning point, a certain small range of deviation is allowed for the remaining storage units to be calibrated, which not only does not affect the calibration effect but also improves the calibration efficiency.

[0092] Exemplarily, according to the preset image acquisition distance, control the robotic arm to lock at the position corresponding to the Z-axis, that is, the robotic arm will not move in the Z-axis direction hereafter. Then, control the robotic arm to move on the X-axis and Y-axis so that when the deviation between the center of the acquisition field (x, y) of the image acquisition device on the robotic arm and the center of the first identification point (xi, yi) corresponding to the second material in the storage unit to be calibrated is less than 2 pixels, that is, |x - xi| ≤ 2 & |y - yi| ≤ 2, it can be considered that the two coincide, and the second positioning point is found.

[0093] In step S1022, take the position of the robotic arm when the center of the acquisition field of the image acquisition device coincides with the center of the first identification point as the second positioning point, and obtain the first position data corresponding to the second positioning point.

[0094] It should be noted that the above autofocus process can also be applied to the scenario of focusing when determining the fourth position data corresponding to the first positioning point of the target storage unit.

[0095] The following will elaborate on the above step S103. As Figure 3 shown, determining the second position data of the second picking point and the third position data of the second placing point corresponding to the second material according to the first position data and the first calibration data in the above step S103 may include: In step S1031, determine the relative position between the first position data and the fourth position data.

[0096] In step S1032, determine the second position data corresponding to the second picking point and the third position data corresponding to the second placing point according to the relative position and the first relative position relationship.

[0097] In a possible implementation manner, if the relative position relationships between the positioning points corresponding to all material storage units on the material rack and the picking points and placing points are the same, then when knowing the relative position between the first position data and the fourth position data, the second position data corresponding to the second picking point and the third position data corresponding to the second placing point can be directly determined according to the relative position and the first relative position relationship.

[0098] In another possible implementation, if the positions of the picking points and the placing points corresponding to all the material storage units on the material rack are the same, that is, the fifth position data of the first placing point corresponding to the target storage unit can be directly used as the second position data corresponding to the second picking point, and the sixth position data of the first placing point corresponding to the target storage unit can be used as the third position data corresponding to the second placing point.

[0099] In another possible implementation, if the relative position relationships between the positioning points and the picking points and the placing points corresponding to all the material storage units on the material rack are different, and the positions of the picking points and the placing points corresponding to all the material storage units on the material rack are also different, then the second relative position relationships between the respective picking points and the third relative position relationships between the respective placing points can be determined in advance. After determining the first calibration data, based on this relative position, the first relative position relationship, the second relative position relationship, and the third relative position relationship, the second position data corresponding to the second picking point and the third position data corresponding to the second placing point are determined.

[0100] By using the above method, in the scenario of visual calibration for multiple workstations, only the first calibration data of the target storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration result (i.e., the first calibration data) of the target storage unit, the automatic calibration of the storage units to be calibrated can be achieved. In this way, the time required for debugging and calibration is significantly shortened, the efficiency of visual calibration is effectively improved, and the multi-station visual calibration process is significantly optimized.

[0101] Figure 4 is a flowchart of a visual calibration method shown according to an exemplary embodiment, as Figure 4 shown, the method includes the following steps.

[0102] In step S201, enter the autofocus process.

[0103] First, the focus ring and the exposure ring of the image acquisition device can be fixed. Then, control the robotic arm to move directly above the ROI position of the material storage unit. After that, set an appropriate step size and move the robotic arm in the vertical direction to reach multiple moving positions, and obtain the corresponding material images. Through the corresponding evaluation function, determine the Z-axis position corresponding to the image with the highest evaluation score among the multiple material images, and control the robotic arm to move to this Z-axis position, thereby completing the autofocus process.

[0104] In step S202, obtain the position data of the positioning point.

[0105] In step S203, it is determined whether the current material storage unit is the target storage unit.

[0106] If so, step S204 is executed; If not, step S206 is executed.

[0107] In step S204, the robotic arm performs a nine-point correction and rotation correction process.

[0108] Exemplarily, for the nine-point correction, the reference plane and 9 sampling points can be determined first, the initial position data is obtained, the deviation value and compensation amount are calculated by comparison, and finally the compensation amount is input into the control system for real-time adjustment to complete the correction. When performing rotation correction, an angle sensor is used to detect the deviation, the correction strategy is determined according to the deviation and the motion state, and the control system issues an instruction to the rotation joint motor accordingly to drive the joint to rotate to achieve rotation angle correction. The specific implementation manner thereof can refer to the step flow in the related technology and will not be elaborated herein.

[0109] In step S205, the operator teaches the first picking point and the first placing point corresponding to the target storage unit to obtain the first relative position relationship among the first positioning point, the first picking point, and the first placing point corresponding to the first material in the target storage unit.

[0110] This first relative position relationship is the first calibration data corresponding to the first material in the target storage unit.

[0111] In step S206, according to this first position data and this first calibration data, the second position data of the second picking point and the third position data of the second placing point corresponding to the second material in the storage unit to be calibrated are determined.

[0112] In step S207, according to this first position data, this second position data, and this third position data, the second relative position relationship among the second positioning point, the second picking point, and the second placing point corresponding to the second material in the storage unit to be calibrated is calibrated.

[0113] Regarding the method in the above embodiments, the specific manner of performing operations in each step has been Figures 1 to 3 described in detail in the embodiments related to this method and will not be elaborated herein.

[0114] Using the above method, in the scenario of visual calibration for multiple workstations, it is only necessary to pre-calibrate the first calibration data of the target storage unit among multiple material storage units. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material therein can be automatically obtained. Subsequently, based on the first position data and the calibration result of the target storage unit (i.e., the first calibration data), the automatic calibration of the storage units to be calibrated can be achieved. In this way, the time required for debugging and calibration is significantly shortened, the efficiency of visual calibration is effectively improved, and the multi-station visual calibration process is significantly optimized.

[0115] Figure 5 is a block diagram of a visual calibration device shown according to an exemplary embodiment. As Figure 5 shown, the device 300 includes: A first acquisition module 301, configured to acquire first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among multiple material storage units; the first calibration data is used to characterize the first relative position relationship among the first positioning point, the first material taking point, and the first material placing point of the first material in the target storage unit; the positioning point is used to characterize the position point of the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit. A second acquisition module 302, configured to acquire first position data of the second positioning point of the second material in the storage unit to be calibrated; the storage unit to be calibrated is a non-calibrated material storage unit among multiple material storage units. A determination module 303, configured to determine the second position data of the second material taking point and the third position data of the second material placing point corresponding to the second material according to the first position data and the first calibration data. A calibration module 304, configured to calibrate the second relative position relationship among the second positioning point, the second material taking point, and the second material placing point of the second material in the storage unit to be calibrated according to the first position data, the second position data, and the third position data.

[0116] Optionally, the second acquisition module 302 is configured to control the robotic arm to move so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; the position of the robotic arm when the center of the acquisition field of view of the image acquisition device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is acquired.

[0117] Optionally, the second acquisition module 302 is configured to acquire a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when the image acquisition device acquires the second material; based on the preset image acquisition distance, control the robotic arm to move so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0118] Optionally, the preset image acquisition distance is preset in the following manner: Control the robotic arm to move to an initial position, where the initial position is the position of the robotic arm when the acquisition field of view of the image acquisition device on the robotic arm includes the second material; Based on the specified direction corresponding to the initial position, control the robotic arm to move within the target movement range in the specified direction to acquire the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range; Determine the preset image acquisition distance according to the multiple material images.

[0119] Optionally, the determining the preset image acquisition distance according to the multiple material images includes: For each material image, determine an evaluation score corresponding to the material image; the higher the evaluation score, the higher the clarity of the material image; Take the material image with the highest evaluation score as the target image, and take the movement position corresponding to the target image as the target position; Determine the preset image acquisition distance according to the target position.

[0120] Optionally, the determining the evaluation score corresponding to the material image includes: Determine the gradient value corresponding to each pixel point in the material image; Determine the evaluation score corresponding to the material image according to the multiple gradient values.

[0121] Optionally, the acquiring the material images acquired by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range includes: When the residence time of the robotic arm at each movement position reaches a preset time threshold, control the image acquisition device to acquire the material image.

[0122] Optionally, the first calibration data further includes fourth position data corresponding to the first positioning point, and the determining module 303 is configured to determine the relative position between the first position data and the fourth position data; according to the relative position and the first relative position relationship, determine the second position data corresponding to the second material taking point and the third position data corresponding to the second material placing point.

[0123] With the above device, in the scenario of visual calibration for multiple workstations, only the first calibration data of the target storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning points of the second material therein can be automatically obtained. Subsequently, based on the first position data and the calibration result of the target storage unit (i.e., the first calibration data), the automatic calibration of the storage units to be calibrated can be achieved. In this way, the time required for debugging and calibration is significantly shortened, the efficiency of visual calibration is effectively improved, and the multi-station visual calibration process is significantly optimized.

[0124] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0125] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the visual calibration method provided by the present disclosure are implemented.

[0126] Figure 6 FIG. 400 is a block diagram of an electronic device 400 shown according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0127] Referring to Figure 6 , the electronic device 400 may include one or more of the following components: a processing component 402, a memory 404, a power component 406, a multimedia component 408, an audio component 410, an input / output interface 412, a sensor component 414, and a communication component 416.

[0128] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above visual calibration method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0129] The memory 404 is configured to store various types of data to support the operation of the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, and the like. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0130] The power supply component 406 provides power to various components of the electronic device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 400.

[0131] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0132] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 further includes a speaker for outputting audio signals.

[0133] The input / output interface 412 provides an interface between the processing component 402 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power-on button, and a lock button.

[0134] The sensor assembly 414 includes one or more sensors for providing an assessment of various aspects of the state of the electronic device 400. For example, the sensor assembly 414 can detect the on / off state of the electronic device 400, the relative positioning of components, such as the display and keypad of the electronic device 400. The sensor assembly 414 can also detect a change in the position of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and the temperature change of the electronic device 400. The sensor assembly 414 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0135] The communication component 416 is configured to facilitate communication between the electronic device 400 and other devices in a wired or wireless manner. The electronic device 400 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0136] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described vision calibration method.

[0137] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions that can be executed by a processor 420 of the electronic device 400 to complete the above-described vision calibration method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0138] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for performing the above-described vision calibration method when executed by the programmable device.

[0139] Those skilled in the art can also understand that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of both. Whether such a function is implemented by hardware or software depends on the specific application and the design requirements of the entire system. For each specific application, those skilled in the art can use various methods to implement the described function, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present application.

[0140] In the above detailed description, reference is made to the accompanying drawings, which illustrate by way of illustration specific aspects in which the present disclosure may be practiced. In this regard, directional or positional relationship terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. may be used with reference to the orientation of the described figures. Since the components of the described device can be positioned in a plurality of different orientations, the directional terms are for illustrative purposes and not restrictive. It should be understood that other aspects can be utilized and structural or logical changes can be made without departing from the concepts of the present disclosure. Therefore, the following detailed description should not be taken in a limiting sense.

[0141] It should be understood that, unless otherwise specifically stated, the features of the various embodiments of the present disclosure described herein can be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of any two or more thereof; similarly, "at least one of..." includes any one of the related listed items and any combination of any two or more thereof.

[0142] In addition, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as being advantageous as compared to other aspects or designs.

[0143] Similarly, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and the drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. Specifically with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if not structurally equivalent to the disclosed structure. Additionally, although a particular feature of the present disclosure may have been disclosed with respect to only one of several implementations, such a feature may, as may be desired and advantageous for any given or particular application, be combined with one or more other features of other implementations. Further, with respect to the use of "comprises," "comprising," "has," "having," "includes," or "including" in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term "including."

[0144] Other embodiments of the present disclosure will readily occur to those of ordinary skill in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

[0145] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes may be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A visual calibration method, characterized in that: The method comprises: Acquire first calibration data corresponding to a target storage unit; the target storage unit is a material storage unit that has been pre-calibrated among a plurality of material storage units; the first calibration data is used to characterize a first relative position relationship among a first positioning point, a first material picking point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point is used to characterize a position point corresponding to the robotic arm when an image acquisition device on the robotic arm recognizes a material on the material storage unit; Acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among multiple material storage units; Determine, according to the first position data and the first calibration data, second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material; According to the first position data, the second position data and the third position data, the second relative position relationship between the second positioning point, the second material picking point and the second material discharging point corresponding to the second material in the storage unit to be calibrated is calibrated.

2. The method according to claim 1, characterized in that The step of obtaining the first position data of the second positioning point corresponding to the second material in the storage unit to be calibrated includes: Controlling the movement of the robotic arm so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; The position of the robotic arm when the center of the image acquisition field of view coincides with the center of the first identification point is taken as the second positioning point, and the first position data corresponding to the second positioning point is acquired.

3. The method according to claim 2, characterized in that The controlling the movement of the mechanical arm so that the center of the acquisition field of view of the image acquisition device on the mechanical arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated comprises: Obtaining a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when the image acquisition device acquires the second material; Based on the preset image acquisition distance, the robot arm is controlled to move so that the center of the acquisition field of view of the image acquisition device on the robot arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

4. The method according to claim 3, characterized in that The preset image acquisition distance is preset in the following manner: Controlling the robotic arm to move to an initial position, wherein the initial position is the position of the robotic arm when the image acquisition device on the robotic arm has a capture field of view that includes the second material; Based on the specified direction corresponding to the initial position, controlling the robot arm to move within a target moving range in the specified direction, so as to obtain material images captured by the image acquisition device when the robot arm reaches a plurality of moving positions within the target moving range; The preset image acquisition distance is determined according to the multiple material images.

5. The method according to claim 4, characterized in that Determining the preset image acquisition distance according to the plurality of material images includes: For each material image, determining an evaluation score corresponding to the material image; a higher evaluation score indicates a higher clarity of the material image; The material image with the highest evaluation score is used as the target image, and the moving position corresponding to the target image is used as the target position; The preset image acquisition distance is determined according to the target position.

6. The method according to claim 5, characterized in that Determining the evaluation score corresponding to the material image includes: Determine the gradient value corresponding to each pixel in the material image; An evaluation score corresponding to the material image is determined according to the multiple gradient values.

7. The method according to claim 4, characterized in that When the robot arm reaches a plurality of moving positions within the target moving range, the material images captured by the image acquisition device include: When the residence time of the robot arm at each moving position reaches a preset time threshold, the image acquisition device is controlled to acquire the material image.

8. The method according to any one of claims 1 to 7, characterized in that The first calibration data also includes fourth position data corresponding to the first positioning point, and determining the second position data of the second material picking point and the third position data of the second material discharging point corresponding to the second material according to the first position data and the first calibration data includes: determining a relative position between the first position data and the fourth position data; According to the relative position and the first relative position relationship, second position data corresponding to the second material taking point and third position data corresponding to the second material discharging point are determined.

9. A visual calibration device, characterized in that: The device comprises: The first acquisition module is configured to acquire first calibration data corresponding to a target storage unit; the target storage unit is a material storage unit that has been pre-calibrated among a plurality of material storage units; the first calibration data is used to characterize a first relative position relationship between a first positioning point, a first material picking point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point is used to characterize a position point corresponding to the robotic arm when an image acquisition device on the robotic arm recognizes a material on the material storage unit; A second acquisition module is configured to acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units; A determination module is configured to determine second position data of a second material picking point and third position data of a second material discharging point corresponding to the second material according to the first position data and the first calibration data; The calibration module is configured to calibrate the second relative position relationship between the second positioning point, the second material picking point and the second material discharge point corresponding to the second material in the storage unit to be calibrated according to the first position data, the second position data and the third position data.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the steps of the method described in any one of claims 1 to 8 when calling the executable instructions stored in the memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images