Visual calibration method, device, electronic device, storage medium and program product

Abstract

The present disclosure relates to a visual calibration method, device, electronic device, storage medium and program product, and relates to the field of computer technology. The method includes: obtaining first calibration data corresponding to a target storage unit; the target storage unit is a material storage unit that has been pre-calibrated among multiple material storage units; obtaining first position data of a second positioning point corresponding to a second material in the storage unit to be calibrated; determining second position data of a second material picking point and third position data of a second material discharge point corresponding to the second material based on the first position data and the first calibration data; calibrating the second relative position relationship between the second positioning point, the second material picking point and the second material discharge point corresponding to the second material in the storage unit to be calibrated based on the first position data, the second position data and the third position data. In this way, the time required for debugging and calibration is greatly shortened, the efficiency of visual calibration is effectively improved, and the multi-station visual calibration process is significantly optimized.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a visual calibration method, device, electronic device, storage medium, and program product. Background Art

[0002] Smart manufacturing is a human-machine integrated intelligent system composed of robots and human experts, which performs intelligent activities during the manufacturing process. In order for robots to complete tasks more accurately, it is crucial to clearly understand their spatial position, posture, and relative relationship to surrounding objects. This relies on robot visual calibration.

[0003] However, in the prior art, robot vision calibration mostly relies on manual operations, which requires professionals to manually complete many tedious steps, making the process extremely time-consuming and labor-intensive. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a visual calibration method, device, electronic device, storage medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, a visual calibration method is provided, the method comprising:

[0006] Acquire first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among a plurality of material storage units; the first calibration data is used to characterize a first relative positional relationship between a first positioning point, a first material retrieving point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point is used to characterize a position corresponding to the robotic arm when an image acquisition device on the robotic arm recognizes a material on the material storage unit;

[0007] Acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units;

[0008] Determine second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material according to the first position data and the first calibration data;

[0009] According to the first position data, the second position data and the third position data, the second relative position relationship between the second positioning point, the second material taking point and the second material discharging point corresponding to the second material in the storage unit to be calibrated is calibrated.

[0010] Optionally, obtaining first position data of a second positioning point corresponding to a second material in the storage unit to be calibrated includes:

[0011] Controlling the movement of the robotic arm so that the center of the field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated;

[0012] The position of the robotic arm when the center of the image acquisition device's field of view coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is obtained.

[0013] Optionally, controlling the movement of the robotic arm so that the center of the capture field of view of the image capture device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated includes:

[0014] Obtaining a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when the image acquisition device acquires the second material;

[0015] Based on the preset image acquisition distance, the movement of the robotic arm is controlled so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0016] Optionally, the preset image acquisition distance is pre-set in the following manner:

[0017] Controlling the robotic arm to move to an initial position, where the initial position is the position of the robotic arm when a capture field of view of an image capture device on the robotic arm includes the second material;

[0018] Based on the specified direction corresponding to the initial position, controlling the robotic arm to move within a target movement range in the specified direction, so as to obtain material images captured by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range;

[0019] The preset image acquisition distance is determined based on the multiple material images.

[0020] Optionally, determining the preset image acquisition distance based on the plurality of material images includes:

[0021] For each material image, determining an evaluation score corresponding to the material image; a higher evaluation score indicates a higher clarity of the material image;

[0022] The material image with the highest evaluation score is used as the target image, and the moving position corresponding to the target image is used as the target position;

[0023] The preset image acquisition distance is determined according to the target position.

[0024] Optionally, determining the evaluation score corresponding to the material image includes:

[0025] Determine the gradient value corresponding to each pixel in the material image;

[0026] An evaluation score corresponding to the material image is determined according to the multiple gradient values.

[0027] Optionally, when acquiring the plurality of moving positions of the robotic arm within the target moving range, the material images captured by the image capture device include:

[0028] When the dwell time of the robot arm at each moving position reaches a preset time threshold, the image acquisition device is controlled to acquire the material image.

[0029] Optionally, the first calibration data further includes fourth position data corresponding to the first positioning point, and determining the second position data of the second material picking point and the third position data of the second material discharging point corresponding to the second material based on the first position data and the first calibration data includes:

[0030] determining a relative position between the first position data and the fourth position data;

[0031] According to the relative position and the first relative position relationship, second position data corresponding to the second material taking point and third position data corresponding to the second material discharging point are determined.

[0032] According to a second aspect of an embodiment of the present disclosure, a visual calibration device is provided, the device comprising:

[0033] a first acquisition module configured to acquire first calibration data corresponding to a target storage unit; the target storage unit being a pre-calibrated material storage unit among a plurality of material storage units; the first calibration data being used to characterize a first relative positional relationship between a first positioning point, a first material retrieving point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point being used to characterize a position point corresponding to the robotic arm when an image acquisition device on the robotic arm recognizes a material on the material storage unit;

[0034] A second acquisition module is configured to acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units;

[0035] A determination module is configured to determine second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material based on the first position data and the first calibration data;

[0036] The calibration module is configured to calibrate the second relative position relationship between the second positioning point, the second material picking point and the second material discharge point corresponding to the second material in the storage unit to be calibrated based on the first position data, the second position data and the third position data.

[0037] Optionally, the second acquisition module is configured to control the movement of the robotic arm so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; the position of the robotic arm when the center of the acquisition field of the image acquisition device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is acquired.

[0038] Optionally, the second acquisition module is configured to acquire a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when acquiring the second material; based on the preset image acquisition distance, the movement of the robotic arm is controlled so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0039] Optionally, the preset image acquisition distance is pre-set in the following manner:

[0040] Controlling the robotic arm to move to an initial position, where the initial position is the position of the robotic arm when a capture field of view of an image capture device on the robotic arm includes the second material;

[0041] Based on the specified direction corresponding to the initial position, controlling the robotic arm to move within a target movement range in the specified direction, so as to obtain material images captured by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range;

[0042] The preset image acquisition distance is determined based on the multiple material images.

[0043] Optionally, determining the preset image acquisition distance based on the plurality of material images includes:

[0044] For each material image, determining an evaluation score corresponding to the material image; a higher evaluation score indicates a higher clarity of the material image;

[0045] The material image with the highest evaluation score is used as the target image, and the moving position corresponding to the target image is used as the target position;

[0046] The preset image acquisition distance is determined according to the target position.

[0047] Optionally, determining the evaluation score corresponding to the material image includes:

[0048] Determine the gradient value corresponding to each pixel in the material image;

[0049] An evaluation score corresponding to the material image is determined according to the multiple gradient values.

[0050] Optionally, when acquiring the plurality of moving positions of the robotic arm within the target moving range, the material images captured by the image capture device include:

[0051] When the dwell time of the robot arm at each moving position reaches a preset time threshold, the image acquisition device is controlled to acquire the material image.

[0052] Optionally, the first calibration data also includes fourth position data corresponding to the first positioning point, and the determination module is configured to determine the relative position between the first position data and the fourth position data; based on the relative position and the first relative position relationship, determine the second position data corresponding to the second material picking point and the third position data corresponding to the second material discharge point.

[0053] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the visual calibration method provided in the first aspect of the present disclosure when calling the executable instructions stored on the memory.

[0054] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the visual calibration method provided in the first aspect of the present disclosure are implemented.

[0055] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the visual calibration method provided in the first aspect of the present disclosure.

[0056] The technical solutions provided by the embodiments of the present disclosure may include the following advantageous effects: First, first calibration data corresponding to a target storage unit is obtained; the target storage unit is a pre-calibrated material storage unit among multiple material storage units; the first calibration data is used to represent a first relative positional relationship between a first positioning point, a first material retrieval point, and a first material discharge point corresponding to a first material in the target storage unit; the positioning point represents the position of the robotic arm when an image acquisition device on the robotic arm identifies a material on the material storage unit. Second, first positional data of a second positioning point corresponding to a second material in a storage unit to be calibrated is obtained; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units. Then, based on the first positional data and the first calibration data, second positional data of a second material retrieval point and third positional data of a second material discharge point corresponding to the second material are determined. Finally, based on the first positional data, the second positional data, and the third positional data, the second relative positional relationship between the second positioning point, the second material retrieval point, and the second material discharge point corresponding to the second material in the storage unit to be calibrated is calibrated. Using this method, when performing visual calibration on multiple workstations, only the first calibration data of the target material storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration results of the target storage unit (i.e., the first calibration data), the storage units to be calibrated can be automatically calibrated. This significantly shortens the time required for debugging and calibration, effectively improves the efficiency of visual calibration, and significantly optimizes the multi-workstation visual calibration process.

[0057] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0059] Figure 1 The figure is a flowchart of a visual calibration method according to an exemplary embodiment.

[0060] Figure 2 The figure is a flowchart of another visual calibration method according to an exemplary embodiment.

[0061] Figure 3 The figure is a flowchart of another visual calibration method according to an exemplary embodiment.

[0062] Figure 4The figure is a flowchart of a visual calibration method according to an exemplary embodiment.

[0063] Figure 5 The figure is a block diagram of a visual calibration device according to an exemplary embodiment.

[0064] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0066] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0067] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily to be construed as implying a particular order or precedence. In addition, in the description with reference to the accompanying drawings, the same reference numerals in different drawings represent the same elements.

[0068] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0069] In the description of this disclosure, unless otherwise specified, "plurality" refers to two or more than two, and other quantifiers are similar; "at least one item", "one or more items" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item a can represent any number of a; for another example, one or more items among a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural; "and / or" is a type of relationship that describes the association of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " indicates that the related objects are in an "or" relationship.

[0070] Although operations or steps are described in a particular order in the drawings in the embodiments of the present disclosure, this should not be understood as requiring that these operations or steps be performed in the particular order shown or in a serial order, or that all of the operations or steps shown be performed to obtain a desired result. In the embodiments of the present disclosure, these operations or steps may be performed serially; these operations or steps may also be performed in parallel; or some of these operations or steps may be performed.

[0071] Before introducing the visual calibration method, device, electronic device, storage medium and program product provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. The present disclosure can be applied to scenarios in which robots in the field of intelligent manufacturing perform intelligent activities. In one application scenario, the robot needs to grab materials from multiple material drawers and place them. After receiving the instruction, the robotic arm flexibly rotates the joints, locates the position of the drawer, and accurately extends the gripper to open the drawer. Then, accurately grab the material, then steadily raise the arm, move to the designated placement area (i.e., the material placement point) according to the planned path, slowly put down the material, complete the grabbing and placing action, and then quickly return to the initial position to prepare for the next task. In this scenario, in order to enable the robot to complete the task more accurately, clarify its position and posture in space and its relative relationship with surrounding objects, it is often necessary to perform visual calibration on the robot.

[0072] For robotic devices where a single arm handles multiple material drawers, the camera is typically mounted on the arm. During device commissioning, each drawer requires a one-to-one calibration between the arm and the drawer. Currently, this calibration is performed manually. First, corresponding camera recognition points must be established for each drawer. These marks (also known as fiducials, which, after being identified by an image acquisition device (such as a camera), provide the device with accurate position and orientation (translation / rotation) information and serve as the core benchmark for visual positioning) are photographed and identified. Second, corresponding pick-up and drop-out points must be established for each drawer, and precise manual teaching must be performed. For example, if a robotic device has 16 drawers, 16 camera recognition points must be established, 16 pick-up and 16 drop-out points must be manually taught, and 16 visual calibrations must be performed.

[0073] The inventors have found in long-term practice that the above-mentioned visual calibration method mainly has the following two problems. On the one hand, the efficiency of manual teaching is low and very time-consuming, and the accuracy of judgment through human eye observation is affected by the debugging experience of technicians. A large amount of repetitive manual labor makes the calibration efficiency very low, which seriously restricts the production rhythm and overall efficiency improvement of smart factories, and it is difficult to meet the needs of the rapid development of intelligent manufacturing. On the other hand, the viewing angle and field of view of the human eye are relatively limited. When an object in the field of view of the human eye is blocked by other objects, light cannot enter the human eye, and the line of sight will be blocked. The drawers of multi-drawer equipment are distributed in the upper and lower layers of the equipment. The drawers on the lower layer often have blocked vision, difficult to teach, etc., and there is a high risk of collision.

[0074] In order to solve the above-mentioned technical problems, the present invention provides a visual calibration method, device, electronic device, storage medium and program product. In the scenario of visual calibration of multiple workstations, it is only necessary to pre-calibrate the first calibration data of the target storage unit among multiple material storage units. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration result of the target storage unit (i.e., the first calibration data), automatic calibration of the storage unit to be calibrated can be achieved. In this way, the time required for debugging and calibration is greatly shortened, the efficiency of visual calibration is effectively improved, and the multi-workstation visual calibration process is significantly optimized.

[0075] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0076] Figure 1This is a flowchart of a visual calibration method according to an exemplary embodiment. The method can be used in a terminal device, which can be, for example, a robot. In some embodiments, the robot can include a robotic arm and an image acquisition device. The robotic arm serves as an execution unit, and is used to grab and place materials from a plurality of material storage units on a material rack. For example, the material rack can include a plurality of material storage units arranged in an up-down order, and corresponding materials are placed in each material storage unit. The material storage unit can be, for example, a material drawer. The robotic arm achieves high-precision movement with multi-degree-of-freedom joints, and has the capabilities of flexible grabbing, stable handling, and precise assembly. It is the "execution hand" that supports the robot to complete actual operations. The image acquisition device is provided on the robotic arm, and is used to collect image information of the material on the material storage unit. By acquiring visual data such as the position, posture, and type of the material in real time, and after processing, the robot is provided with environmental perception capabilities, which assists the robotic arm in accurately locating the material, planning the motion path, and ensuring the accuracy of the grabbing and placing operations. The two work together to enable the robot to achieve intelligent interaction of "visual guidance + precise execution" in material processing scenarios. As Figure 1 As shown, the visual calibration method may include the following steps.

[0077] In step S101, first calibration data corresponding to a target storage unit is obtained.

[0078] Among them, the target storage unit is a material storage unit that has been pre-calibrated among multiple material storage units; the first calibration data is used to characterize the first relative position relationship between the first positioning point, the first material picking point and the first material discharge point corresponding to the first material in the target storage unit; the positioning point is used to characterize the position point corresponding to the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit.

[0079] In some embodiments, in order to improve the calibration speed of the terminal device, a material storage unit can be selected from multiple material storage units as a main storage unit, that is, a target storage unit, and the target storage unit can be pre-calibrated.

[0080] Specifically, the first calibration data of the target storage unit can be pre-calibrated and obtained by the following steps:

[0081] First, a first positioning point corresponding to the first material in the target storage unit may be determined.

[0082] The first positioning point can represent the position of the robotic arm when the image acquisition device on the robotic arm identifies the first material on the target storage unit. The first positioning point can also be understood as the first photographic point corresponding to the target storage unit, at which the first material is photographically positioned.

[0083] In order to more accurately grasp and place materials, identification points, also known as Mark points, can be pre-set on the materials. The Mark point can be a position point or a position area. The Mark point can be set at the edge or center of the material according to actual needs. When the center of the image acquisition device's field of view coincides with the identification point, it can be considered that the robotic arm has reached the positioning point. At this time, the positioning point can be automatically focused and photographed to identify the center coordinates of the current positioning point. For example, when the center of the image acquisition device's field of view coincides with the second identification point of the first material, the position point is considered to be the first positioning point. Further, the fourth position data corresponding to the first positioning point can be determined. Among them, the fourth position data can, for example, include the center coordinates of the second identification point, that is, the center coordinates (x, y) of the Mark point.

[0084] Secondly, after completing the positioning of the first positioning point, the first material taking point and the first material discharge point can be further determined.

[0085] In order to ensure the accuracy of subsequent calibration, the position calibration of the first material picking point and the first material discharge point of the target storage unit can be carried out by manual teaching to determine the fifth position data corresponding to the first material picking point and the sixth position data corresponding to the first material discharge point in turn.

[0086] Finally, based on the fourth position data corresponding to the first positioning point, the fifth position data corresponding to the first material picking point and the sixth position data corresponding to the first material discharge point, the first relative position relationship between the first positioning point, the first material picking point and the first material discharge point is determined to realize the calibration of the target storage unit.

[0087] In addition, in order to further improve the accuracy of the robot arm in grasping and placing materials, the robot arm can also be subjected to a nine-point correction and rotation correction process based on the above calibration results. For example, the nine-point correction can first determine the reference plane and 9 sampling points, obtain the initial position data, compare and calculate the deviation value and the compensation amount, and finally input the compensation amount into the control system for real-time adjustment to complete the correction. During rotation correction, the angle sensor is used to detect the deviation, and the correction strategy is determined according to the deviation and the motion state. The control system sends instructions to the rotary joint motor accordingly to drive the joint to rotate and achieve rotation angle correction. Its specific implementation method can refer to the step process in the relevant technology, and will not be described in detail here.

[0088] After completing manual teaching, the first calibration data of the target storage unit can be obtained, thereby ensuring that the material picking position and material placing position of the robot arm to the target storage unit are accurate.

[0089] In step S102, first position data of a second positioning point corresponding to a second material in the storage unit to be calibrated is obtained.

[0090] The storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units. It can be understood that in this embodiment, only the target storage unit needs to be pre-calibrated, and the remaining storage units to be calibrated can be automatically calibrated in the following manner, which greatly saves labor costs and effectively improves the overall calibration efficiency.

[0091] In some embodiments, the movement of the robotic arm can be controlled so that the center of the capture field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated, and the position of the robotic arm when the center of the capture field of view of the image acquisition device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is obtained.

[0092] To improve calibration efficiency and accuracy, the identification points corresponding to all material storage units on the material rack can be set to be consistent. In this way, when subsequently calibrating the first position data of the second positioning points of other storage units to be calibrated, the movement distance of the robot arm can be shortened, improving calibration efficiency.

[0093] In step S103, second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material are determined according to the first position data and the first calibration data.

[0094] Since the first calibration data records the first relative position relationship between the first positioning point, the first material picking point and the first material discharge point corresponding to the first material, after determining the first position data corresponding to the second positioning point of the second material, the second position data of the second material picking point and the third position data corresponding to the second material discharge point can be determined based on the first position data and the first calibration data.

[0095] As can be seen from the foregoing, the second positioning point is the position point corresponding to the robotic arm when the image acquisition device on the robotic arm identifies the second material on the storage unit to be calibrated. In other words, when the robotic arm is at the second positioning point, the image acquisition device on it is able to capture an image of the second material. Therefore, in one implementation, the center position of the second material can also be determined based on the second positioning point, and the second material collection point can be determined based on the center position and the vertical distance between the robotic arm and the second material. In order to improve the efficiency of the second material collection point calibration, the first identification point can be set at the center position of the second material, which can shorten the time it takes for the robotic arm to move to find the center position of the second material.

[0096] In step S104, the second relative position relationship between the second positioning point, the second material taking point and the second material discharge point corresponding to the second material in the storage unit to be calibrated is calibrated based on the first position data, the second position data and the third position data.

[0097] After obtaining the first position data, the second position data and the third position data, that is, mastering the positions of the second positioning point, the second material picking point and the second material discharge point, the second relative position relationship between the second positioning point, the second material picking point and the second material discharge point corresponding to the second material in the storage unit to be calibrated can be determined, thereby completing the calibration of the remaining storage units to be calibrated.

[0098] Using this method, when performing visual calibration on multiple workstations, only the first calibration data of the target material storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration results of the target storage unit (i.e., the first calibration data), the storage units to be calibrated can be automatically calibrated. This significantly shortens the time required for commissioning and calibration, effectively improves the efficiency of visual calibration, and significantly optimizes the multi-workstation visual calibration process.

[0099] Image clarity is an important indicator of image quality. If the industrial lens is not focused accurately, the image will become blurry and unclear, thus affecting the visual calibration effect. Existing industrial lens focusing solutions all require the debugging personnel to fix the lens focus ring after the camera lens is installed, and then manually adjust the focus. Specifically, first, the working distance of the camera lens is adjusted with a fixed step size, and at the same time, a photo is taken and the current image quality is manually judged. If the current image quality meets the optimal focus effect judged by the human, it is considered that the focus is currently achieved. Otherwise, the working distance of the camera lens is adjusted by a fixed step size and judged until the final focus is achieved.

[0100] However, existing manual focusing solutions are greatly influenced by the subjective judgment of the debugging personnel and are highly dependent on the debugging experience of the debugging personnel. It is easy for the focus to be unclear, which in turn affects the subsequent positioning accuracy.

[0101] Based on the above situation, and to improve focusing accuracy and subsequent positioning precision, this disclosure proposes an autofocus method for an image acquisition device to focus and photograph materials and determine positioning points. By automatically calculating the optimal working distance for each material storage unit, autofocus is achieved using objective standards, eliminating the problem of poor imaging quality caused by manual camera lens focusing, which is affected by personal subjective experience and debugging experience.

[0102] The following takes the above step S102 as an example to describe the auto-focus method provided by the present disclosure in detail. Figure 2 As shown, the step S102 of obtaining the first position data of the second positioning point corresponding to the second material in the storage unit to be calibrated may include the following steps:

[0103] In step S1021, the robot arm is controlled to move so that the center of the acquisition field of the image acquisition device on the robot arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0104] First, a preset image acquisition distance corresponding to the second material may be obtained.

[0105] The preset image acquisition distance is the distance between the image acquisition device and the second material when capturing the second material. The preset image acquisition distance can be understood as the position at which the image acquisition device captures the second material, resulting in the highest image clarity. In other words, the preset image acquisition distance is the optimal focusing distance for the image acquisition device to capture the second material.

[0106] The preset image acquisition distance can be pre-set in the following ways:

[0107] (1) Control the robot arm to move to the initial position.

[0108] The initial position is the position of the robotic arm when the capture field of view of the image acquisition device on the robotic arm includes the second material. For example, the initial position can be determined based on the ROI (Region Of Interest, which refers to a specific area of ​​interest to the user extracted from an image or video. This area can be an object, a specific part of a scene, or an area with certain characteristics) position of the storage unit to be calibrated, that is, at the initial position, the image acquisition device can capture the ROI position of the second material. The ROI position can be understood as the position area that includes the second material. The initial position can be, for example, directly above the ROI position, and in order to avoid the risk of collision, the height corresponding to the initial position (that is, the distance between the image acquisition device and the material) can be slightly larger than the preset working distance of the robotic arm.

[0109] In some embodiments, the focus ring of an image capture device mounted on a robotic arm can be rotated to a first preset position (e.g., the middle) and fixed, while the exposure ring of the lens can be rotated to a second preset position (e.g., the middle) and fixed. The robotic arm equipped with the image capture device can then be moved to a position above the second material, i.e., the initial position. In other words, the focus distance and exposure parameters of the image capture device are fixed at this point. By adjusting the distance between the image capture device and the material, the image captured within the field of view of the image capture device can be made clearer.

[0110] (2) Based on the specified direction corresponding to the initial position, the robot arm is controlled to move within a target movement range in the specified direction, so as to obtain material images captured by the image acquisition device when the robot arm reaches multiple movement positions within the target movement range.

[0111] The designated direction may include vertically upward, thereby ensuring that there is no risk of collision. The target movement range may be a preset vertically upward range, and the target movement range may cover the maximum variable range of the zoom ring of the image acquisition device, that is, the target movement range may be determined based on the maximum variable range.

[0112] In some embodiments, based on the specified direction corresponding to the initial position, the robotic arm can be controlled to move within the target moving range of the specified direction. Every time it moves a certain step distance (for example, 0.1 mm), it reaches a moving position. At this time, the image acquisition device can be controlled to capture an image of the material containing the second material.

[0113] In other embodiments, the image capture device may shake after the robot arm moves, thereby causing distortion in the material image. Therefore, to ensure the quality of the material image, the image capture device is controlled to capture the material image only when the robot arm's dwell time at each movement position reaches a preset time threshold.

[0114] For example, if the zoom ring of the image acquisition device can rotate with a maximum variable range of 3mm and a step movement distance of 0.1mm, it should be set to move 30 times. In addition, after each robotic arm is in place, a 200ms delay can be made before taking a picture to avoid image deformation caused by movement.

[0115] (3) Determine the preset image acquisition distance based on the multiple images of the material.

[0116] In some embodiments, the preset image acquisition distance may be determined by the following steps:

[0117] a) For each material image, determine the evaluation score corresponding to the material image.

[0118] The higher the evaluation score, the higher the clarity of the material image.

[0119] Specifically, the evaluation score corresponding to each material image can be determined using a pre-set evaluation function. For example, the evaluation function may include, but is not limited to, any of the following: a variance function (which evaluates image clarity by calculating the variance or standard deviation of the image to construct an evaluation function), a Laplace function (which performs a convolution operation on the grayscale value of each pixel in the image to obtain a gradient matrix, and calculates the square sum of the matrices to obtain an evaluation function), an energy gradient function (which uses the difference between adjacent points to calculate the gradient of a point as an evaluation function), a Brenner function (which calculates the gradient in a direction by calculating the difference in the grayscale values ​​of pixels separated by two units and then squaring this difference as an evaluation function), and a Tenegrad function (which uses the Sobel operator to evaluate horizontal and vertical gradient values, amplifies image edge gradients using a squaring operation, and defines the evaluation function as the square sum of the gradients).

[0120] In a possible implementation, the gradient value corresponding to each pixel in the material image may be determined first, and then the evaluation score corresponding to the material image may be determined based on the multiple gradient values.

[0121] b) The material image with the highest evaluation score is used as the target image, and the moving position corresponding to the target image is used as the target position.

[0122] The material image with the highest evaluation score is the image with the highest definition. Therefore, the material image with the highest evaluation score can be used as the target image, and the moving position corresponding to the target image can be used as the target position.

[0123] c) determining the preset image acquisition distance according to the target position.

[0124] Determine the preset image acquisition distance based on the target position on the Z axis.

[0125] Then, based on the preset image acquisition distance, the robot arm is controlled to move so that the center of the acquisition field of the image acquisition device on the robot arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0126] In the case where the identification points corresponding to all material storage units are consistent, since it has been pre-calibrated that the position of the first positioning point of the target storage unit is completely coincident with the center of the acquisition field of view and the second identification point, the robot arm is controlled to move based on the preset image acquisition distance so that the center of the acquisition field of view of the image acquisition device on the robot arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated. The method may also include:

[0127] Based on the preset image acquisition distance, the movement of the robotic arm is controlled so that when the deviation between the acquisition field center of the image acquisition device on the robotic arm and the center of the first identification point corresponding to the second material in the storage unit to be calibrated is less than the preset pixel unit, the two can be considered to coincide.

[0128] That is to say, on the basis of ensuring the accuracy of the first positioning point, a certain small range of deviation is allowed for the remaining storage units to be calibrated, which neither affects the calibration effect nor improves the calibration efficiency.

[0129] For example, based on a preset image acquisition distance, the robotic arm is controlled to lock at the position corresponding to the Z axis, meaning that the robotic arm subsequently does not move in the Z axis direction. The robotic arm is then controlled to move along the X and Y axes so that when the deviation between the capture field of view center (x, y) of the image acquisition device on the robotic arm and the center of the first identification point (xi, yi) corresponding to the second material in the storage unit to be calibrated is less than 2 pixels, that is, |x-xi|≤2 & |y-yi|≤2, the two are considered to coincide, and the second positioning point has been found.

[0130] In step S1022, the position of the robotic arm when the center of the capturing field of view of the image capturing device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is obtained.

[0131] It should be noted that the above-mentioned automatic focusing process can also be applied to a scenario where the target storage unit is determined to focus when the fourth position data corresponding to the first positioning point is determined.

[0132] The following describes the above step S103 in detail. Figure 3 As shown, in the above step S103, determining the second position data of the second material picking point and the third position data of the second material discharging point corresponding to the second material according to the first position data and the first calibration data may include:

[0133] In step S1031 , the relative position between the first position data and the fourth position data is determined.

[0134] In step S1032, second position data corresponding to the second material taking point and third position data corresponding to the second material discharging point are determined based on the relative position and the first relative position relationship.

[0135] In one possible implementation, if the positioning points corresponding to all material storage units on the material rack have the same relative position relationship with the material picking point and the material discharge point, then when the relative position between the first position data and the fourth position data is known, the second position data corresponding to the second material picking point and the third position data corresponding to the second material discharge point can be directly determined based on the relative position and the first relative position relationship.

[0136] In another possible implementation method, if the positions of the picking points and discharging points corresponding to all material storage units on the material rack are the same, that is, the fifth position data of the first discharging point corresponding to the target storage unit can be directly used as the second position data corresponding to the second picking point, and the sixth position data of the first discharging point corresponding to the target storage unit can be used as the third position data corresponding to the second discharging point.

[0137] In another possible implementation, if the relative positional relationships between the positioning points corresponding to all material storage units on the material rack and the material retrieval points and discharge points are different, and the positions of the material retrieval points and discharge points corresponding to all material storage units on the material rack are also different, then a second relative positional relationship between each material retrieval point and a third relative positional relationship between each discharge point can be predetermined. After determining the first calibration data, the second positional data corresponding to the second material retrieval point and the third positional data corresponding to the second discharge point are determined based on the relative positions, the first relative positional relationship, the second relative positional relationship, and the third relative positional relationship.

[0138] Using this method, when performing visual calibration on multiple workstations, only the first calibration data of the target material storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration results of the target storage unit (i.e., the first calibration data), the storage units to be calibrated can be automatically calibrated. This significantly shortens the time required for commissioning and calibration, effectively improves the efficiency of visual calibration, and significantly optimizes the multi-workstation visual calibration process.

[0139] Figure 4 is a flow chart of a visual calibration method according to an exemplary embodiment. Figure 4 As shown, the method includes the following steps.

[0140] In step S201 , the auto-focus process is entered.

[0141] First, the focus and exposure rings of the image acquisition device can be fixed. Then, the robotic arm is controlled to move directly above the ROI location of the material storage unit. After that, an appropriate step size is set and the robotic arm is moved vertically to multiple positions, capturing corresponding material images. Using a corresponding evaluation function, the Z-axis position corresponding to the image with the highest evaluation score among the multiple material images is determined, and the robotic arm is controlled to move to that Z-axis position, completing the autofocus process.

[0142] In step S202, the position data of the positioning point is obtained.

[0143] In step S203, it is determined whether the current material storage unit is the target storage unit.

[0144] If yes, execute step S204;

[0145] If not, execute step S206.

[0146] In step S204 , the robotic arm performs a nine-point correction and rotation correction process.

[0147] For example, nine-point correction can first determine the reference plane and nine sampling points, obtain the initial position data, compare and calculate the deviation value and the compensation amount, and finally input the compensation amount into the control system for real-time adjustment to complete the correction. During rotation correction, the angle sensor is used to detect the deviation, and the correction strategy is determined based on the deviation and motion state. The control system then sends instructions to the rotary joint motor to drive the joint to rotate and achieve rotation angle correction. Its specific implementation method can refer to the step-by-step process in the relevant technology, and will not be detailed here.

[0148] In step S205, the first material-retrieving point and the first material-discharging point corresponding to the target storage unit are manually taught to obtain a first relative position relationship among the first positioning point, the first material-retrieving point and the first material-discharging point corresponding to the first material in the target storage unit.

[0149] The first relative position relationship is the first calibration data corresponding to the first material in the target storage unit.

[0150] In step S206, second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material in the storage unit to be calibrated are determined based on the first position data and the first calibration data.

[0151] In step S207, the second relative position relationship between the second positioning point, the second material taking point and the second material discharging point corresponding to the second material in the storage unit to be calibrated is calibrated based on the first position data, the second position data and the third position data.

[0152] Regarding the method in the above embodiment, the specific way of performing the operation in each step has been Figures 1 to 3 The method is described in detail in the embodiments and will not be elaborated here.

[0153] Using this method, when performing visual calibration on multiple workstations, only the first calibration data of the target material storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration results of the target storage unit (i.e., the first calibration data), the storage units to be calibrated can be automatically calibrated. This significantly shortens the time required for commissioning and calibration, effectively improves the efficiency of visual calibration, and significantly optimizes the multi-workstation visual calibration process.

[0154] Figure 5 is a block diagram of a visual calibration device according to an exemplary embodiment. Figure 5 As shown, the device 300 includes:

[0155] The first acquisition module 301 is configured to acquire first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among a plurality of material storage units; the first calibration data is used to represent a first relative positional relationship between a first positioning point, a first material picking point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point is used to represent a position point corresponding to the robotic arm when the image acquisition device on the robotic arm recognizes the material on the material storage unit;

[0156] The second acquisition module 302 is configured to acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units;

[0157] A determination module 303 is configured to determine second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material based on the first position data and the first calibration data;

[0158] The calibration module 304 is configured to calibrate the second relative position relationship between the second positioning point, the second material picking point and the second material discharge point corresponding to the second material in the storage unit to be calibrated based on the first position data, the second position data and the third position data.

[0159] Optionally, the second acquisition module 302 is configured to control the movement of the robotic arm so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; the position of the robotic arm when the center of the acquisition field of the image acquisition device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is acquired.

[0160] Optionally, the second acquisition module 302 is configured to acquire a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when acquiring the second material; based on the preset image acquisition distance, the movement of the robotic arm is controlled so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

[0161] Optionally, the preset image acquisition distance is pre-set in the following manner:

[0162] Controlling the robotic arm to move to an initial position, where the initial position is the position of the robotic arm when a capture field of view of an image capture device on the robotic arm includes the second material;

[0163] Based on a specified direction corresponding to the initial position, controlling the robotic arm to move within a target movement range in the specified direction, so as to obtain images of the material captured by the image capture device when the robotic arm reaches a plurality of movement positions within the target movement range;

[0164] The preset image acquisition distance is determined based on the plurality of material images.

[0165] Optionally, determining the preset image acquisition distance based on the plurality of material images includes:

[0166] For each material image, determining an evaluation score corresponding to the material image; a higher evaluation score indicates a higher clarity of the material image;

[0167] The material image with the highest evaluation score is used as the target image, and the moving position corresponding to the target image is used as the target position;

[0168] The preset image acquisition distance is determined according to the target position.

[0169] Optionally, determining the evaluation score corresponding to the material image includes:

[0170] Determine the gradient value corresponding to each pixel in the material image;

[0171] An evaluation score corresponding to the material image is determined based on the multiple gradient values.

[0172] Optionally, when acquiring the plurality of moving positions of the robot arm within the target moving range, the material images captured by the image capture device include:

[0173] When the dwell time of the robot arm at each moving position reaches a preset time threshold, the image acquisition device is controlled to acquire the material image.

[0174] Optionally, the first calibration data also includes fourth position data corresponding to the first positioning point, and the determination module 303 is configured to determine the relative position between the first position data and the fourth position data; based on the relative position and the first relative position relationship, determine the second position data corresponding to the second material picking point and the third position data corresponding to the second material discharge point.

[0175] Using this device, when performing visual calibration on multiple workstations, only the first calibration data of the target material storage unit among multiple material storage units needs to be pre-calibrated. For the remaining storage units to be calibrated, the first position data corresponding to the second positioning point of the second material can be automatically obtained. Subsequently, based on the first position data and the calibration results of the target storage unit (i.e., the first calibration data), the storage units to be calibrated can be automatically calibrated. This significantly shortens the time required for debugging and calibration, effectively improves the efficiency of visual calibration, and significantly optimizes the multi-workstation visual calibration process.

[0176] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0177] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the visual calibration method provided by the present disclosure when the program instructions are executed by a processor.

[0178] Figure 6 4 is a block diagram of an electronic device 400 according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0179] Reference Figure 6 , the electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output interface 412 , a sensor component 414 , and a communication component 416 .

[0180] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the visual calibration method described above. In addition, the processing component 402 may include one or more modules to facilitate interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.

[0181] The memory 404 is configured to store various types of data to support operations on the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0182] The power supply assembly 406 provides power to the various components of the electronic device 400. The power supply assembly 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 400.

[0183] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, it may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the electronic device 400 is in an operating mode, such as a capture mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and the rear-facing camera can have a fixed optical lens system or have focal length and optical zoom capabilities.

[0184] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0185] The input / output interface 412 provides an interface between the processing component 402 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0186] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of components, such as the display and keypad of the electronic device 400. The sensor assembly 414 can also detect changes in the position of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and temperature changes of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0187] The communication component 416 is configured to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0188] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned visual calibration method.

[0189] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions. The instructions can be executed by the processor 420 of the electronic device 400 to perform the above-mentioned visual calibration method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0190] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device, and has a code portion for performing the above-mentioned visual calibration method when executed by the programmable device.

[0191] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented through electronic hardware, computer software, or a combination of both. Whether such functions are implemented through hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0192] In the foregoing detailed description, reference is made to the accompanying drawings, which illustrate, by way of illustration, specific aspects of the present disclosure in which it may be practiced. In this regard, terms indicating directions or expressing positional relationships, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," "clockwise," "counterclockwise," "axial," "radial," "circumferential," etc., may be used with reference to the orientation of the figures being described. Since the components of the described devices may be positioned in a plurality of different orientations, the directional terms may be used for illustrative purposes rather than restrictive. It should be understood that other aspects may be utilized and structural or logical changes may be made without departing from the concepts of the present disclosure. Therefore, the following detailed description should not be taken in a limiting sense.

[0193] It should be understood that, unless otherwise specifically noted, the features of the various embodiments of the present disclosure described herein may be combined with each other. As used herein, the term "and / or" includes any one of the relevant listed items and any combination of any two or more thereof; similarly, "at least one of" includes any one of the relevant listed items and any combination of any two or more thereof.

[0194] Furthermore, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs.

[0195] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. With particular regard to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if not structurally equivalent to the disclosed structure. In addition, although particular features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms "include," "have," "have," "have," or variations thereof are used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term "comprising."

[0196] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

[0197] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A visual calibration method, characterized in that: The method comprises: Acquire first calibration data corresponding to a target storage unit; the target storage unit is a pre-calibrated material storage unit among a plurality of material storage units; the first calibration data is used to characterize a first relative positional relationship between a first positioning point, a first material retrieving point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point is used to characterize a position corresponding to the robotic arm when an image acquisition device on the robotic arm recognizes a material on the material storage unit; Acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units; Determine, based on the first position data and the first calibration data, second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material; Calibrate a second relative position relationship between a second positioning point, a second material taking point, and a second material discharging point corresponding to the second material in the storage unit to be calibrated according to the first position data, the second position data, and the third position data; The obtaining of first position data of a second positioning point corresponding to a second material in the storage unit to be calibrated includes: Controlling the movement of the robotic arm so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; The position of the robotic arm when the center of the image acquisition device's field of view coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is obtained.

2. The method according to claim 1, characterized in that The controlling the movement of the robotic arm so that the center of the capture field of view of the image capture device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated includes: Obtaining a preset image acquisition distance corresponding to the second material; the preset image acquisition distance is the distance between the image acquisition device and the second material when the image acquisition device acquires the second material; Based on the preset image acquisition distance, the movement of the robotic arm is controlled so that the center of the acquisition field of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated.

3. The method according to claim 2, characterized in that The preset image acquisition distance is preset in the following manner: Controlling the robotic arm to move to an initial position, where the initial position is the position of the robotic arm when a capture field of view of an image capture device on the robotic arm includes the second material; Based on the specified direction corresponding to the initial position, controlling the robotic arm to move within a target movement range in the specified direction, so as to obtain material images captured by the image acquisition device when the robotic arm reaches multiple movement positions within the target movement range; The preset image acquisition distance is determined based on the multiple material images.

4. The method according to claim 3, characterized in that The determining the preset image acquisition distance according to the plurality of material images includes: For each material image, determining an evaluation score corresponding to the material image; a higher evaluation score indicates a higher clarity of the material image; The material image with the highest evaluation score is used as the target image, and the moving position corresponding to the target image is used as the target position; The preset image acquisition distance is determined according to the target position.

5. The method according to claim 4, characterized in that Determining the evaluation score corresponding to the material image includes: Determine the gradient value corresponding to each pixel in the material image; An evaluation score corresponding to the material image is determined according to the multiple gradient values.

6. The method according to claim 3, characterized in that The material images acquired by the image acquisition device when the robot arm reaches multiple moving positions within the target moving range include: When the dwell time of the robot arm at each moving position reaches a preset time threshold, the image acquisition device is controlled to acquire the material image.

7. The method according to any one of claims 1 to 6, characterized in that The first calibration data further includes fourth position data corresponding to the first positioning point, and determining the second position data of the second material picking point and the third position data of the second material discharging point corresponding to the second material based on the first position data and the first calibration data includes: determining a relative position between the first position data and the fourth position data; According to the relative position and the first relative position relationship, second position data corresponding to the second material taking point and third position data corresponding to the second material discharging point are determined.

8. A visual calibration device, characterized in that: The device comprises: a first acquisition module configured to acquire first calibration data corresponding to a target storage unit; the target storage unit being a pre-calibrated material storage unit among a plurality of material storage units; the first calibration data being used to characterize a first relative positional relationship between a first positioning point, a first material retrieving point, and a first material discharging point corresponding to a first material in the target storage unit; the positioning point being used to characterize a position point corresponding to the robotic arm when an image acquisition device on the robotic arm recognizes a material on the material storage unit; A second acquisition module is configured to acquire first position data of a second positioning point corresponding to a second material in a storage unit to be calibrated; the storage unit to be calibrated is an uncalibrated material storage unit among the multiple material storage units; A determination module is configured to determine second position data of a second material taking point and third position data of a second material discharging point corresponding to the second material based on the first position data and the first calibration data; a calibration module configured to calibrate a second relative position relationship between a second positioning point, a second material taking point, and a second material discharging point corresponding to the second material in the storage unit to be calibrated based on the first position data, the second position data, and the third position data; The second acquisition module is configured to control the movement of the robotic arm so that the center of the acquisition field of view of the image acquisition device on the robotic arm coincides with the center of the first identification point corresponding to the second material in the storage unit to be calibrated; the position of the robotic arm when the center of the acquisition field of view of the image acquisition device coincides with the center of the first identification point is used as the second positioning point, and the first position data corresponding to the second positioning point is acquired.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the steps of the method according to any one of claims 1 to 7 when calling the executable instructions stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images