Material feeding method, device, storage medium and electronic device

Abstract

The application provides a material online method and device, a storage medium and an electronic equipment. The method comprises the following steps: obtaining to-be-grabbed material information, and determining a grabbing coordinate according to the to-be-grabbed material information; determining whether the grabbing coordinate is located in a predetermined working area; in the case that the grabbing coordinate is located in the predetermined working area, determining whether the level of a material placement area where the to-be-grabbed material is located is consistent with the current allowed grabbing level according to the triggering state of a plurality of sensors; in the case that the level is consistent with the current allowed grabbing level, controlling a robot to grab the to-be-grabbed material according to the grabbing coordinate, so as to complete the online of the to-be-grabbed material. The application solves the problem that the robot is prone to collision in the process of grabbing the material in the prior art, thereby reducing the production efficiency.

Description

Technical Field

[0001] This application relates to the fields of robot control and industrial vision technology, and more specifically, to a method for loading materials onto a production line, a device for loading materials onto a production line, a computer-readable storage medium, and an electronic device. Background Technology

[0002] Common material handling methods include using robots to pick up materials from pallets and accurately place them at designated workstations. In existing technologies, robots often collide with workbenches or other materials during the material picking process, leading to decreased production efficiency and increased maintenance costs. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, computer-readable storage medium, and electronic device for loading materials onto a production line, so as to at least solve the problem in the prior art that robots are prone to collisions during the material handling process, which leads to a decrease in production efficiency.

[0004] To achieve the above objectives, according to one aspect of this application, a method for loading materials online is provided, comprising: acquiring information about materials to be grasped, and determining grasping coordinates based on the information about materials to be grasped, wherein the information about materials to be grasped includes the size and position of the materials to be grasped, and the grasping coordinates characterize data on the spatial position and grasping posture that a robot should reach when grasping the materials to be grasped based on the size and position; determining whether the grasping coordinates are located within a predetermined working area; if the grasping coordinates are located within the predetermined working area, determining whether the level of the material placement area where the materials to be grasped are located is consistent with the currently allowed grasping level based on the triggering states of multiple sensors; and if the level is consistent with the currently allowed grasping level, controlling the robot to grasp the materials to be grasped based on the grasping coordinates to complete the loading of the materials to be grasped online.

[0005] Optionally, determining whether the grasping coordinates are located within a predetermined working area includes: identifying the boundary of the material placement area where the material to be grasped is located, and determining a verification point located on the boundary; determining a first coordinate as the position coordinate of the verification point, and controlling the robot to move towards the first coordinate for a preset time, and controlling the robot to stop moving after the preset time is reached; determining the position coordinate of the robot when it stops as a second coordinate; determining that the grasping coordinates are located within the predetermined working area when the first coordinate is equal to the second coordinate; and determining that the grasping coordinates are not located within the predetermined working area when the first coordinate is not equal to the second coordinate.

[0006] Optionally, after controlling the robot to grasp the material to be grasped according to the grasping coordinates, the method further includes: acquiring an image of a bottom support and determining the center of gravity coordinates of the bottom support, the bottom support being used to place the material to be grasped; determining the placement height and offset angle of the material to be grasped according to the image, and obtaining the placement coordinates based on the center of gravity coordinates, the placement height being the target height the robot descends when placing the material to be grasped, the offset angle being the deviation angle between the current orientation of the bottom support and a preset standard orientation; and controlling the robot to place the material to be grasped on the bottom support according to the placement coordinates.

[0007] Optionally, after obtaining the information of the material to be grasped, the method further includes: obtaining the current position of the robot; if the current position of the robot is not located at the preset initial position, controlling the robot to stop the current action and move to the preset initial position.

[0008] Optionally, each level of the material placement area is equipped with a sensor. Based on the triggering states of multiple sensors, it is determined whether the level of the material placement area where the material to be grasped is located is consistent with the currently allowed grasping level. This includes: determining the height value of the grasping coordinates of the material to be grasped; if the height value is greater than a first preset height value and multiple sensors detect a target signal, determining that the level where the material to be grasped is located is consistent with the currently allowed grasping level; if the height value is less than or equal to the first preset height value and greater than a second preset height value, and a predetermined sensor detects the target signal, determining that the level where the material to be grasped is located is consistent with the currently allowed grasping level, wherein the first preset height value is greater than or equal to the second preset height value; and if the height value is less than or equal to the second preset height value, and the sensor at the bottom layer of the material placement area detects the target signal, determining that the level where the material to be grasped is located is consistent with the currently allowed grasping level.

[0009] Optionally, acquiring information about the material to be grasped and determining the grasping coordinates based on the information about the material to be grasped includes: acquiring point cloud data of the material to be grasped, wherein the point cloud data represents the size and position information of the material to be grasped; comparing the point cloud data with a preset standard material template to obtain the grasping coordinates, wherein the standard material template includes the point cloud data obtained by collecting preset standard materials and the corresponding grasping coordinates.

[0010] Optionally, comparing the point cloud data with a preset standard material template includes: when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is greater than or equal to a preset value, determining the grasping coordinates corresponding to the point cloud data in the standard material template as the grasping coordinates of the material to be grasped; when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is less than the preset value, outputting an alarm message indicating recognition failure and pausing the current grasping task.

[0011] According to another aspect of this application, a material loading device is provided, comprising: an acquisition unit, configured to acquire information about a material to be grasped and determine grasping coordinates based on the information about the material to be grasped, wherein the information about the material to be grasped includes the size and position of the material to be grasped, and the grasping coordinates characterize data of the spatial position and grasping posture that a robot should reach when grasping the material to be grasped based on the size and position; a determination unit, configured to determine whether the grasping coordinates are located within a predetermined working area, and if the grasping coordinates are located within the predetermined working area, determine whether the level of the material placement area where the material to be grasped is located is consistent with the currently allowed grasping level based on the triggering state of multiple sensors; and a control unit, configured to control the robot to grasp the material to be grasped according to the grasping coordinates when the level is consistent with the currently allowed grasping level, so as to complete the loading of the material to be grasped.

[0012] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0013] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any one of the methods described.

[0014] The technical solution of this application first obtains information about the material to be grasped, including its size and position, and determines the grasping coordinates based on this information. Next, it determines whether the grasping coordinates are within a predetermined working area. If the grasping coordinates are within the predetermined working area, it determines whether the layer of the material placement area where the material to be grasped is located matches the currently allowed grasping layer based on the triggering status of multiple sensors. Finally, if the layer matches the currently allowed grasping layer, the robot is controlled to grasp the material according to the grasping coordinates, thus completing the loading of the material. This solution obtains the grasping coordinates by acquiring the size and position of the material to be grasped and determines that the grasping coordinates are within a predetermined working area before grasping, thereby ensuring that the robot can reach the grasping coordinates and avoiding collisions and other abnormal situations caused by the robot failing to reach the grasping coordinates and executing an incorrect path. By determining the trigger status information of multiple sensors, the current level of the material is verified. The material is grasped only when the current level meets the conditions that allow grasping, thereby avoiding collisions between the robot and the guardrail or the material. This solves the problem in the prior art where robots are prone to collisions during material grasping, which leads to a decrease in production efficiency. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 A flowchart illustrating a method for material delivery according to an embodiment of this application is shown.

[0017] Figure 2 A schematic diagram of a compressor online system according to an embodiment of this application is shown;

[0018] Figure 3 A flowchart illustrating yet another method for material delivery according to an embodiment of this application is shown.

[0019] Figure 4 A structural block diagram of a material loading device according to an embodiment of this application is shown. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] As described in the background section, existing technologies often involve collisions during the material handling process, leading to decreased production efficiency. To address these technical problems, embodiments of this application provide a method for loading materials onto a production line, a device for loading materials onto a production line, a computer-readable storage medium, and an electronic device.

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0025] Figure 1 This is a flowchart of a material delivery method according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0026] Step S101: Obtain information about the material to be grasped, and determine the grasping coordinates based on the information about the material to be grasped. The information about the material to be grasped includes the size and position of the material to be grasped. The grasping coordinates represent the data of the spatial position and grasping posture that the robot should reach when grasping the material to be grasped based on the size and position.

[0027] Specifically, information about the material to be grasped is obtained by taking pictures with a 3D camera to obtain the size and position of the material to be grasped. According to the pre-set grasping priority and the preset grasping order rules, such as prioritizing the compressor in the X direction, a grasping coordinate for robot grasping is determined and output.

[0028] Step S102: Determine whether the above-mentioned grabbing coordinates are located within the predetermined working area. If the above-mentioned grabbing coordinates are within the predetermined working area, determine whether the level of the material placement area where the material to be grabbed is located is consistent with the currently allowed grabbing level based on the triggering status of multiple sensors.

[0029] Specifically, the aforementioned sensors are through-beam grating sensors or photoelectric switches deployed on each layer containing the material. These sensors are used to determine in real time whether material exists on each layer. If the height corresponding to the grasping coordinate matches the layer detected by the sensor, the robot is allowed to perform the grasping operation. For example, if there are three layers of material to be grasped, and three sets of grating sensors are installed: the trigger state of the upper grating is DI_TOP = 1, the trigger state of the middle grating is DI_MID = 1, and the trigger state of the lower grating is DI_BOT = 1. If the Z-coordinate (height coordinate) in the grasping coordinate is within the height range of the "middle layer," and the corresponding grating signals are DI_TOP = 0, DI_MID = 1, and DI_BOT = 1, then the robot is currently grasping the middle layer, and the sensor also indicates that there is indeed material to be grasped in the middle layer. Therefore, the conditions are met, and the robot can grasp the material. Conversely, if the Z coordinate is in the lower layer range, but DI_TOP=1, DI_MID=1, DI_BOT=1 (indicating that there is material to be grabbed in the upper layer), it means that what needs to be grabbed is actually the lower layer material to be grabbed that is blocked by the upper layer, so grabbing is not allowed.

[0030] Step S103: If the above-mentioned level is consistent with the currently allowed level for grasping, control the robot to grasp the material to be grasped according to the grasping coordinates, so as to complete the online loading of the material to be grasped.

[0031] The above embodiments first acquire information about the material to be grasped, including its size and location, and then determine the grasping coordinates based on this information. Next, it is determined whether the grasping coordinates are within a predetermined working area. If the grasping coordinates are within the predetermined working area, the trigger status of multiple sensors is used to determine whether the layer of the material placement area where the material to be grasped is located matches the currently allowed grasping layer. Finally, if the layer matches the currently allowed grasping layer, the robot is controlled to grasp the material according to the grasping coordinates, thus completing the loading of the material. This solution obtains the grasping coordinates by acquiring the size and location of the material to be grasped and determines that the grasping coordinates are within a predetermined working area before grasping, thereby ensuring that the robot can reach the grasping coordinates and avoiding collisions and other abnormal situations caused by the robot failing to reach the grasping coordinates and executing an incorrect path. By determining the trigger status information of multiple sensors, the current level of the material is verified. The material is grasped only when the current level meets the conditions that allow grasping, thereby avoiding collisions between the robot and the guardrail or the material. This solves the problem in the prior art where robots are prone to collisions during material grasping, which leads to a decrease in production efficiency.

[0032] Existing solutions only identify gripping points through images of materials. This solution verifies whether the gripping point is within the safe operating range and checks the current level of the material. It then grips the material only if the current level meets the conditions for permissible gripping, thereby avoiding collisions between the robot and guardrails or materials. This solves the problem of robots easily colliding during material gripping, which leads to a decrease in production efficiency in existing technologies.

[0033] In one optional embodiment, determining whether the gripping coordinates are located within a predetermined working area includes: identifying the boundary of the material placement area where the material to be gripped is located, and determining a verification point located on the boundary; determining a first coordinate as the position coordinate of the verification point, controlling the robot to move towards the first coordinate for a preset time, and controlling the robot to stop moving after the preset time is reached; determining the position coordinate of the robot when it stops as a second coordinate; determining that the gripping coordinates are located within the predetermined working area when the first coordinate is equal to the second coordinate; and determining that the gripping coordinates are not located within the predetermined working area when the first coordinate is not equal to the second coordinate.

[0034] In the above embodiments, by setting multiple verification points located on the boundaries of the material placement area and controlling the robot to move sequentially to each verification point and recording the actual arrival position for comparison, it is possible to effectively verify whether the robot's current working area covers the identified material area. When the robot can accurately move to all verification points, it indicates that the currently identified material area is within the robot's working range, thus ensuring the execution of subsequent grasping actions. If the robot cannot accurately reach a certain verification point, it indicates that part of the area is outside its working range, resulting in an non-compliant grasping point. This avoids the robot going out of bounds, colliding, or failing to execute, further preventing collisions that may occur during the robot's grasping of materials.

[0035] Specifically, an image of the current material placement area is captured using a 3D camera, and the four corner points of the area's boundary are identified as verification points, such as the four corners of a rectangular area. The robot's end effector is controlled to move sequentially towards these four verification points, performing the following operations for each verification point: the position coordinates of the verification point are set as the first coordinate; the robot is controlled to move towards the first coordinate; after the robot stops moving, its current position is recorded as the second coordinate; the second coordinate is compared with the preset first coordinate to determine whether it is within the set tolerance range (e.g., position error does not exceed ±2mm). If the robot can reach all four verification points sequentially (i.e., all four first coordinates match the corresponding second coordinates), it indicates that the robot can cover the entire material placement area, and thus the grasping coordinate is determined to be within the predetermined working area; if the robot cannot accurately reach any verification point (i.e., the deviation between the second coordinate and the first coordinate exceeds the tolerance range), it is determined that the currently identified grasping coordinate is not within the workable area.

[0036] In another alternative embodiment, after controlling the robot to grasp the material to be grasped according to the grasping coordinates, the method further includes: acquiring an image of the bottom support and determining the center of gravity coordinates of the bottom support, the bottom support being used to place the material to be grasped; determining the placement height and offset angle of the material to be grasped according to the image, and obtaining the placement coordinates based on the center of gravity coordinates, the placement height being the target height that the robot descends when placing the material to be grasped, the offset angle being the deviation angle between the current orientation of the bottom support and the preset standard orientation; and controlling the robot to place the material to be grasped on the bottom support according to the placement coordinates.

[0037] In the above embodiments, by extracting the actual position and orientation of the base and comparing it with a preset standard template, the robot can perform precise operations based on the real-time calculated placement coordinates, ensuring that the compressor is correctly placed in the center of the bottom support, avoiding offset or tilting. By calculating the offset angle of the bottom support, the robot can automatically adjust the material posture during placement, adapting to the base with random placement orientations. The placement height can be preset according to different material models, effectively preventing collisions caused by materials being placed too high or not contacting the base, further improving placement reliability. Self-correction control from grasping to placement is achieved, further reducing the risk of collisions due to misoperation.

[0038] Specifically, after the robot successfully grasps the material to be grasped, the system takes a picture of the bottom support component using a 2D camera installed in a fixed position, acquires image information, identifies feature points on the base, and calculates the center of gravity coordinates (X, Y) of the base as the reference position for the lateral placement of the material. Next, based on the acquired material image, the system matches the standard placement height corresponding to the material from a template pre-stored in the system, i.e., the target height the robot needs to descend to. Simultaneously, the system compares the current orientation of the base identified in the image with the orientation of the standard template, calculating an offset angle, representing the rotation angle difference between the actual current orientation and the preset orientation of the base. The above data are combined to form a six-dimensional placement coordinate system (X, Y, Zh, EZ, 0, 180), where: X, Y: center of gravity of the base; Zh: placement height; EZ: offset angle around the Z-axis; EX = 180, EY = 0: default placement posture (e.g., the material needs to be placed upright, bottom down). The robot moves its end effector according to the aforementioned placement coordinates, adjusts its posture, and descends to the designated height, accurately placing the compressor onto the base. It then releases its gripper and retracts its manipulator, completing one material placement cycle.

[0039] In some exemplary embodiments, after obtaining the information of the material to be grasped, the method further includes: obtaining the current position of the robot; and if the current position of the robot is not located at the preset initial position, controlling the robot to stop its current action and move to the preset initial position.

[0040] In the above embodiments, if the robot starts the grasping action directly before it is in the initial position, it may cause path planning errors or posture mismatches, resulting in collisions with materials or guardrails. By judging the abnormal position and triggering the reset process, the robot can subsequently execute standardized path planning and grasping and placing actions, improving the accuracy and stability of loading and unloading, avoiding safety hazards caused by trajectory planning errors, thereby improving the safety of the system and further reducing the risk of collisions with materials or guardrails.

[0041] In other exemplary embodiments, each of the aforementioned levels of the material placement area is equipped with the aforementioned sensor. Based on the triggering state of multiple sensors, it is determined whether the level of the material placement area where the material to be grasped is located is consistent with the currently permitted level for grasping, including:

[0042] Determine the height value of the gripping coordinates of the material to be gripped;

[0043] If the height value is greater than the first preset height value and multiple sensors detect the target signal, it is determined that the level where the material to be grabbed is located is consistent with the currently allowed level for grabbing.

[0044] If the height value is less than or equal to the first preset height value and greater than the second preset height value, and the predetermined sensor detects the target signal, it is determined that the level where the material to be grasped is located is consistent with the currently allowed level for grasping, and the first preset height value is greater than or equal to the second preset height value.

[0045] If the height value is less than or equal to the second preset height value, and the sensor at the bottom layer of the material placement area detects the target signal, it is determined that the layer where the material to be grabbed is located is consistent with the currently allowed layer for grabbing.

[0046] In the above embodiments, some materials may have reflective surfaces, resulting in unclear images captured by the camera and inaccurate height recognition. Furthermore, when three layers of material are received, the compressors at the top, middle, and bottom layers are at similar heights. If the robot mistakenly grasps the middle layer when grasping the top layer, or vice versa, it can easily cause material tipping, breakage, and collisions with surrounding materials at the robot's end effector. By using the output Z-axis height value (the actual height coordinate of the material) and the trigger status of the grating sensors at each layer, a dual judgment is made to ensure that the compressor being grasped is indeed the correct layer, thus preventing the robot from mistakenly grasping other layers and causing collisions. When visual misidentification of height leads to inconsistent layer levels, a new image is taken for recognition to avoid incorrect grasping.

[0047] Specifically, for example, the material feeding system is configured to handle the incoming material situation where materials are stacked vertically in three layers. A set of opposite-emitting grating sensors is respectively installed in front of each layer of materials, for detecting whether there is material on this layer. The three layers correspond from top to bottom as: top layer (TOP), middle layer (MID), and bottom layer (BOT). The robot obtains the spatial coordinates of the compressor grasping point through a 3D camera, including the grasping height Z. In order to ensure that the compressor grasped by the robot is indeed at the layer allowed for grasping, the checking of the layer where the compressor is located is realized through the corresponding relationship between the Z height value and the grating signals. The preset height values of the three layers are set as follows: boundary height between top layer and middle layer (first preset height): HTOP=800mm, boundary height between middle layer and bottom layer (second preset height): HMID=550mm, reference height of bottom layer: HBOT=300mm. The PLC inputs corresponding to each group of grating sensors are: DI_TOP: top layer grating signal, DI_MID: middle layer grating signal, DI_BOT: bottom layer grating signal. When the grasping coordinate Z detected by the robot is a certain value, it is judged whether the layer where the grasping coordinate Z is located is the current layer allowed for grasping in combination with the sensor input signals. The specific details are as follows: Case 1: Grasping coordinate Z>HTOP, which indicates that the material is located at the top layer, and at the same time it is detected that: DI_TOP=1, DI_MID=1, DI_BOT=1. Judgment result: the current layer is the top layer, which is consistent with the layer allowed for grasping, and the grasping action can be performed; Case 2: HMID<Z≤HTOP, which indicates that the compressor is located at the middle layer, and the grating status at this time should be: DI_TOP=0, DI_MID=1, DI_BOT=1. Judgment result: the current layer is the middle layer, and if the upper computer allows grasping the middle layer, grasping is performed; Case 3: HBOT<Z≤HMID, which indicates that the compressor is located at the bottom layer, and the grating status at this time should be: DI_TOP=0, DI_MID=0, DI_BOT=1. Judgment result: the current layer is the bottom layer, and if grasping the bottom layer is allowed, execution continues; Case 4: The grasping height Z does not match the grating signal, for example: Z=580mm (which is within the height range of the middle layer), but the grating signals are: DI_TOP=1, DI_MID=1, DI_BOT=1 (which indicates that there is still material on the top layer), which indicates that the lower layer may be misjudged as the middle layer due to visual misidentification. At this time, it is determined that the current height is inconsistent with the sensor status, the grasping execution is rejected, and the 3D camera is controlled to take pictures again to obtain compressor information, and the grasping coordinate is recalculated until the height matches the sensor status.

[0048] In one optional approach, obtaining information about the material to be grasped and determining the grasping coordinates based on the information about the material to be grasped includes: obtaining point cloud data of the material to be grasped, wherein the point cloud data represents the size and position information of the material to be grasped; comparing the point cloud data with a preset standard material template to obtain the grasping coordinates, wherein the standard material template includes the point cloud data obtained from collecting preset standard materials and the corresponding grasping coordinates.

[0049] In the above embodiments, the actual size and spatial location of the material are obtained by acquiring point cloud data, and compared one by one with pre-established standard material templates to determine the optimal grasping posture and spatial coordinates. The standard material template library can cover materials of different models and batches. Flexible matching through template comparison allows the system to adapt to specification differences without recalibration, improving the system's versatility and maintainability. Automatic template comparison for obtaining grasping data reduces the risk of grasping failures or collisions due to incorrect parameter settings, thus improving production line efficiency.

[0050] In some exemplary solutions of this application, comparing the point cloud data with a preset standard material template includes: when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is greater than or equal to a preset value, determining the grasping coordinates corresponding to the point cloud data in the standard material template as the grasping coordinates of the material to be grasped; when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is less than the preset value, outputting an alarm message indicating recognition failure and pausing the current grasping task.

[0051] In the above embodiments, by comparing the real-time acquired point cloud data of the material to be grasped with a preset standard material template and setting a similarity threshold, the grasping coordinates are determined only when there is a high degree of matching, avoiding erroneous grasping due to misidentification or fuzzy recognition. When the similarity is lower than the preset threshold, a recognition failure alarm is output and the current task is paused, helping to avoid system malfunctions due to abnormal material shape or position changes, thereby improving overall operational safety. By introducing an automatic comparison and threshold judgment mechanism, no manual intervention is required, improving the degree of automation.

[0052] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the material online method of this application will be described in detail below with reference to specific embodiments.

[0053] This embodiment relates to a specific method for material loading, specifically, it provides a method for reducing collisions during compressor loading. A schematic diagram of the compressor loading system is shown below. Figure 2The deployment process is based on a 3D vision system. By setting up the robot's working area, adding gratings, and automating the entire compressor deployment process, collisions between the vision system and manual operation are reduced. Taking a three-layer compressor as an example (generally, there are three layers of incoming materials, but at the end of an order, there may be two or one layer), each layer of compressor is equipped with a set of through-beam gratings (a total of three sets of gratings). The grating signals are connected to the PLC, and the interaction between the PLC and the robot's I / O is through the PLC. The grating input signals on the robot end are DI_TOP, DI_MID, and DI_BOT. The robot is a six-axis robot, fixed on the worktable with a ZYX Euler angle type. The 3D camera is installed with the eye outside the hand, and after installation, hand-eye calibration with the robot is required; the 2D camera, after being fixedly installed, requires translation and rotation calibration. The purpose of 3D and 2D camera calibration is to convert camera coordinates to robot coordinates. The base has three protrusions arranged in a triangle for placing the compressor. In the host computer software, the 3D camera has taken pictures to create a compressor template, and the 2D camera has taken pictures to create a base template. like Figure 3 As shown, it includes the following steps:

[0054] Step S1: The 3D camera takes pictures to obtain compressor information (diameter, height and relative position of main and auxiliary tanks), compares it with the compressor information of the production task, and if they match, continues production; otherwise, an alarm is triggered and manual handling is required.

[0055] Step S2: Based on the compressor information (diameter, height, and relative position of the main and auxiliary tanks) acquired by the 3D camera, a vision scheme is matched, and the 2D and 3D schemes are switched. Then, the host computer software interacts with the PLC, and the industrial robot's movements are controlled externally through the PLC to achieve automation and avoid collisions between the robot and the compressor caused by human factors. This example uses an ABB robot; other robot types can refer to the relevant configuration. The robot-side input signals (PLC output signals) DI_Stop, DI_PPToMain, DI_MotorsOn, and DI_Start are respectively bound to the robot stop action, program reset action, motor power-on action, and start action. The PLC sets DI_Stop-DI_PPToMain-DI_MotorsOn-DI_Start to 1 in the following order to control the robot stop, program reset, motor power-on, and start.

[0056] Step S3: The 3D camera matches the acquired compressor point cloud information with the template compressor point cloud information, outputting the compressor grasping coordinates (X, Y, Z, EZ, EY, EX). Before grasping the compressor, the grasping information needs to be verified to eliminate robot collisions caused by errors in the vision system output. The setting of the compressor material working area (XY horizontal plane) can eliminate compressor grasping points not in the working space, avoiding collisions between the robot and the guardrail caused by the vision system. The specific implementation method is as follows: After the compressor material moves into the 3D camera's field of view, the 3D camera identifies the four corners of the material placement area and outputs the XYZ coordinates. The robot sets constants and assigns values ​​to Xmin = min(X1, X2, X3, X4), Xmax = max(X1, X2, X3, Y4), Ymin = min(Y1, Y2, Y3, Y4), Ymax = max(Y1, Y2, Y3, Y4), and Z = max(Z1, Z2, Z3, Z4). Then, the robot moves to the four corners of the material placement area to verify whether the material working area is accurate. If it is inaccurate, it re-identifies and sets the coordinates. After the compressor material working area is set, if the robot receives the coordinates of the compressor to be grasped from the 3D camera, and X is within the range of [Xmin, Xmax] and Y is within the range of [Ymin, Ymax], then it continues; otherwise, the 3D camera takes another picture to obtain the coordinates of the compressor to be grasped.

[0057] The compressor gripping height Z-verification method is as follows: The robot sets and assigns values ​​to the grating mounting heights HTOP, HMID, and HBOT. When Z is greater than HTOP, the compressor to be gripped is on the upper layer, and the grating signals DI_TOP = 1, DI_MID = 1, and DI_BOT = 1. When Z is within the range [HMID, HTOP], the compressor to be gripped is on the middle layer, and the grating signals DI_TOP = 0, DI_MID = 1, and DI_BOT = 1. When Z is within the range [HBOT, HMID], the compressor to be gripped is on the lower layer, and the grating signals DI_TOP = 0, DI_MID = 0, and DI_BOT = 1. Before gripping the compressor, the robot verifies whether the compressor gripping height corresponds to the grating trigger state. If they correspond, the compressor gripping is performed; otherwise, the 3D camera re-captures the compressor gripping point, thereby reducing collisions between the robot and the compressor caused by the vision system. For example, the compressor Z to be grasped by the robot is within the range of [HBOT, HMID], but at this time DI_TOP=1, DI_MID=1, DI_BOT=1 (there is a compressor on the upper layer). At this time, the compressor to be grasped by the robot and the grating trigger state conflict (after grasping, a collision will occur between the robot and the compressor). The 3D camera takes a new picture to obtain the compressor grasping point.

[0058] Step S4: The 2D camera takes a picture to identify the base tray and outputs the placement coordinates (X, Y, Zh, EZ) to the robot. XY represents the coordinates of the center of gravity of the base tray, which has three protrusions arranged in a triangle. Zh is the compressor placement height constructed by the host computer software (related to the compressor model), and EZ is the offset angle of the base tray relative to the template base tray. The robot constructs the compressor placement coordinates (X, Y, Zh, EZ, 0, 180) and places the compressor on the base tray. When the compressor material is placed vertically with its bottom surface parallel to the horizontal plane, EY = 0 and EX = 180 during the robot's compressor gripping and placement process.

[0059] This application also provides a specific embodiment for determining whether the gripping coordinates are located within a predetermined working area, including: analyzing the boundary of the material placement area, such as the edge of a material frame or the flat area of ​​a pallet, selecting a verification point P_check on the boundary, which is coplanar with the gripping coordinates or in its adjacent area. P_check is set as the first coordinate P1 = (X1, Y1, Z1), and the robot is controlled to move towards P1 at a specific speed in the XYZ direction for a duration of T = 1s. After the movement ends, the current position of the robot's end effector P2 = (X2, Y2, Z2) is read. If P1 = P2 (the error is within the allowable threshold), it indicates that P1 is not obstructed or limited, and the gripping point is considered valid; if P1 ≠ P2, it is considered that there is a physical obstacle or recognition deviation, the gripping coordinates are determined to be invalid, the task is paused, and re-identification is performed. The above embodiment sets multiple verification points at the edge of the material placement area and controls the robot to move sequentially to each verification point to record its actual position. The deviation between the verification point coordinates and the robot's actual position can be compared to determine whether the currently identified material area is within the robot's working coverage area. If the robot can accurately reach all verification points, it indicates that its workspace covers all identified material areas, ensuring the smooth execution of subsequent grasping operations. Conversely, if a certain verification point cannot be accurately reached, it means that some identified material areas are outside the robot's effective operating range. In this case, the grasping point can be determined to be non-compliant in a timely manner, preventing the robot from failing to grasp or colliding due to out-of-bounds operation.

[0060] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0061] This application also provides a material loading apparatus. It should be noted that the material loading apparatus of this application can be used to execute the material loading method provided in this application. This apparatus is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0062] The following describes the material loading apparatus provided in the embodiments of this application.

[0063] Figure 4 This is a schematic diagram of a material loading device according to an embodiment of this application. Figure 4 As shown, the device includes:

[0064] The first acquisition unit 10 is used to acquire information about the material to be grasped and to determine the grasping coordinates based on the information about the material to be grasped. The information about the material to be grasped includes the size and position of the material to be grasped. The grasping coordinates represent data on the spatial position and grasping posture that the robot should reach based on the size and position when grasping the material to be grasped.

[0065] Specifically, information about the material to be grasped is obtained by taking pictures with a 3D camera to obtain the size and position of the material to be grasped. According to the pre-set grasping priority and the preset grasping order rules, such as prioritizing the compressor in the X direction, a grasping coordinate for robot grasping is determined and output.

[0066] The determining unit 20 is used to determine whether the grasping coordinates are located within the predetermined working area. If the grasping coordinates are within the predetermined working area, it determines whether the level of the material placement area where the material to be grasped is located is consistent with the currently allowed grasping level based on the triggering status of multiple sensors.

[0067] Specifically, the aforementioned sensors are through-beam grating sensors or photoelectric switches deployed on each layer containing the material. These sensors are used to determine in real time whether material exists on each layer. If the height corresponding to the grasping coordinate matches the layer detected by the sensor, the robot is allowed to perform the grasping operation. For example, if there are three layers of material to be grasped, and three sets of grating sensors are installed: the trigger state of the upper grating is DI_TOP = 1, the trigger state of the middle grating is DI_MID = 1, and the trigger state of the lower grating is DI_BOT = 1. If the Z-coordinate (height coordinate) in the grasping coordinate is within the height range of the "middle layer," and the corresponding grating signals are DI_TOP = 0, DI_MID = 1, and DI_BOT = 1, then the robot is currently grasping the middle layer, and the sensor also indicates that there is indeed material to be grasped in the middle layer. Therefore, the conditions are met, and the robot can grasp the material. Conversely, if the Z coordinate is in the lower layer range, but DI_TOP=1, DI_MID=1, DI_BOT=1 (indicating that there is material to be grabbed in the upper layer), it means that what needs to be grabbed is actually the lower layer material to be grabbed that is blocked by the upper layer, so grabbing is not allowed.

[0068] The first control unit 30 is used to control the robot to grab the material to be grabbed according to the grabbing coordinates when the above-mentioned level is consistent with the currently allowed grabbing level, so as to complete the loading of the material to be grabbed.

[0069] In the above embodiments, the first acquisition unit first acquires information about the material to be grasped, including its size and position, and determines the grasping coordinates based on this information. Next, the determining unit determines whether the grasping coordinates are within a predetermined working area. If the grasping coordinates are within the predetermined working area, the trigger status of multiple sensors determines whether the layer of the material placement area where the material to be grasped is located is consistent with the currently allowed grasping layer. Finally, the first control unit, if the layer is consistent with the currently allowed grasping layer, controls the robot to grasp the material to be grasped according to the grasping coordinates, thus completing the loading of the material. This solution obtains the grasping coordinates by acquiring the size and position of the material to be grasped and determines that the grasping coordinates are within a predetermined working area before grasping, thereby ensuring that the robot can reach the grasping coordinates and avoiding collisions and other abnormal situations caused by the robot failing to reach the grasping coordinates and executing an incorrect path. By determining the trigger status information of multiple sensors, the current level of the material is verified. The material is grasped only when the current level meets the conditions that allow grasping, thereby avoiding collisions between the robot and the guardrail or the material. This solves the problem in the prior art where robots are prone to collisions during material grasping, which leads to a decrease in production efficiency.

[0070] As an optional solution, the determining unit includes: an identification module for identifying the boundary of the material placement area where the material to be grasped is located, and determining a verification point, wherein the verification point is located on the boundary; a first determining module for determining a first coordinate as the position coordinate of the verification point, controlling the robot to move towards the first coordinate for a preset time, and controlling the robot to stop moving after the preset time is reached; a second determining module for determining the position coordinate of the robot when it stops as a second coordinate; a third determining module for determining that the grasping coordinate is located within the predetermined working area when the first coordinate is equal to the second coordinate; and a fourth determining module for determining that the grasping coordinate is not located within the predetermined working area when the first coordinate is not equal to the second coordinate.

[0071] In the above embodiments, by setting multiple verification points located on the boundaries of the material placement area and controlling the robot to move sequentially to each verification point and recording the actual arrival position for comparison, it is possible to effectively verify whether the robot's current working area covers the identified material area. When the robot can accurately move to all verification points, it indicates that the currently identified material area is within the robot's working range, thus ensuring the execution of subsequent grasping actions. If the robot cannot accurately reach a certain verification point, it indicates that part of the area is outside its working range, resulting in an non-compliant grasping point. This avoids the robot going out of bounds, colliding, or failing to execute, further preventing collisions that may occur during the robot's grasping of materials.

[0072] Specifically, an image of the current material placement area is captured using a 3D camera, and the four corner points of the area's boundary are identified as verification points, such as the four corners of a rectangular area. The robot's end effector is controlled to move sequentially towards these four verification points, performing the following operations for each verification point: the position coordinates of the verification point are set as the first coordinate; the robot is controlled to move towards the first coordinate; after the robot stops moving, its current position is recorded as the second coordinate; the second coordinate is compared with the preset first coordinate to determine whether it is within the set tolerance range (e.g., position error does not exceed ±2mm). If the robot can reach all four verification points sequentially (i.e., all four first coordinates match the corresponding second coordinates), it indicates that the robot can cover the entire material placement area, and thus the grasping coordinate is determined to be within the predetermined working area; if the robot cannot accurately reach any verification point (i.e., the deviation between the second coordinate and the first coordinate exceeds the tolerance range), it is determined that the currently identified grasping coordinate is not within the workable area.

[0073] In one optional embodiment, the first control unit includes: a first acquisition module, configured to acquire an image of a bottom support member and determine the center-of-gravity coordinates of the bottom support member, wherein the bottom support member is used to place the material to be grasped; a fifth determination module, configured to determine the placement height and offset angle of the material to be grasped based on the image, and obtain placement coordinates based on the center-of-gravity coordinates, wherein the placement height is the target height that the robot descends when placing the material to be grasped, and the offset angle is the deviation angle between the current orientation of the bottom support member and a preset standard orientation; and a control module, configured to control the robot to place the material to be grasped onto the bottom support member according to the placement coordinates.

[0074] In the above embodiments, by extracting the actual position and orientation of the base and comparing it with a preset standard template, the robot can perform precise operations based on the real-time calculated placement coordinates, ensuring that the compressor is correctly placed in the center of the bottom support, avoiding offset or tilting. By calculating the offset angle of the bottom support, the robot can automatically adjust the material posture during placement, adapting to the base with random placement orientations. The placement height can be preset according to different material models, effectively preventing collisions caused by materials being placed too high or not contacting the base, further improving placement reliability. Self-correction control from grasping to placement is achieved, further reducing the risk of collisions due to misoperation.

[0075] Specifically, after the robot successfully grasps the material to be grasped, the system takes a picture of the bottom support component using a 2D camera installed in a fixed position, acquires image information, identifies feature points on the base, and calculates the center of gravity coordinates (X, Y) of the base as the reference position for the lateral placement of the material. Next, based on the acquired material image, the system matches the standard placement height corresponding to the material from a template pre-stored in the system, i.e., the target height the robot needs to descend to. Simultaneously, the system compares the current orientation of the base identified in the image with the orientation of the standard template, calculating an offset angle, representing the rotation angle difference between the actual current orientation and the preset orientation of the base. The above data are combined to form a six-dimensional placement coordinate system (X, Y, Zh, EZ, 0, 180), where: X, Y: center of gravity of the base; Zh: placement height; EZ: offset angle around the Z-axis; EX = 180, EY = 0: default placement posture (e.g., the material needs to be placed upright, bottom down). The robot moves its end effector according to the aforementioned placement coordinates, adjusts its posture, and descends to the designated height, accurately placing the compressor onto the base. It then releases its gripper and retracts its manipulator, completing one material placement cycle.

[0076] In some exemplary embodiments, the apparatus further includes: a second acquisition unit for acquiring the current position of the robot; and a second control unit for controlling the robot to stop its current action and move to the preset initial position when the current position of the robot is not located at the preset initial position.

[0077] In the above embodiments, if the robot starts the grasping action directly before it is in the initial position, it may cause path planning errors or posture mismatches, resulting in collisions with materials or guardrails. By judging the abnormal position and triggering the reset process, the robot can subsequently execute standardized path planning and grasping and placing actions, improving the accuracy and stability of loading and unloading, avoiding safety hazards caused by trajectory planning errors, thereby improving the safety of the system and further reducing the risk of collisions with materials or guardrails.

[0078] In some other exemplary embodiments, the determining unit includes: a sixth determining module, configured to determine the height value of the grasping coordinates of the material to be grasped; a seventh determining module, configured to determine that the layer where the material to be grasped is located is consistent with the currently allowed grasping layer when the height value is greater than a first preset height value and multiple sensors detect the target signal; an eighth determining module, configured to determine that the layer where the material to be grasped is located is consistent with the currently allowed grasping layer when the height value is less than or equal to the first preset height value and greater than a second preset height value, and a predetermined sensor detects the target signal, wherein the first preset height value is greater than or equal to the second preset height value; and a ninth determining module, configured to determine that the layer where the material to be grasped is located is consistent with the currently allowed grasping layer when the height value is less than or equal to the second preset height value and the sensor at the bottom layer of the material placement area detects the target signal.

[0079] In the above embodiments, some materials may have reflective surfaces, resulting in unclear images captured by the camera and inaccurate height recognition. Furthermore, when three layers of material are received, the compressors at the top, middle, and bottom layers are at similar heights. If the robot mistakenly grasps the middle layer when grasping the top layer, or vice versa, it can easily cause material tipping, breakage, and collisions with surrounding materials at the robot's end effector. By using the output Z-axis height value (the actual height coordinate of the material) and the trigger status of the grating sensors at each layer, a dual judgment is made to ensure that the compressor being grasped is indeed the correct layer, thus preventing the robot from mistakenly grasping other layers and causing collisions. When visual misidentification of height leads to inconsistent layer levels, a new image is taken for recognition to avoid incorrect grasping.

[0080] Specifically, for example, the material feeding system is configured to handle the incoming material condition where materials are stacked vertically in three layers. A set of opposite-emitting grating sensors are respectively installed in front of each layer of materials, configured to detect whether there is material on the layer. The three layers from top to bottom are respectively defined as: top layer (TOP), middle layer (MID), and bottom layer (BOT). The robot obtains the spatial coordinates of the grasping point of the compressor through a 3D camera, including the grasping height Z. To ensure that the compressor grasped by the robot is indeed at a layer that allows grasping, the verification of the layer where the compressor is located is implemented through the corresponding relationship between the Z height value and the grating signal. The preset height values of the three layers are set as follows: boundary height between the top layer and the middle layer (first preset height): HTOP=800mm, boundary height between the middle layer and the bottom layer (second preset height): HMID=550mm, reference height of the bottom layer: HBOT=300mm. The PLC inputs corresponding to each set of grating sensors are: DI_TOP: top layer grating signal, DI_MID: middle layer grating signal, DI_BOT: bottom layer grating signal. When the grasping coordinate Z detected by the robot is a certain value, it is combined with the sensor input signal to determine whether the layer thereof is the current layer that allows grasping. The specific process is as follows: Case 1: Grasping coordinate Z>HTOP, indicating that the material is on the top layer. At the same time, it is detected that: DI_TOP=1, DI_MID=1, DI_BOT=1. Judgment result: the current layer is the top layer and is consistent with the layer that allows grasping, so the grasping action can be executed; Case 2: HMID<Z≤HTOP, indicating that the compressor is on the middle layer. At this time, the grating status should be: DI_TOP=0, DI_MID=1, DI_BOT=1. Judgment result: the current layer is the middle layer, if the upper computer allows grasping the middle layer, then execute grasping; Case 3: HBOT<Z≤HMID, indicating that the compressor is on the bottom layer. At this time, the grating status should be: DI_TOP=0, DI_MID=0, DI_BOT=1. Judgment result: the current layer is the bottom layer, if the bottom layer is allowed to be grasped, then continue to execute; Case 4: The grasping height Z does not match the grating signal. For example, Z=580mm (which is within the height range of the middle layer), but the grating signal is: DI_TOP=1, DI_MID=1, DI_BOT=1 (indicating that there is still material on the top layer), which indicates that visual misidentification may have caused misjudgment that the bottom layer is the middle layer. At this time, it is determined that the current height does not match the sensor status, grasping execution is rejected, and the 3D camera is controlled to take pictures again to obtain compressor information and recalculate the grasping coordinates until the height matches the sensor status.

[0081] In one optional embodiment, the first acquisition unit further includes: a second acquisition module, used to acquire point cloud data of the material to be grasped, wherein the point cloud data represents the size and position information of the material to be grasped; and a comparison module, used to compare the point cloud data with a preset standard material template to obtain the grasping coordinates, wherein the standard material template includes the point cloud data obtained by collecting preset standard materials and the corresponding grasping coordinates.

[0082] In the above embodiments, the actual size and spatial location of the material are obtained by acquiring point cloud data, and compared one by one with pre-established standard material templates to determine the optimal grasping posture and spatial coordinates. The standard material template library can cover materials of different models and batches. Flexible matching through template comparison allows the system to adapt to specification differences without recalibration, improving the system's versatility and maintainability. Automatic template comparison for obtaining grasping data reduces the risk of grasping failures or collisions due to incorrect parameter settings, thus improving production line efficiency.

[0083] In some exemplary solutions of this application, the comparison module includes: a determination submodule, configured to determine the grasping coordinates corresponding to the point cloud data in the standard material template as the grasping coordinates of the material to be grasped when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is greater than or equal to a preset value; and an output submodule, configured to output an alarm message indicating recognition failure and suspend the current grasping task when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is less than the preset value.

[0084] In the above embodiments, by comparing the real-time acquired point cloud data of the material to be grasped with a preset standard material template and setting a similarity threshold, the grasping coordinates are determined only when there is a high degree of matching, avoiding erroneous grasping due to misidentification or fuzzy recognition. When the similarity is lower than the preset threshold, a recognition failure alarm is output and the current task is paused, helping to avoid system malfunctions due to abnormal material shape or position changes, thereby improving overall operational safety. By introducing an automatic comparison and threshold judgment mechanism, no manual intervention is required, improving the degree of automation.

[0085] The aforementioned material loading device includes a processor and a memory. The first acquisition unit, the determining unit, and the first control unit are all stored as program units in the memory, and the processor executes these program units to achieve their respective functions. All of the aforementioned modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0086] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and adjusting kernel parameters can at least address the problem in existing technologies where robots are prone to collisions during material handling, leading to decreased production efficiency.

[0087] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0088] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the method of putting materials online.

[0089] Specifically, the methods for putting materials online include:

[0090] Step S101: Obtain information about the material to be grasped, and determine the grasping coordinates based on the information about the material to be grasped. The information about the material to be grasped includes the size and position of the material to be grasped. The grasping coordinates represent the data of the spatial position and grasping posture that the robot should reach when grasping the material to be grasped based on the size and position.

[0091] Specifically, information about the material to be grasped is obtained by taking pictures with a 3D camera to obtain the size and position of the material to be grasped. According to the pre-set grasping priority and the preset grasping order rules, such as prioritizing the compressor in the X direction, a grasping coordinate for robot grasping is determined and output.

[0092] Step S102: Determine whether the above-mentioned grabbing coordinates are located within the predetermined working area. If the above-mentioned grabbing coordinates are within the predetermined working area, determine whether the level of the material placement area where the material to be grabbed is located is consistent with the currently allowed grabbing level based on the triggering status of multiple sensors.

[0093] Specifically, the aforementioned sensors are through-beam grating sensors or photoelectric switches deployed on each layer containing the material. These sensors are used to determine in real time whether material exists on each layer. If the height corresponding to the grasping coordinate matches the layer detected by the sensor, the robot is allowed to perform the grasping operation. For example, if there are three layers of material to be grasped, and three sets of grating sensors are installed: the trigger state of the upper grating is DI_TOP = 1, the trigger state of the middle grating is DI_MID = 1, and the trigger state of the lower grating is DI_BOT = 1. If the Z-coordinate (height coordinate) in the grasping coordinate is within the height range of the "middle layer," and the corresponding grating signals are DI_TOP = 0, DI_MID = 1, and DI_BOT = 1, then the robot is currently grasping the middle layer, and the sensor also indicates that there is indeed material to be grasped in the middle layer. Therefore, the conditions are met, and the robot can grasp the material. Conversely, if the Z coordinate is in the lower layer range, but DI_TOP=1, DI_MID=1, DI_BOT=1 (indicating that there is material to be grabbed in the upper layer), it means that what needs to be grabbed is actually the lower layer material to be grabbed that is blocked by the upper layer, so grabbing is not allowed.

[0094] Step S103: If the above-mentioned level is consistent with the currently allowed level for grasping, control the robot to grasp the material to be grasped according to the grasping coordinates, so as to complete the online loading of the material to be grasped.

[0095] In one embodiment of this application, determining whether the grasping coordinates are located within a predetermined working area includes: identifying the boundary of the material placement area where the material to be grasped is located, and determining a verification point located on the boundary; determining a first coordinate as the position coordinate of the verification point, controlling the robot to move towards the first coordinate for a preset time, and controlling the robot to stop moving after the preset time is reached; determining the position coordinate of the robot when it stops as a second coordinate; determining that the grasping coordinates are located within the predetermined working area when the first coordinate is equal to the second coordinate; and determining that the grasping coordinates are not located within the predetermined working area when the first coordinate is not equal to the second coordinate.

[0096] In one embodiment of this application, after controlling the robot to grasp the material to be grasped according to the grasping coordinates, the method further includes: acquiring an image of a bottom support and determining the center of gravity coordinates of the bottom support, wherein the bottom support is used to place the material to be grasped; determining the placement height and offset angle of the material to be grasped according to the image, and obtaining placement coordinates based on the center of gravity coordinates, wherein the placement height is the target height that the robot descends when placing the material to be grasped, and the offset angle is the deviation angle between the current orientation of the bottom support and the preset standard orientation; and controlling the robot to place the material to be grasped on the bottom support according to the placement coordinates.

[0097] In one embodiment of this application, after obtaining the information of the material to be grasped, the method further includes: obtaining the current position of the robot; if the current position of the robot is not located at the preset initial position, controlling the robot to stop the current action and move to the preset initial position.

[0098] In one embodiment of this application, each level of the material placement area is equipped with the aforementioned sensor. Based on the triggering states of multiple sensors, determining whether the level of the material placement area containing the material to be grasped is consistent with the currently allowed grasping level includes: determining the height value of the grasping coordinates of the material to be grasped; if the height value is greater than a first preset height value and multiple sensors detect a target signal, determining that the level containing the material to be grasped is consistent with the currently allowed grasping level; if the height value is less than or equal to the first preset height value and greater than a second preset height value, and a predetermined sensor detects the target signal, determining that the level containing the material to be grasped is consistent with the currently allowed grasping level, wherein the first preset height value is greater than or equal to the second preset height value; and if the height value is less than or equal to the second preset height value, and the sensor at the bottom layer of the material placement area detects the target signal, determining that the level containing the material to be grasped is consistent with the currently allowed grasping level.

[0099] In one embodiment of this application, obtaining information about the material to be grasped and determining the grasping coordinates based on the information about the material to be grasped includes: obtaining point cloud data of the material to be grasped, wherein the point cloud data represents the size and position information of the material to be grasped; comparing the point cloud data with a preset standard material template to obtain the grasping coordinates, wherein the standard material template includes the point cloud data obtained by collecting preset standard materials and the corresponding grasping coordinates.

[0100] In one embodiment of this application, comparing the point cloud data with a preset standard material template includes: when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is greater than or equal to a preset value, determining the grasping coordinates corresponding to the point cloud data in the standard material template as the grasping coordinates of the material to be grasped; when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is less than the preset value, outputting an alarm message indicating recognition failure and pausing the current grasping task.

[0101] This invention provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps: Step S101, acquiring information about the material to be grasped, and determining grasping coordinates based on the information about the material to be grasped. The information about the material to be grasped includes the size and position of the material to be grasped, and the grasping coordinates represent data on the spatial position and grasping posture that the robot should reach based on the size and position when grasping the material to be grasped.

[0102] Specifically, information about the material to be grasped is obtained by taking pictures with a 3D camera to obtain the size and position of the material to be grasped. According to the pre-set grasping priority and the preset grasping order rules, such as prioritizing the compressor in the X direction, a grasping coordinate for robot grasping is determined and output.

[0103] Step S102: Determine whether the above-mentioned grabbing coordinates are located within the predetermined working area. If the above-mentioned grabbing coordinates are within the predetermined working area, determine whether the level of the material placement area where the material to be grabbed is located is consistent with the currently allowed grabbing level based on the triggering status of multiple sensors.

[0104] Specifically, the aforementioned sensors are through-beam grating sensors or photoelectric switches deployed on each layer containing the material. These sensors are used to determine in real time whether material exists on each layer. If the height corresponding to the grasping coordinate matches the layer detected by the sensor, the robot is allowed to perform the grasping operation. For example, if there are three layers of material to be grasped, and three sets of grating sensors are installed: the trigger state of the upper grating is DI_TOP = 1, the trigger state of the middle grating is DI_MID = 1, and the trigger state of the lower grating is DI_BOT = 1. If the Z-coordinate (height coordinate) in the grasping coordinate is within the height range of the "middle layer," and the corresponding grating signals are DI_TOP = 0, DI_MID = 1, and DI_BOT = 1, then the robot is currently grasping the middle layer, and the sensor also indicates that there is indeed material to be grasped in the middle layer. Therefore, the conditions are met, and the robot can grasp the material. Conversely, if the Z coordinate is in the lower layer range, but DI_TOP=1, DI_MID=1, DI_BOT=1 (indicating that there is material to be grabbed in the upper layer), it means that what needs to be grabbed is actually the lower layer material to be grabbed that is blocked by the upper layer, so grabbing is not allowed.

[0105] Step S103: If the above-mentioned level is consistent with the currently allowed level for grasping, control the robot to grasp the material to be grasped according to the grasping coordinates, so as to complete the online loading of the material to be grasped.

[0106] In one embodiment of this application, determining whether the grasping coordinates are located within a predetermined working area includes: identifying the boundary of the material placement area where the material to be grasped is located, and determining a verification point located on the boundary; determining a first coordinate as the position coordinate of the verification point, controlling the robot to move towards the first coordinate for a preset time, and controlling the robot to stop moving after the preset time is reached; determining the position coordinate of the robot when it stops as a second coordinate; determining that the grasping coordinates are located within the predetermined working area when the first coordinate is equal to the second coordinate; and determining that the grasping coordinates are not located within the predetermined working area when the first coordinate is not equal to the second coordinate.

[0107] In one embodiment of this application, after controlling the robot to grasp the material to be grasped according to the grasping coordinates, the method further includes: acquiring an image of a bottom support and determining the center of gravity coordinates of the bottom support, wherein the bottom support is used to place the material to be grasped; determining the placement height and offset angle of the material to be grasped according to the image, and obtaining placement coordinates based on the center of gravity coordinates, wherein the placement height is the target height that the robot descends when placing the material to be grasped, and the offset angle is the deviation angle between the current orientation of the bottom support and the preset standard orientation; and controlling the robot to place the material to be grasped on the bottom support according to the placement coordinates.

[0108] In one embodiment of this application, after obtaining the information of the material to be grasped, the method further includes: obtaining the current position of the robot; if the current position of the robot is not located at the preset initial position, controlling the robot to stop the current action and move to the preset initial position.

[0109] In one embodiment of this application, each level of the material placement area is equipped with the aforementioned sensor. Based on the triggering states of multiple sensors, determining whether the level of the material placement area containing the material to be grasped is consistent with the currently allowed grasping level includes: determining the height value of the grasping coordinates of the material to be grasped; if the height value is greater than a first preset height value and multiple sensors detect a target signal, determining that the level containing the material to be grasped is consistent with the currently allowed grasping level; if the height value is less than or equal to the first preset height value and greater than a second preset height value, and a predetermined sensor detects the target signal, determining that the level containing the material to be grasped is consistent with the currently allowed grasping level, wherein the first preset height value is greater than or equal to the second preset height value; and if the height value is less than or equal to the second preset height value, and the sensor at the bottom layer of the material placement area detects the target signal, determining that the level containing the material to be grasped is consistent with the currently allowed grasping level.

[0110] In one embodiment of this application, obtaining information about the material to be grasped and determining the grasping coordinates based on the information about the material to be grasped includes: obtaining point cloud data of the material to be grasped, wherein the point cloud data represents the size and position information of the material to be grasped; comparing the point cloud data with a preset standard material template to obtain the grasping coordinates, wherein the standard material template includes the point cloud data obtained by collecting preset standard materials and the corresponding grasping coordinates.

[0111] In one embodiment of this application, comparing the point cloud data with a preset standard material template includes: when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is greater than or equal to a preset value, determining the grasping coordinates corresponding to the point cloud data in the standard material template as the grasping coordinates of the material to be grasped; when the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is less than the preset value, outputting an alarm message indicating recognition failure and pausing the current grasping task.

[0112] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0113] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0119] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0120] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0123] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0124] 1) The material loading method of this application first obtains the material information to be grabbed, including the size and position of the material, and determines the grabbing coordinates based on the material information; then, it determines whether the grabbing coordinates are located within a predetermined working area. If the grabbing coordinates are within the predetermined working area, it determines whether the layer of the material placement area where the material to be grabbed is located is consistent with the currently allowed grabbing layer based on the trigger status of multiple sensors; finally, if the layer is consistent with the currently allowed grabbing layer, it controls the robot to grab the material to be grabbed according to the grabbing coordinates to complete the loading of the material. This solution obtains the grabbing coordinates by acquiring the size and position of the material to be grabbed, and determines that the grabbing coordinates are located within the predetermined working area before grabbing, thereby ensuring that the robot can reach the grabbing coordinates and avoiding collisions and other abnormal situations caused by the robot failing to reach the grabbing coordinates and executing the wrong path. By determining the trigger status information of multiple sensors, the current level of the material is verified. The material is grasped only when the current level meets the conditions that allow grasping, thereby avoiding collisions between the robot and the guardrail or the material. This solves the problem in the prior art where robots are prone to collisions during material grasping, which leads to a decrease in production efficiency.

[0125] 2) The material loading device of this application first acquires material information, including the size and position of the material to be grabbed, through a first acquisition unit, and determines the grabbing coordinates based on the material information. Next, a determination unit determines whether the grabbing coordinates are within a predetermined working area. If the grabbing coordinates are within the predetermined working area, the device determines whether the layer of the material placement area where the material to be grabbed is located is consistent with the currently allowed grabbing layer, based on the triggering status of multiple sensors. Finally, a first control unit, if the layer is consistent with the currently allowed grabbing layer, controls the robot to grab the material according to the grabbing coordinates to complete the loading of the material. This solution obtains the grabbing coordinates by acquiring the size and position of the material to be grabbed, and determines that the grabbing coordinates are within a predetermined working area before grabbing, thereby ensuring that the robot can reach the grabbing coordinates and avoiding collisions and other abnormal situations caused by the robot failing to reach the grabbing coordinates and executing an incorrect path. By determining the trigger status information of multiple sensors, the current level of the material is verified. The material is grasped only when the current level meets the conditions that allow grasping, thereby avoiding collisions between the robot and the guardrail or the material. This solves the problem in the prior art where robots are prone to collisions during material grasping, which leads to a decrease in production efficiency.

[0126] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for loading materials online, characterized in that, include: The robot acquires information about the material to be grasped and determines the grasping coordinates based on the information about the material to be grasped. The information about the material to be grasped includes the size and position of the material to be grasped. The grasping coordinates represent data on the spatial position and grasping posture that the robot should reach when grasping the material to be grasped, based on the size and position. Determine whether the grasping coordinates are within the predetermined working area. If the grasping coordinates are within the predetermined working area, determine whether the level of the material placement area where the material to be grasped is located is consistent with the currently allowed grasping level based on the triggering status of multiple sensors. If the level is consistent with the currently allowed level for grasping, the robot is controlled to grasp the material to be grasped according to the grasping coordinates, so as to complete the loading of the material to be grasped. Determining whether the grasping coordinates are within a predetermined working area includes: identifying the boundary of the material placement area where the material to be grasped is located, and determining a verification point, wherein the verification point is located on the boundary; The first coordinate is determined as the position coordinate of the verification point, and the robot is controlled to move towards the first coordinate for a preset time, and the robot stops moving after the preset time is reached; the position coordinate when the robot stops is determined as the second coordinate; if the first coordinate is equal to the second coordinate, the grasping coordinate is determined to be located within the predetermined working area; if the first coordinate is not equal to the second coordinate, the grasping coordinate is determined to be not located within the predetermined working area.

2. The method according to claim 1, characterized in that, After controlling the robot to grasp the material to be grasped according to the grasping coordinates, the method further includes: Acquire an image of the bottom support component and determine the coordinates of the center of gravity of the bottom support component, which is used to place the material to be grasped; The placement height and offset angle of the material to be grasped are determined based on the image, and the placement coordinates are obtained based on the center of gravity coordinates. The placement height is the target height that the robot descends when placing the material to be grasped, and the offset angle is the deviation angle between the current orientation of the bottom support and the preset standard orientation. Based on the placement coordinates, the robot is controlled to place the material to be grasped onto the bottom support.

3. The method according to claim 1, characterized in that, After obtaining the information of the material to be grabbed, the method further includes: Obtain the current position of the robot; If the robot's current position is not at the preset initial position, control the robot to stop its current action and move to the preset initial position.

4. The method according to claim 1, characterized in that, Each level of the material placement area is equipped with a sensor. Based on the triggering status of multiple sensors, it is determined whether the level of the material placement area where the material to be grasped is located is consistent with the currently allowed level for grasping, including: Determine the height value of the gripping coordinates of the material to be gripped; If the height value is greater than the first preset height value and multiple sensors detect the target signal, it is determined that the layer where the material to be grabbed is located is consistent with the currently allowed layer for grabbing. If the height value is less than or equal to the first preset height value and greater than the second preset height value, and the predetermined sensor detects the target signal, it is determined that the layer where the material to be grasped is located is consistent with the currently allowed layer for grasping, and the first preset height value is greater than or equal to the second preset height value; If the height value is less than or equal to the second preset height value, and the sensor at the bottom layer of the material placement area detects the target signal, it is determined that the layer where the material to be grabbed is located is consistent with the currently allowed layer for grabbing.

5. The method according to claim 1, characterized in that, Obtaining information about the material to be grabbed, and determining the grabbing coordinates based on the information about the material to be grabbed, includes: Obtain point cloud data of the material to be grasped, wherein the point cloud data represents information about the size and position of the material to be grasped; The point cloud data is compared with a preset standard material template to obtain the grab coordinates. The standard material template includes the point cloud data obtained by collecting preset standard materials and the corresponding grab coordinates.

6. The method according to claim 5, characterized in that, The point cloud data is compared with a preset standard material template, including: When the similarity between the point cloud data of the material to be grasped and the point cloud data in the standard material template is greater than or equal to a preset value, the grasping coordinates corresponding to the point cloud data in the standard material template are determined as the grasping coordinates of the material to be grasped. When the similarity between the point cloud data of the material to be captured and the point cloud data in the standard material template is less than the preset value, an alarm message indicating recognition failure is output and the current capturing task is paused.

7. A device for feeding materials onto a production line, characterized in that, include: The acquisition unit is used to acquire information about the material to be grasped and determine the grasping coordinates based on the information about the material to be grasped. The information about the material to be grasped includes the size and position of the material to be grasped. The grasping coordinates represent data on the spatial position and grasping posture that the robot should reach when grasping the material to be grasped, based on the size and position. A determining unit is used to determine whether the grasping coordinates are located within a predetermined working area. If the grasping coordinates are located within the predetermined working area, the unit determines whether the level of the material placement area where the material to be grasped is located is consistent with the currently allowed grasping level based on the triggering status of multiple sensors. The control unit is configured to control the robot to grab the material to be grabbed according to the grabbing coordinates when the level is consistent with the currently allowed grabbing level, so as to complete the loading of the material to be grabbed. The determining unit includes: an identification module, used to identify the boundary of the material placement area where the material to be grasped is located, and to determine a verification point, wherein the verification point is located on the boundary; a first determining module, used to determine a first coordinate as the position coordinate of the verification point, and to control the robot to move toward the first coordinate for a preset time, and to control the robot to stop moving after the preset time is reached; a second determining module, used to determine the position coordinate of the robot when it stops as a second coordinate; a third determining module, used to determine that the grasping coordinate is located within the predetermined working area when the first coordinate is equal to the second coordinate; and a fourth determining module, used to determine that the grasping coordinate is not located within the predetermined working area when the first coordinate is not equal to the second coordinate.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 6.

Images