Vehicle door blanking processing method and device based on material frame positioning and computer equipment

Abstract

The invention relates to a vehicle door blanking processing method, device and equipment based on material frame positioning. The method comprises the steps of converting initial material frame point cloud data of a material frame in a current scene into material frame point cloud data under a robot coordinate system, and performing plane fitting processing on the material frame point cloud data to obtain a plurality of current card slot feature points in the current scene; and on the basis of the plurality of current card slot feature points and a plurality of template card slot feature points of the template material frame, feature point comparison is carried out to obtain a plurality of homonymy feature point combinations of which the feature point distances are smaller than a preset distance threshold. Respectively constructing a scene coordinate matrix and a template coordinate matrix based on the plurality of homonymy feature point combinations, and performing matrix operation on the scene coordinate matrix and the template coordinate matrix to obtain an offset matrix; and material frame pose difference data are generated according to the offset matrix, the material frame pose difference data are fed back to the robot, and the robot is indicated to conduct vehicle door discharging treatment according to the material frame pose difference data. By means of the method, the vehicle door discharging treatment efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of automation technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for handling door unloading based on material frame positioning. Background Technology

[0002] With the development of automation technology, a method has emerged in automobile production where six-axis robots are used to automatically transport car doors to material frame slots for automatic door unloading. However, during the automatic unloading process, the actual position and orientation of the material frame relative to the robot's base coordinate system are uncertain and subject to change. If the robot operates according to a preset theoretical coordinate system, violent collisions between the robot, the material, and the material frame can easily occur, leading to resource loss and equipment damage.

[0003] Traditional technologies typically employ two-dimensional vision-based recognition methods, using image processing techniques such as edge detection or template matching to perform edge recognition and position determination on the acquired material frame image, thereby obtaining the actual position of the material frame. Alternatively, deep learning object detection technology can be used to perform object detection on the acquired material frame image to obtain the actual position of the material frame. As a result, the robot can perform actions based on the actual position of the material frame, moving the transport door to the material frame slot to achieve automatic unloading.

[0004] However, traditional methods for identifying the position of the material frame are easily affected by different lighting conditions and rely heavily on identifying single features on the material frame, such as its edge or shape. In practical applications, the surface of the material frame may be obscured by parts due to wear, oil stains, or different lighting conditions, leading to the loss of key feature points, which can easily result in positioning failure, inaccurate identification of the material frame position, errors in the unloading process, and the need for repeated operations. Summary of the Invention

[0005] Therefore, it is necessary to provide a door unloading method, apparatus, computer equipment, computer-readable storage medium, and computer program product based on material frame positioning that can improve the efficiency of door unloading processing, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a door unloading method based on material frame positioning, comprising: acquiring initial material frame point cloud data of the current scene material frame; converting the initial material frame point cloud data into material frame point cloud data in the robot coordinate system; performing plane fitting processing on the material frame point cloud data to obtain multiple current slot feature points in the current scene; acquiring multiple template slot feature points corresponding to a pre-configured template material frame; comparing feature points based on the multiple template slot feature points and the multiple current slot feature points to obtain multiple combinations of identically named feature points whose feature point distance is less than a preset distance threshold; constructing a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame based on the multiple combinations of identically named feature points; performing matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain an offset matrix between the current scene material frame and the template material frame; generating material frame pose difference data based on the offset matrix; and feeding back the material frame pose difference data to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0007] Secondly, this application also provides a door unloading processing device based on material frame positioning, comprising: a plane fitting processing module, used to collect initial material frame point cloud data of the current scene material frame, convert the initial material frame point cloud data into material frame point cloud data in the robot coordinate system, and perform plane fitting processing on the material frame point cloud data to obtain multiple current slot feature points in the current scene; and a feature point comparison module, used to obtain multiple template slot feature points corresponding to a pre-configured template material frame, and perform feature point comparison based on the multiple template slot feature points and the multiple current slot feature points to obtain feature point distances less than a preset distance threshold. The system includes a combination of multiple identically named feature points; a coordinate matrix construction module, used to construct a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame based on the combination of multiple identically named feature points; and a material frame pose difference data generation module, used to perform matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene material frame and the template material frame, and generate material frame pose difference data based on the offset matrix, and feed the material frame pose difference data back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0008] Thirdly, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring initial point cloud data of the current scene's material frame; converting the initial point cloud data of the material frame into point cloud data of the material frame in the robot coordinate system; and performing plane fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene; acquiring multiple template slot feature points corresponding to a pre-configured template material frame; and performing feature processing based on the multiple template slot feature points and the multiple current slot feature points. Point comparison is performed to obtain multiple combinations of identical feature points whose distance to each other is less than a preset distance threshold. Based on these multiple combinations of identical feature points, a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame are constructed. Matrix operations are performed on the scene coordinate matrix and the template coordinate matrix to obtain an offset matrix between the current scene material frame and the template material frame. Material frame pose difference data is generated based on the offset matrix and fed back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0009] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it performs the following steps: acquiring initial point cloud data of the current scene's material frame; converting the initial point cloud data of the material frame into point cloud data of the material frame in the robot coordinate system; and performing planar fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene; acquiring multiple template slot feature points corresponding to a pre-configured template material frame; and comparing the feature points based on the multiple template slot feature points and the multiple current slot feature points to obtain... A combination of multiple identically named feature points whose distance to each other is less than a preset distance threshold is obtained. Based on the multiple combinations of identically named feature points, a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame are constructed respectively. Matrix operations are performed on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene material frame and the template material frame. Material frame pose difference data is generated based on the offset matrix. The material frame pose difference data is fed back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0010] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps: acquiring initial point cloud data of a current scene material frame; converting the initial point cloud data of the material frame into point cloud data of the material frame in a robot coordinate system; performing plane fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene; acquiring multiple template slot feature points corresponding to a pre-configured template material frame; comparing feature points based on the multiple template slot feature points and the multiple current slot feature points to obtain multiple combinations of feature points with the same name whose feature point distance is less than a preset distance threshold; constructing a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame based on the multiple combinations of feature points with the same name; performing matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain an offset matrix between the current scene material frame and the template material frame; generating material frame pose difference data based on the offset matrix; and feeding back the material frame pose difference data to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0011] In the aforementioned door unloading processing method, device, computer equipment, computer-readable storage medium, and computer program product based on material frame positioning, the initial material frame point cloud data of the current scene is collected, the initial material frame point cloud data is converted into material frame point cloud data in the robot coordinate system, the material frame point cloud data is subjected to plane fitting processing to obtain multiple current slot feature points in the current scene, and multiple template slot feature points corresponding to the pre-configured template material frame are obtained. Thus, based on the multiple template slot feature points and the multiple current slot feature points, feature point comparison can be performed to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold. This allows for accurate determination of the positional change of the current scene material frame relative to the template material frame, thereby achieving accurate positioning of the current scene material frame. Furthermore, based on multiple combinations of identical feature points, a scene coordinate matrix corresponding to the current scene frame and a template coordinate matrix corresponding to the template frame are constructed respectively. Matrix operations are then performed on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene frame and the template frame. Based on the offset matrix, frame pose difference data is generated and fed back to the robot. This allows the robot to be instructed to perform door unloading based on the frame pose difference data. This achieves accurate planning of a collision-free, high-precision grasping trajectory based on the frame pose difference data, ensuring the safety, smoothness, and efficiency of the entire unloading process, and improving the efficiency of door unloading. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is an application environment diagram of a door unloading method based on material frame positioning in one embodiment;

[0014] Figure 2 This is a flowchart illustrating a door unloading method based on material frame positioning in one embodiment.

[0015] Figure 3 This is a schematic diagram illustrating feature point comparison based on multiple template slot feature points and multiple current slot feature points in one embodiment;

[0016] Figure 4 This is a schematic diagram of the process for obtaining multiple current card slot feature points in the current scene in one embodiment;

[0017] Figure 5 This is a flowchart illustrating a door unloading method based on material frame positioning in another embodiment.

[0018] Figure 6 This is a structural block diagram of a door unloading processing device based on material frame positioning in one embodiment;

[0019] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] The door unloading method based on material frame positioning provided in this application can be applied to, for example... Figure 1In the application environment shown, robot 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Robot 102 can be a robot terminal equipped with different types of controllers or control algorithms, featuring a robotic arm for grasping target objects, such as car doors, and performing actions based on control data (such as pose data for a material frame) fed back from server 104 to place the target object, such as a car door, into the location of the material frame. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0022] Specifically, both robot 102 and server 104 can be used independently to execute the door unloading processing method based on material frame positioning provided in this application embodiment. Robot 102 and server 104 can also collaboratively execute the door unloading processing method based on material frame positioning provided in this application embodiment. For example, taking the collaborative execution of the door unloading processing method based on material frame positioning provided in this application embodiment by robot 102 and server 104, server 104 collects the initial material frame point cloud data of the current scene material frame, converts the initial material frame point cloud data into material frame point cloud data in the robot coordinate system, and performs plane fitting processing on the material frame point cloud data to obtain multiple current slot feature points in the current scene. Specifically, server 104 obtains multiple template slot feature points corresponding to a pre-configured template material frame, and performs feature point comparison based on the multiple template slot feature points and multiple current slot feature points to obtain multiple combinations of identically named feature points whose feature point distance is less than a preset distance threshold. Based on these multiple combinations of identically named feature points, a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame are constructed respectively. Furthermore, the server 104 performs matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene material frame and the template material frame, and generates material frame pose difference data based on the offset matrix. The material frame pose difference data is then fed back to the robot 102 to instruct the robot 102 to perform door unloading processing based on the material frame pose difference data.

[0023] In one exemplary embodiment, such as Figure 2 As shown, a method for unloading car doors based on material frame positioning is provided, which is then applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S202 to S208. Wherein:

[0024] Step S202: Collect the initial point cloud data of the current scene's material frame, convert the initial point cloud data of the material frame into point cloud data of the material frame in the robot coordinate system, and perform plane fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene.

[0025] Specifically, the server uses image acquisition devices, such as a 3D structured light camera integrated into the robot's end effector, to acquire point clouds of the slots in the current scene's material frame from different locations, obtaining initial point cloud data for the current scene's material frame. Simultaneously, while acquiring the initial point cloud data, the robot's shooting posture needs to be recorded to transform the point cloud data from the camera coordinate system to the robot coordinate system. That is, by converting the data obtained from the camera to the robot's end effector, the same physical relationship is established between the initial point cloud data in the robot's end effector coordinate system and the world coordinate system, thus achieving the goal of converting the initial point cloud data into point cloud data in the robot's coordinate system.

[0026] Furthermore, after obtaining the point cloud data of the material frame, the server performs preliminary segmentation on the point cloud data of the material frame to obtain the point cloud data of each individual slot in the material frame, and then performs secondary segmentation on the slot point cloud data to obtain the point cloud data of multiple slot surface blocks included in a single slot. A single slot may include multiple slot surface blocks, and each slot surface block can be a nylon block surface; that is, the point cloud data of a slot surface block corresponds to the point cloud data of one nylon block surface.

[0027] Specifically, for each card slot surface block point cloud data, the server performs planar fitting processing on the point cloud data to obtain the card slot surface block plane. For each card slot surface block plane, the plane normal vector of the card slot surface block can be calculated. After obtaining the point cloud data of each card slot surface block, the card slot surface blocks whose angle values ​​meet the filtering criteria are retained based on the angle value between the plane normal vector of the card slot surface block and the Z-axis. Finally, the card slot centroid is determined based on the card slot surface blocks that meet the filtering criteria, and the card slot centroid is used as the corresponding card slot feature point. After obtaining the card slot feature points corresponding to each card slot, multiple current card slot feature points in the current scene are obtained.

[0028] Step S204: Obtain multiple template slot feature points corresponding to the pre-configured template material frame, and perform feature point comparison based on the multiple template slot feature points and multiple current slot feature points to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold.

[0029] Specifically, the server compares feature points based on multiple template card slot feature points and multiple current card slot feature points to obtain the feature point distance between each template card slot feature point and each current card slot feature point. Specifically, it calculates the Euclidean distance between each template card slot feature point and each current card slot feature point. For example, taking a set of 3 template card slot feature points (A1, B1, C1) and 3 current card slot feature points (A2, B2, C2), it is necessary to calculate the Euclidean distances between A1 and A2, A1 and B2, A1 and C2, B1 and A2, B1 and B2, B1 and C2, C1 and A2, C1 and B2, and C1 and C2.

[0030] Specifically, for each template card slot feature point, the server obtains a pre-set distance threshold and compares the feature distance between each template card slot feature point and each current card slot feature point with the pre-set distance threshold. Current card slot feature points whose feature distances are less than the pre-set distance threshold are identified as target card slot feature points matching the template card slot feature points. For example, for three template card slot feature points (A1, B1, C1) and three current card slot feature points (A2, B2, C2), after calculating the feature distances between each template card slot feature point and each current card slot feature point, such as A1-A2, B1-B2, and C1-C2, if the feature distances are less than the pre-set distance thresholds, then A2 is identified as a target card slot feature point matching template card slot feature point A1, B2 is identified as a target card slot feature point matching template card slot feature point B1, and C2 is identified as a target card slot feature point matching template card slot feature point C1.

[0031] Furthermore, the server combines template slot feature points with target slot feature points that match the template slot feature points to obtain combinations of feature points with the same name, until multiple template slot feature points are traversed to obtain multiple combinations of feature points with the same name. For example, after obtaining target slot feature points A2 that match template slot feature point A1, B2 that match template slot feature point B1, and C2 that match template slot feature point C1, the server combines template slot feature point A1 and target slot feature point A2 to obtain the combination of feature points with the same name A1A2. Similarly, combining template slot feature point B1 and target slot feature point B2 yields the combination of feature points with the same name B1B2, and combining resource and template slot feature point C1 with target slot feature point C2 yields the combination of feature points with the same name C1C2.

[0032] In an exemplary embodiment, before feature comparison, the server needs to acquire multiple template slot feature points of the template frame under standard conditions. This process specifically includes: calling an image acquisition device, such as a 3D area array structured light camera integrated into the robot's end effector, to acquire point clouds of the slots in the template frame at different locations, thereby obtaining template frame point cloud data. Simultaneously, when acquiring the template frame point cloud data, the robot's shooting posture needs to be recorded to convert the point cloud data in the camera coordinate system to the robot coordinate system. That is, by converting the data obtained by the camera to the robot's end effector, the same physical relationship is established between the template frame point cloud data in the robot's end effector coordinate system and the world coordinate system, thereby achieving the purpose of converting the template frame point cloud data into template frame point cloud data in the robot coordinate system.

[0033] After obtaining the point cloud data of the template frame, the server performs initial segmentation to obtain the point cloud data of each individual template slot within the template frame. Then, it performs secondary segmentation to obtain the point cloud data of multiple template slot surface blocks included in a single template slot. Each template slot can include multiple template slot surface blocks, and each template slot surface block can be a nylon block surface; that is, the point cloud data of a template slot surface block corresponds to the point cloud data of one nylon block surface.

[0034] Furthermore, for each template slot surface block point cloud data, the server performs planar fitting processing on the template slot surface block point cloud data to obtain the template slot surface block plane. For each template slot surface block plane, the plane normal vector of the template slot surface block can be calculated. After obtaining the point cloud data of each template slot surface block, the template slot surface blocks whose angle values ​​meet the screening criteria are retained based on the angle between the plane normal vector and the Z-axis. Finally, the template slot centroid is determined based on the template slot surface blocks that meet the screening criteria, and the template slot centroid is used as the template slot feature point corresponding to the template slot. This process continues until the template slot feature points corresponding to each template slot are obtained, resulting in multiple template slot feature points corresponding to the template material frame.

[0035] In one exemplary embodiment, such as Figure 3 As shown, this provides a schematic diagram of feature point comparison based on multiple template slot feature points and multiple current slot feature points. Figure 3 It can be seen that a standard material frame consists of two layers with the same structure. Each layer includes left and right sides, with four slots on each side. In other words, a standard material frame includes 16 slots.

[0036] Specifically, in this embodiment of the application, taking a single layer of a standard material frame as an example, the left and right sides of the single layer of the material frame are respectively provided with 4 slots, including 8 slots. Specifically, the template material frame includes 8 template slot feature points (A1, B1, C1, D1, E1, F1, G1, H1), and the current scene material frame also includes 8 current slot feature points (A2, B2, C2, D2, E2, F2, G2, H2). By calculating each template slot feature point and each current slot feature point, the process is completed. After determining the feature point distance between the groove feature points, the identified combinations of corresponding feature points include: A1A2, B1B2, C1C2, D1D2, E1E2, F1F2, G1G2, and H1H2. By arranging multiple combinations of corresponding feature points left and right, the left-side combinations of corresponding feature points include A1A2, B1B2, C1C2, and D1D2, while the right-side combinations include E1E2, F1F2, G1G2, and H1H2.

[0037] Step S206: Based on multiple combinations of feature points with the same name, construct the scene coordinate matrix corresponding to the current scene frame and the template coordinate matrix corresponding to the template frame.

[0038] Specifically, for multiple combinations of feature points with the same name, the server arranges them according to the material frame slot layout. That is, the material frame is divided into two layers, each layer has two sides, and both sides have the same number of slots, for example, four slots on each side. After arranging the multiple combinations of feature points with the same name according to the material frame slot layout, the arranged combinations of feature points with the same name are obtained. Since the standard material frame has two layers, the server needs to further determine the multiple feature point pairs belonging to the same layer. This is so that, based on the multiple feature point pairs at the same layer, a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame can be constructed.

[0039] For example, when the corresponding feature point combinations obtained in a single layer of a standard material frame include A1A2, B1B2, C1C2, D1D2, E1E2, F1F2, G1G2, and H1H2, these combinations are arranged left and right. The left side of this arrangement includes A1A2, B1B2, C1C2, and D1D2, while the right side includes E1E2, F1F2, G1G2, and H1H2. Therefore, the multiple feature point pairs in the same layer specifically include A1A2, B1B2, C1C2, and D1D2 on the left, and E1E2, F1F2, G1G2, and H1H2 on the right. Similarly, the standard material frame also includes a second layer, and the processing method for the second layer is the same as that for the first layer, resulting in multiple feature point pairs located on the left and right sides of the second layer.

[0040] Specifically, after obtaining multiple feature point pairs belonging to the same layer, the server constructs an initial first scene coordinate axis corresponding to the current scene material frame and an initial first template coordinate axis corresponding to the template material frame based on the feature point coordinates of each of the multiple feature point pairs belonging to the same layer. The server then performs a cross product between the third standard coordinate axis and the initial first scene coordinate axis to obtain a second scene coordinate axis corresponding to the current scene material frame, and performs a cross product between the third standard coordinate axis and the initial first template coordinate axis to obtain a second template coordinate axis corresponding to the template material frame.

[0041] For example, when the server constructs the initial first scene coordinate axis corresponding to the current scene frame and the initial first template coordinate axis corresponding to the template frame based on the coordinates of multiple feature points belonging to the same layer, it specifically constructs the initial X-axis by subtracting the coordinates of the left and right points of the same layer, then summing and averaging the results to form the initial X-axis, which includes the initial first scene coordinate axis and the initial first template coordinate axis. The third standard coordinate axis is specifically the standard Z-axis, which is obtained by cross-product of the standard Z-axis and the initial first scene coordinate axis (i.e., the initial X-axis) to obtain the second scene coordinate axis corresponding to the current scene frame, i.e., the Y-axis of the current scene frame, and by cross-product of the standard Z-axis and the initial first template coordinate axis (i.e., the initial X-axis) to obtain the second template coordinate axis corresponding to the template frame, i.e., the Z-axis of the template frame.

[0042] Furthermore, the server obtains the first scene coordinate axis corresponding to the current scene material frame by cross-product of the third standard coordinate axis and the second scene coordinate axis, and obtains the first template coordinate axis corresponding to the template material frame by cross-product of the third standard coordinate axis and the second template coordinate axis. Thus, the server can construct the scene coordinate matrix corresponding to the current scene material frame based on the first scene coordinate axis, the second scene coordinate axis and the third standard coordinate axis, and construct the template coordinate matrix corresponding to the template material frame based on the first template coordinate axis, the second template coordinate axis and the third standard coordinate axis.

[0043] For example, the server obtains the first scene coordinate axis (i.e., the final X-axis of the scene material frame) corresponding to the current scene material frame by cross-product of the standard Z-axis and the second scene coordinate axis (i.e., the Y-axis of the current scene material frame), and obtains the first template coordinate axis (i.e., the final Z-axis of the template material frame) corresponding to the template material frame by cross-product of the standard Z-axis and the second template coordinate axis (i.e., the Y-axis of the template material frame). Finally, based on the first scene coordinate axis (i.e., the final X-axis of the scene material frame), the second scene coordinate axis (i.e., the Y-axis of the current scene material frame), and the third standard coordinate axis (i.e., the standard Z-axis), a scene coordinate matrix corresponding to the current scene material frame is constructed, and based on the first template coordinate axis (i.e., the final X-axis of the template material frame), the second template coordinate axis (i.e., the Y-axis of the template material frame), and the third standard coordinate axis (i.e., the standard Z-axis), a template coordinate matrix corresponding to the template material frame is constructed.

[0044] Step S208: Perform matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene material frame and the template material frame, generate material frame pose difference data based on the offset matrix, and feed the material frame pose difference data back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0045] Specifically, the server performs matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the rotation matrix between the current scene material frame and the template material frame. That is, the values ​​of the X, Y, and Z axes of the scene coordinate matrix and the template coordinate matrix are used to form the first three columns of the material frame pose matrix (including the first three columns of the scene material frame pose matrix and the first three columns of the template material frame pose matrix). Based on the values ​​of the X, Y, and Z axes of the scene coordinate matrix and the template coordinate matrix, the inverse matrix operation is performed to obtain the rotation matrix between the current scene material frame and the template material frame (that is, the first three columns of the offset matrix).

[0046] Specifically, the server calculates the mean value of the feature point coordinates of multiple combinations of feature points with the same name. Specifically, it sums the centroids of multiple combinations of feature points with the same name and then takes the average value to obtain the mean value. The obtained mean value is used as the last column of the material frame pose matrix (that is, as the last column of the offset matrix). Specifically, it is used as the translation matrix between the current scene material frame and the template material frame. By combining the rotation matrix and the translation matrix, the offset matrix between the current scene material frame and the template material frame can be obtained.

[0047] Furthermore, after obtaining the offset matrix between the current scene material frame and the template material frame, the process also includes: the server generating material frame pose difference data based on the offset matrix, including converting the rotation matrix in the offset matrix into angle values ​​Euler angles according to the Euler angle formula of the six-axis robot, while keeping the translation matrix unchanged, using the obtained angle values ​​Euler angles and translation matrix as material frame pose difference data, and feeding the material frame pose difference data back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0048] After obtaining the material frame pose difference data, the robot specifically acts according to the material frame pose difference data, grabs the door and docks with the material frame, so as to unload the material or product in the material frame through the door and load it onto the transport vehicle, thereby realizing automatic unloading of the door.

[0049] In the above-mentioned door unloading processing method based on material frame positioning, the initial material frame point cloud data of the current scene is collected, and the initial material frame point cloud data is converted into material frame point cloud data in the robot coordinate system. The material frame point cloud data is subjected to plane fitting processing to obtain multiple current slot feature points in the current scene, and multiple template slot feature points corresponding to the pre-configured template material frame are obtained. Based on the multiple template slot feature points and multiple current slot feature points, feature point comparison can be performed to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold. Thus, the position change of the current scene material frame relative to the template material frame can be accurately obtained, and the accurate positioning of the current scene material frame can be achieved. Furthermore, based on multiple combinations of identical feature points, a scene coordinate matrix corresponding to the current scene frame and a template coordinate matrix corresponding to the template frame are constructed respectively. Matrix operations are then performed on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene frame and the template frame. Based on the offset matrix, frame pose difference data is generated and fed back to the robot. This allows the robot to be instructed to perform door unloading based on the frame pose difference data. This achieves accurate planning of a collision-free, high-precision grasping trajectory based on the frame pose difference data, ensuring the safety, smoothness, and efficiency of the entire unloading process, and improving the efficiency of door unloading.

[0050] In one exemplary embodiment, such as Figure 4 As shown, the steps for obtaining multiple current slot feature points in the current scene, that is, performing planar fitting processing on the material frame point cloud data to obtain multiple current slot feature points in the current scene, specifically include the following steps S402 to S406. Wherein:

[0051] Step S402: Based on the angle-constrained region growth segmentation strategy, the point cloud data of the material frame is initially segmented to obtain the point cloud data of the card slots corresponding to each of the multiple card slots.

[0052] Specifically, the server performs preliminary segmentation of the material frame point cloud data according to the region growth segmentation strategy with angle constraints, obtains multiple initial point clouds, and uses any one of the multiple initial point clouds as a seed point cloud to obtain multiple neighboring point clouds located in the neighborhood of the seed point cloud corresponding to the seed point cloud.

[0053] Furthermore, the server calculates the angle difference between each adjacent point cloud and the seed point cloud to obtain a first preset angle difference threshold corresponding to the first preset filtering condition. It then compares the angle difference with the first preset angle difference threshold, identifying adjacent point clouds with angle differences less than the first preset angle difference threshold as those satisfying the first preset filtering condition. These adjacent point clouds satisfying the first preset filtering condition are then used as the same surface point cloud corresponding to the seed point cloud. This allows the seed point cloud and the same surface point cloud to be grouped into the same region, resulting in the card slot point cloud data corresponding to a single card slot. The preset angle difference threshold can be set and adjusted according to actual application scenarios or requirements, and is not limited to a single or certain values.

[0054] Meanwhile, the server marks the filtered identical surface point clouds as segmented point clouds and traverses multiple initial point clouds until all initial point clouds are marked as segmented point clouds, thus obtaining the card slot point cloud data corresponding to each of the multiple card slots.

[0055] The angle-constrained region growing segmentation strategy is specifically implemented as follows: The normal vectors of all point clouds are calculated. Then, one or more point clouds are used as "seed points." The angle differences between the seed point and all points in its neighborhood are calculated. Points belonging to the same smooth surface (such as a wall or tabletop) should have similar angle differences. If the angle difference of a neighboring point meets a set threshold, that point is added to the current region and marked as segmented, becoming a new seed point. This process is repeated until the current region can no longer grow (i.e., no new neighboring points meet the condition). Then, the next unsegmented seed point is selected, and a new region growing process begins until all points have been processed and marked as segmented.

[0056] Step S404: Using a clustering segmentation strategy, the point cloud data of multiple card slots are segmented a second time to obtain the point cloud data of the card slot surface blocks corresponding to each of the multiple card slots.

[0057] Specifically, the server employs a clustering segmentation strategy (which could be a Euclidean segmentation strategy) to perform Euclidean segmentation on the multiple slot point cloud data obtained from region growing and segmentation, thus obtaining slot surface block point cloud data corresponding to each slot. Each slot can correspond to multiple slot surface block point cloud data sets.

[0058] Step S406: Perform plane fitting processing on the point cloud data of the card slot surface blocks corresponding to each of the multiple card slots to obtain multiple current card slot feature points in the current scene.

[0059] Specifically, for each card slot, the server performs plane fitting processing on the point cloud data of multiple card slot surface blocks corresponding to the card slot according to the plane fitting strategy, so as to obtain the card slot surface block plane corresponding to each of the multiple card slot surface block point cloud data.

[0060] For example, the plane fitting strategy can specifically be the RANSAC algorithm, also known as the RANdomSampleConsensus algorithm, which can be understood as a random sampling consensus algorithm. It is an iterative algorithm that can still robustly estimate the parameters of a mathematical model when noise and outliers are as high as 50% or more. In other words, by using the RANSAC algorithm framework with adaptive iterative termination, robust plane fitting is performed on the 3D point cloud, and three non-collinear points are randomly selected to generate candidate planes. The consensus measure is that the distance from the point to the plane is less than a preset angle difference threshold. The remaining number of iterations is dynamically estimated and the iterations are terminated in advance. Finally, the card slot surface block plane that passes the consensus measure is obtained.

[0061] Furthermore, the server calculates the plane normal vectors of each of the multiple card slot surface block planes, determines the angle values ​​between the multiple plane normal vectors and the preset coordinate axis (specifically, the standard Z-axis), and obtains the second preset angle difference threshold corresponding to the second preset screening condition. The server compares each angle value with the second preset angle difference threshold to determine the card slot surface block plane with the angle value less than the second preset angle difference threshold as the card slot surface block plane that meets the second preset screening condition, and the card slot surface block plane that meets the second preset screening condition is taken as the valid card slot surface block plane.

[0062] After obtaining the effective card slot surface block plane, the server determines the centroid of the effective card slot surface block plane and uses the centroid of the effective card slot surface block plane as the card slot feature point corresponding to the card slot, until the card slot feature point corresponding to each card slot is obtained. Based on the card slot feature point corresponding to each card slot, multiple current card slot feature points in the current scene are obtained.

[0063] In this embodiment, the point cloud data of the material frame is initially segmented according to the angle-constrained region growth segmentation strategy to obtain the point cloud data of the slots corresponding to each slot. Then, a clustering segmentation strategy is used to perform secondary segmentation on the point cloud data of the slots to obtain the point cloud data of the slot surface blocks corresponding to each slot. Plane fitting processing is then performed on the point cloud data of the slot surface blocks corresponding to each slot to obtain multiple current slot feature points in the current scene. This realizes multiple segmentation and calculation of the point cloud data of the material frame, reducing the possibility of data omission or calculation errors. It can accurately obtain multiple current slot feature points in the current scene. It utilizes local point pair features to effectively overcome occlusion and noise interference. It has the advantages of strong robustness, high accuracy and high computational efficiency. It is suitable for material box positioning in industrial scenarios such as robot grasping and depalletizing, and improves the business processing efficiency of target grasping, door unloading and other operations in industrial scenarios.

[0064] In one exemplary embodiment, such as Figure 5 As shown, a method for unloading car doors based on material frame positioning is provided, specifically including the following steps S501 to S520. Wherein:

[0065] Step S501: Collect the initial point cloud data of the current scene's material frame, and convert the initial point cloud data of the material frame into point cloud data of the material frame in the robot coordinate system.

[0066] Specifically, the server calls an image acquisition device, such as a 3D area array structured light camera integrated into the robot's end effector, to acquire point clouds of the slots in the current scene's material frame at different locations, thereby obtaining the initial point cloud data of the current scene's material frame. The point cloud data in the camera coordinate system is then converted to the robot coordinate system to obtain the point cloud data of the material frame in the robot coordinate system.

[0067] Step S502: According to the region growth segmentation strategy with angle constraints, the material frame point cloud data is initially segmented to obtain multiple initial point clouds. Any one of the multiple initial point clouds is used as a seed point cloud, and multiple adjacent point clouds located in the neighborhood of the seed point cloud corresponding to the seed point cloud are obtained.

[0068] Specifically, the server performs region growth segmentation on the material frame point cloud data according to the region growth segmentation strategy with angle constraints, obtains multiple initial point clouds, takes any one of the multiple initial point clouds as a seed point cloud, and obtains multiple neighboring point clouds located in the neighborhood of the seed point cloud corresponding to the seed point cloud.

[0069] Step S503: Calculate the angle difference between each adjacent point cloud and the seed point cloud, and determine the adjacent point clouds whose angle difference meets the first preset screening condition as the same surface point cloud corresponding to the seed point cloud.

[0070] Specifically, the server calculates the angle difference between each adjacent point cloud and the seed point cloud to obtain a first preset angle difference threshold corresponding to the first preset filtering condition. It then compares the angle differences with the first preset angle difference threshold, identifying adjacent point clouds with angle differences less than the first preset angle difference threshold as those satisfying the first preset filtering condition. These adjacent point clouds satisfying the first preset filtering condition are then designated as the same surface point cloud corresponding to the seed point cloud. The preset angle difference threshold can be set and adjusted according to actual application scenarios or requirements, and is not limited to a single or certain values.

[0071] Step S504: Divide the seed point cloud and the point cloud of the same surface into the same region to obtain the card slot point cloud data corresponding to a single card slot, and mark the point cloud of the same surface as the segmented point cloud.

[0072] Specifically, the server can obtain the card slot point cloud data corresponding to a single card slot by dividing the seed point cloud and the point cloud of the same surface corresponding to the seed point cloud into the same region.

[0073] Step S505: Traverse multiple initial point clouds until all initial point clouds are marked as segmented point clouds, and obtain the card slot point cloud data corresponding to each of the multiple card slots.

[0074] Specifically, the server marks the filtered identical surface point clouds as segmented point clouds and traverses multiple initial point clouds until all initial point clouds are marked as segmented point clouds, thus obtaining the card slot point cloud data corresponding to each of the multiple card slots.

[0075] Step S506: Using a clustering segmentation strategy, the point cloud data of multiple card slots are segmented twice to obtain the point cloud data of the card slot surface blocks corresponding to each of the multiple card slots.

[0076] Specifically, the server uses a clustering segmentation strategy (which may be a Euclidean segmentation strategy) to perform Euclidean segmentation on the multiple card slot point cloud data obtained by region growing and segmentation, thereby obtaining card slot surface block point cloud data corresponding to each of the multiple card slots.

[0077] Step S507: For each card slot, according to the plane fitting strategy, perform plane fitting processing on the point cloud data of multiple card slot surface blocks corresponding to the card slot to obtain the card slot surface block plane corresponding to each of the multiple card slot surface block point cloud data, and calculate the plane normal vector of each of the multiple card slot surface block planes to determine the angle value between the multiple plane normal vectors and the preset coordinate axis.

[0078] Specifically, for each card slot, the server performs plane fitting processing on the point cloud data of multiple card slot surface blocks corresponding to the card slot according to a plane fitting strategy, such as the RANSAC algorithm (i.e., random sampling consensus algorithm), to obtain the card slot surface block plane corresponding to each of the multiple card slot surface block point cloud data.

[0079] Furthermore, the server calculates the plane normal vectors of each of the multiple card slot surface blocks to determine the angle between the multiple plane normal vectors and the preset coordinate axis (specifically, the standard Z-axis).

[0080] Step S508: The card slot surface block plane whose included angle value meets the second preset screening condition is determined as the valid card slot surface block plane, and the centroid of the valid card slot surface block plane is taken as the card slot feature point corresponding to the card slot.

[0081] Specifically, the server obtains the second preset angle difference threshold corresponding to the second preset filtering condition, and compares each included angle value with the second preset angle difference threshold to determine the card slot surface block plane with the included angle value less than the second preset angle difference threshold as the card slot surface block plane that meets the second preset filtering condition, and takes the card slot surface block plane that meets the second preset filtering condition as the valid card slot surface block plane.

[0082] Furthermore, the server determines the centroid of the effective card slot surface block plane and uses the centroid of the effective card slot surface block plane as the card slot feature point corresponding to the card slot.

[0083] Step S509, until the card slot feature points corresponding to each card slot are obtained, and multiple current card slot feature points in the current scene are obtained based on the card slot feature points corresponding to each card slot.

[0084] Specifically, after obtaining the effective card slot surface block plane, the server determines the centroid of the effective card slot surface block plane and uses the centroid of the effective card slot surface block plane as the card slot feature point corresponding to the card slot. The step of obtaining the card slot feature point corresponding to the card slot needs to be repeated until the card slot feature point corresponding to each card slot is successfully obtained. Furthermore, based on the card slot feature point corresponding to each card slot, multiple current card slot feature points in the current scene are obtained.

[0085] Step S510: Obtain multiple template slot feature points corresponding to the pre-configured template material frame; perform feature point comparison based on the multiple template slot feature points and multiple current slot feature points to obtain the feature point distance between each template slot feature point and each current slot feature point.

[0086] Before performing feature comparison, the server needs to obtain multiple template slot feature points corresponding to the pre-configured template material frame. The specific processing steps for obtaining multiple template slot feature points corresponding to the pre-configured template material frame are similar to the process of obtaining multiple current slot feature points in the current scene. The difference lies in the processing objects: one is the point cloud data of the current scene material frame, and the other is the point cloud data of the template material frame.

[0087] Specifically, the server compares feature points based on multiple template card slot feature points and multiple current card slot feature points to obtain the feature point distance between each template card slot feature point and each current card slot feature point. Specifically, it calculates the Euclidean distance between each template card slot feature point and each current card slot feature point. For example, taking a set of 3 template card slot feature points (A1, B1, C1) and 3 current card slot feature points (A2, B2, C2), it is necessary to calculate the Euclidean distances between A1 and A2, A1 and B2, A1 and C2, B1 and A2, B1 and B2, B1 and C2, C1 and A2, C1 and B2, and C1 and C2.

[0088] Step S511: For each template card slot feature point, the current card slot feature point whose feature point distance is less than a preset distance threshold is determined as the target card slot feature point that matches the template card slot feature point.

[0089] Specifically, for each template card slot feature point, the server obtains a pre-set preset distance threshold and compares the feature distance between each template card slot feature point and each current card slot feature point with the preset distance threshold. The current card slot feature point whose feature point distance is less than the preset distance threshold is determined as the target card slot feature point that matches the template card slot feature point.

[0090] For example, for three template card slot feature points (A1, B1, C1) and three current card slot feature points (A2, B2, C2), after calculating the feature point distance between each template card slot feature point and each current card slot feature point, if the feature point distance between A1-A2, B1-B2, and C1-C2 is less than a preset distance threshold, then A2 is determined as the target card slot feature point matching the template card slot feature point A1, B2 is determined as the target card slot feature point matching the template card slot feature point B1, and C2 is determined as the target card slot feature point matching the template card slot feature point C1.

[0091] Step S512: Combine the template card slot feature points and the target card slot feature points that match the template card slot feature points to obtain a combination of feature points with the same name.

[0092] Specifically, by combining template card slot feature points and target card slot feature points that match the template card slot feature points, a combination of feature points with the same name is obtained. For example, combining template card slot feature point A1 and target card slot feature point A2 results in a combination of feature points with the same name A1A2, combining template card slot feature point B1 and target card slot feature point B2 results in a combination of feature points with the same name B1B2, and combining resource and template card slot feature point C1 and target card slot feature point C2 results in a combination of feature points with the same name C1C2.

[0093] Step S513 continues until the traversal of multiple template slot feature points is completed, resulting in multiple combinations of feature points with the same name.

[0094] Specifically, after traversing multiple template card slot feature points and determining that each template card slot feature point is combined with its corresponding target card slot feature point, multiple combinations of feature points with the same name can be obtained.

[0095] Step S514: Arrange multiple combinations of feature points with the same name according to the layout of the material frame slot to obtain multiple combinations of feature points with the same name after arrangement, and determine multiple feature point pairs belonging to the same layer for the multiple combinations of feature points with the same name after arrangement.

[0096] Specifically, for multiple combinations of feature points with the same name, the server arranges them according to the material frame slot layout. That is, the material frame is divided into two layers, each layer has two sides, and both sides have the same number of slots, for example, four slots on each side. After arranging the multiple combinations of feature points with the same name according to the material frame slot layout, the arranged combinations of feature points with the same name are obtained. Since the standard material frame has two layers, the server needs to further determine the multiple feature point pairs belonging to the same layer. This is so that, based on the multiple feature point pairs at the same layer, a scene coordinate matrix corresponding to the current scene material frame and a template coordinate matrix corresponding to the template material frame can be constructed.

[0097] Step S515: Based on the coordinates of multiple feature points belonging to the same layer, construct the initial first scene coordinate axis corresponding to the current scene material frame and the initial first template coordinate axis corresponding to the template material frame.

[0098] Specifically, when the server constructs the initial first scene coordinate axis corresponding to the current scene material frame and the initial first template coordinate axis corresponding to the template material frame based on the coordinates of multiple feature points belonging to the same layer, it does so by subtracting the coordinates of the left and right points of the same layer, then summing them and taking the average normalization process to form the initial X-axis, which includes the initial first scene coordinate axis and the initial first template coordinate axis.

[0099] Step S516: Perform a cross product of the third standard coordinate axis and the initial first scene coordinate axis to obtain the second scene coordinate axis corresponding to the current scene material frame; and perform a cross product of the third standard coordinate axis and the initial first template coordinate axis to obtain the second template coordinate axis corresponding to the template material frame.

[0100] Specifically, the third standard coordinate axis is the standard Z-axis. The server then performs a cross product between the standard Z-axis and the initial first scene coordinate axis (i.e., the initial X-axis) to obtain the second scene coordinate axis corresponding to the current scene material frame, which is the Y-axis of the current scene material frame. The server also performs a cross product between the standard Z-axis and the initial first template coordinate axis (i.e., the initial X-axis) to obtain the second template coordinate axis corresponding to the template material frame, which is the Z-axis of the template material frame.

[0101] Step S517: Perform a cross product of the third standard coordinate axis and the second scene coordinate axis to obtain the first scene coordinate axis corresponding to the current scene material frame; and perform a cross product of the third standard coordinate axis and the second template coordinate axis to obtain the first template coordinate axis corresponding to the template material frame.

[0102] Specifically, the third standard coordinate axis is the standard Z-axis. The server obtains the first scene coordinate axis (the final X-axis of the scene material frame) corresponding to the current scene material frame by cross-product of the standard Z-axis and the second scene coordinate axis (i.e., the Y-axis of the current scene material frame), and obtains the first template coordinate axis (the final Z-axis of the template material frame) corresponding to the template material frame by cross-product of the standard Z-axis and the second template coordinate axis (i.e., the Y-axis of the template material frame).

[0103] Step S518: Based on the first scene coordinate axis, the second scene coordinate axis and the third standard coordinate axis, construct the scene coordinate matrix corresponding to the current scene material frame, and based on the first template coordinate axis, the second template coordinate axis and the third standard coordinate axis, construct the template coordinate matrix corresponding to the template material frame.

[0104] Specifically, the server constructs a scene coordinate matrix corresponding to the current scene material frame based on the first scene coordinate axis (i.e., the final X-axis of the scene material frame), the second scene coordinate axis (i.e., the Y-axis of the current scene material frame), and the third standard coordinate axis (i.e., the standard Z-axis). It also constructs a template coordinate matrix corresponding to the template material frame based on the first template coordinate axis (i.e., the final X-axis of the template material frame), the second template coordinate axis (i.e., the Y-axis of the template material frame), and the third standard coordinate axis (i.e., the standard Z-axis).

[0105] Step S519: Perform matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the rotation matrix between the current scene material frame and the template material frame, and calculate the average value of the feature point coordinates based on the combination of multiple feature points with the same name to obtain the translation matrix between the current scene material frame and the template material frame.

[0106] Specifically, the server performs matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the rotation matrix between the current scene material frame and the template material frame. That is, the values ​​of the X, Y, and Z axes of the scene coordinate matrix and the template coordinate matrix are used to form the first three columns of the material frame pose matrix (including the first three columns of the scene material frame pose matrix and the first three columns of the template material frame pose matrix). Then, based on the values ​​of the X, Y, and Z axes of the scene coordinate matrix and the template coordinate matrix, the inverse matrix operation is performed to obtain the rotation matrix between the current scene material frame and the template material frame (that is, the first three columns of the offset matrix).

[0107] Furthermore, the server calculates the mean value based on the coordinates of multiple feature points that are combined with the same name. Specifically, it sums up the centroids of multiple feature points and then takes the average value to obtain the mean value. The obtained mean value is used as the last column of the material frame pose matrix (that is, as the last column of the offset matrix), which is used as the translation matrix between the current scene material frame and the template material frame.

[0108] Step S520: Combine the rotation matrix and translation matrix to obtain the offset matrix between the current scene material frame and the template material frame, and generate material frame pose difference data based on the offset matrix. Feed the material frame pose difference data back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0109] Specifically, the server can obtain the offset matrix between the current scene material frame and the template material frame by combining the rotation matrix and the translation matrix. For the rotation matrix in the offset matrix, the server converts it into Euler angles according to the Euler angle formula of the six-axis robot, while the translation matrix remains unchanged. The obtained Euler angles and translation matrix are used as the material frame pose difference data, and the material frame pose difference data is fed back to the robot to instruct the robot to perform door unloading processing according to the material frame pose difference data.

[0110] In the above-mentioned door unloading processing method based on material frame positioning, the initial material frame point cloud data of the current scene is collected, and the initial material frame point cloud data is converted into material frame point cloud data in the robot coordinate system. The material frame point cloud data is subjected to plane fitting processing to obtain multiple current slot feature points in the current scene, and multiple template slot feature points corresponding to the pre-configured template material frame are obtained. Based on the multiple template slot feature points and multiple current slot feature points, feature point comparison can be performed to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold. Thus, the position change of the current scene material frame relative to the template material frame can be accurately obtained, and the accurate positioning of the current scene material frame can be achieved. Furthermore, based on multiple combinations of identical feature points, a scene coordinate matrix corresponding to the current scene frame and a template coordinate matrix corresponding to the template frame are constructed respectively. Matrix operations are then performed on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene frame and the template frame. Based on the offset matrix, frame pose difference data is generated and fed back to the robot. This allows the robot to be instructed to perform door unloading based on the frame pose difference data. This achieves accurate planning of a collision-free, high-precision grasping trajectory based on the frame pose difference data, ensuring the safety, smoothness, and efficiency of the entire unloading process, and improving the efficiency of door unloading.

[0111] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] Based on the same inventive concept, this application also provides a door unloading device based on a material frame positioning for implementing the aforementioned door unloading method based on material frame positioning. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more door unloading device embodiments based on material frame positioning provided below can be found in the limitations of the door unloading method based on material frame positioning described above, and will not be repeated here.

[0113] In one exemplary embodiment, such as Figure 6As shown, a door unloading processing device based on material frame positioning is provided, including: a plane fitting processing module 602, a feature point comparison module 604, a coordinate matrix construction module 606, and a material frame pose difference data generation module 608, wherein:

[0114] The planar fitting processing module 602 is used to collect the initial point cloud data of the current scene's material frame, convert the initial point cloud data into point cloud data of the material frame in the robot coordinate system, and perform planar fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene; the feature point comparison module 604 is used to obtain multiple template slot feature points corresponding to the pre-configured template material frame, and perform feature point comparison based on the multiple template slot feature points and multiple current slot feature points to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold; the coordinate matrix construction module 606 is used to construct the scene coordinate matrix corresponding to the current scene's material frame and the template coordinate matrix corresponding to the template material frame based on the multiple combinations of the same-name feature points; the material frame pose difference data generation module 608 is used to perform matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene's material frame and the template material frame, and generate material frame pose difference data based on the offset matrix, and feed the material frame pose difference data back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

[0115] In the aforementioned door unloading processing device based on material frame positioning, the initial material frame point cloud data of the current scene material frame is collected, and the initial material frame point cloud data is converted into material frame point cloud data in the robot coordinate system. The material frame point cloud data is subjected to plane fitting processing to obtain multiple current slot feature points in the current scene, and multiple template slot feature points corresponding to the pre-configured template material frame are obtained. Based on the multiple template slot feature points and multiple current slot feature points, feature point comparison can be performed to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold. Thus, the positional change of the current scene material frame relative to the template material frame can be accurately obtained, and accurate positioning of the current scene material frame can be achieved. Furthermore, based on multiple combinations of identical feature points, a scene coordinate matrix corresponding to the current scene frame and a template coordinate matrix corresponding to the template frame are constructed respectively. Matrix operations are then performed on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene frame and the template frame. Based on the offset matrix, frame pose difference data is generated and fed back to the robot. This allows the robot to be instructed to perform door unloading based on the frame pose difference data. This achieves accurate planning of a collision-free, high-precision grasping trajectory based on the frame pose difference data, ensuring the safety, smoothness, and efficiency of the entire unloading process, and improving the efficiency of door unloading.

[0116] In an exemplary embodiment, the plane fitting processing module is further configured to: perform preliminary segmentation of the material frame point cloud data according to the angle-constrained region growth segmentation strategy to obtain the card slot point cloud data corresponding to each of the multiple card slots; perform secondary segmentation of the multiple card slot point cloud data respectively using a clustering segmentation strategy to obtain the card slot surface block point cloud data corresponding to each of the multiple card slots; and perform plane fitting processing on the card slot surface block point cloud data corresponding to each of the multiple card slots to obtain multiple current card slot feature points in the current scene.

[0117] In an exemplary embodiment, the plane fitting processing module is further configured to: perform preliminary segmentation of the material frame point cloud data according to the region growth segmentation strategy with angle constraints to obtain multiple initial point clouds; take any one of the multiple initial point clouds as a seed point cloud and obtain multiple adjacent point clouds located in the neighborhood of the seed point cloud corresponding to the seed point cloud; calculate the angle difference between each adjacent point cloud and the seed point cloud, determine the adjacent point clouds whose angle difference meets the first preset screening condition as the same surface point cloud corresponding to the seed point cloud, and mark the same surface point cloud as a segmented point cloud; divide the seed point cloud and the same surface point cloud into the same region to obtain the card slot point cloud data corresponding to a single card slot; traverse multiple initial point clouds until all multiple initial point clouds are marked as segmented point clouds to obtain the card slot point cloud data corresponding to each of the multiple card slots.

[0118] In an exemplary embodiment, the plane fitting processing module is further configured to: for each card slot, perform plane fitting processing on the multiple card slot surface block point cloud data corresponding to the card slot according to the plane fitting strategy, to obtain card slot surface block planes corresponding to each of the multiple card slot surface block point cloud data; calculate the plane normal vectors of the multiple card slot surface block planes, and determine the angle values ​​between the multiple plane normal vectors and the preset coordinate axes; determine the card slot surface block planes whose angle values ​​meet the second preset screening conditions as valid card slot surface block planes, and take the plane centroid of the valid card slot surface block planes as the card slot feature points corresponding to the card slots; until the card slot feature points corresponding to each card slot are obtained, and obtain multiple current card slot feature points in the current scene based on the card slot feature points corresponding to each card slot.

[0119] In an exemplary embodiment, the feature point comparison module is further configured to: perform feature point comparison based on multiple template card slot feature points and multiple current card slot feature points to obtain the feature point distance between each template card slot feature point and each current card slot feature point; for each template card slot feature point, determine the current card slot feature points whose feature point distance is less than a preset distance threshold as target card slot feature points that match the template card slot feature points; combine the template card slot feature points and the target card slot feature points that match the template card slot feature points to obtain a combination of feature points with the same name; until the traversal of multiple template card slot feature points is completed to obtain multiple combinations of feature points with the same name.

[0120] In an exemplary embodiment, the coordinate matrix construction module is further configured to: arrange multiple combinations of feature points with the same name according to the layout of the material frame slots to obtain multiple combinations of feature points with the same name after arrangement; determine multiple pairs of feature points belonging to the same layer for the multiple combinations of feature points with the same name after arrangement; construct an initial first scene coordinate axis corresponding to the current scene material frame and an initial first template coordinate axis corresponding to the template material frame based on the feature point coordinates of the multiple feature point pairs belonging to the same layer; perform a cross product of the third standard coordinate axis and the initial first scene coordinate axis to obtain a second scene coordinate axis corresponding to the current scene material frame and a third standard coordinate axis... The cross product of the standard axis and the initial first template coordinate axis is used to obtain the second template coordinate axis corresponding to the template material frame; the cross product of the third standard coordinate axis and the second scene coordinate axis is used to obtain the first scene coordinate axis corresponding to the current scene material frame; the cross product of the third standard coordinate axis and the second template coordinate axis is used to obtain the first template coordinate axis corresponding to the template material frame; based on the first scene coordinate axis, the second scene coordinate axis, and the third standard coordinate axis, a scene coordinate matrix corresponding to the current scene material frame is constructed; and based on the first template coordinate axis, the second template coordinate axis, and the third standard coordinate axis, a template coordinate matrix corresponding to the template material frame is constructed.

[0121] In an exemplary embodiment, the frame pose difference data generation module is further configured to: perform matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain a rotation matrix between the current scene frame and the template frame; perform mean calculation based on the coordinates of multiple feature points with the same name to obtain a translation matrix between the current scene frame and the template frame; and combine the rotation matrix and the translation matrix to obtain an offset matrix between the current scene frame and the template frame.

[0122] Each module in the aforementioned door unloading processing device based on material frame positioning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0123] In one exemplary embodiment, a computer device is provided. This computer device can be a terminal device or a server. Taking the computer device as a server as an example, its internal structure diagram can be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores initial frame point cloud data, frame point cloud data, current slot feature points, pre-configured template frames, template slot feature points, feature point distances, preset distance thresholds, combinations of identical feature points, scene coordinate matrices, template coordinate matrices, offset matrices, and frame pose difference data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a door unloading method based on the positioning of the material frame.

[0124] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0127] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0130] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this application. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for unloading car doors based on material frame positioning, characterized in that, The method includes: Collect initial point cloud data of the current scene's material frame, convert the initial point cloud data of the material frame into point cloud data of the material frame in the robot coordinate system, and perform plane fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene. Get multiple template slot feature points corresponding to the pre-configured template material frame, and compare the feature points based on the multiple template slot feature points and the multiple current slot feature points to obtain a combination of multiple feature points with the same name whose feature point distance is less than a preset distance threshold. Based on the combination of the multiple feature points with the same name, a scene coordinate matrix corresponding to the current scene frame and a template coordinate matrix corresponding to the template frame are constructed respectively. Matrix operations are performed based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene material frame and the template material frame. Material frame pose difference data is generated based on the offset matrix and fed back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

2. The method according to claim 1, characterized in that, The step of performing planar fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene includes: Based on the angle-constrained region growth segmentation strategy, the point cloud data of the material frame is initially segmented to obtain the point cloud data of the card slots corresponding to each of the multiple card slots. A clustering segmentation strategy is adopted to perform secondary segmentation on the point cloud data of multiple card slots to obtain the point cloud data of the card slot surface blocks corresponding to each of the multiple card slots. Planar fitting processing is performed on the point cloud data of the card slot surface blocks corresponding to each of the multiple card slots to obtain multiple current card slot feature points in the current scene.

3. The method according to claim 2, characterized in that, The material frame point cloud data is initially segmented according to the region growth segmentation strategy with angular constraints to obtain the point cloud data of each of the multiple slots, including: Based on the region growth segmentation strategy with angle constraints, the point cloud data of the material frame is initially segmented to obtain multiple initial point clouds; Take any one of the multiple initial point clouds as a seed point cloud, and obtain multiple neighboring point clouds located in the neighborhood of the seed point cloud corresponding to the seed point cloud. Calculate the angle difference between each adjacent point cloud and the seed point cloud, determine the adjacent point clouds whose angle difference meets the first preset filtering condition as the same surface point cloud corresponding to the seed point cloud, and mark the same surface point cloud as the segmented point cloud; The seed point cloud and the point cloud of the same curved surface are divided into the same region to obtain the card slot point cloud data corresponding to a single card slot. Traverse the multiple initial point clouds until all the initial point clouds are marked as segmented point clouds to obtain the card slot point cloud data corresponding to each of the multiple card slots.

4. The method according to claim 2, characterized in that, The point cloud data of the card slot surface blocks corresponding to each of the plurality of card slots are subjected to plane fitting processing to obtain multiple current card slot feature points in the current scene, including: For each card slot, according to the plane fitting strategy, plane fitting processing is performed on the multiple card slot surface block point cloud data corresponding to the card slot to obtain the card slot surface block plane corresponding to each of the multiple card slot surface block point cloud data. Calculate the plane normal vectors of each of the multiple card slot surface blocks, and determine the angle values ​​between the multiple plane normal vectors and the preset coordinate axes; The card slot surface block plane whose included angle value meets the second preset screening condition is determined as the valid card slot surface block plane, and the centroid of the plane of the valid card slot surface block plane is taken as the card slot feature point corresponding to the card slot. The process continues until the card slot feature points corresponding to each card slot are obtained. Based on the card slot feature points corresponding to each card slot, multiple current card slot feature points in the current scene are obtained.

5. The method according to any one of claims 1 to 4, characterized in that, The step of comparing feature points based on the multiple template slot feature points and the multiple current slot feature points to obtain a combination of multiple identically named feature points whose feature point distance is less than a preset distance threshold includes: Based on the multiple template card slot feature points and the multiple current card slot feature points, feature point comparison is performed to obtain the feature point distance between each template card slot feature point and each current card slot feature point; For each template card slot feature point, the current card slot feature point whose distance from the feature point is less than the preset distance threshold is determined as the target card slot feature point that matches the template card slot feature point; Combine the template slot feature points with the target slot feature points that match the template slot feature points to obtain a feature point combination with the same name; The process continues until all feature points of the multiple template slots are traversed, resulting in multiple combinations of feature points with the same name.

6. The method according to any one of claims 1 to 4, characterized in that, The step of constructing a scene coordinate matrix corresponding to the current scene bounding box and a template coordinate matrix corresponding to the template bounding box based on the combination of the multiple feature points with the same name includes: The multiple combinations of feature points with the same name are arranged according to the layout of the material frame slots to obtain the arranged multiple combinations of feature points with the same name. For the multiple combinations of feature points with the same name after the arrangement, determine multiple pairs of feature points belonging to the same layer; Based on the coordinates of multiple feature points belonging to the same layer, construct an initial first scene coordinate axis corresponding to the current scene frame and an initial first template coordinate axis corresponding to the template frame; The third standard coordinate axis and the initial first scene coordinate axis are cross-producted to obtain the second scene coordinate axis corresponding to the current scene material frame; the third standard coordinate axis and the initial first template coordinate axis are cross-producted to obtain the second template coordinate axis corresponding to the template material frame. The cross product of the third standard coordinate axis and the second scene coordinate axis is used to obtain the first scene coordinate axis corresponding to the current scene material frame; the cross product of the third standard coordinate axis and the second template coordinate axis is used to obtain the first template coordinate axis corresponding to the template material frame. Based on the first scene coordinate axis, the second scene coordinate axis, and the third standard coordinate axis, a scene coordinate matrix corresponding to the current scene material frame is constructed, and based on the first template coordinate axis, the second template coordinate axis, and the third standard coordinate axis, a template coordinate matrix corresponding to the template material frame is constructed.

7. The method according to any one of claims 1 to 4, characterized in that, The step of performing matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene frame and the template frame includes: Matrix operations are performed based on the scene coordinate matrix and the template coordinate matrix to obtain the rotation matrix between the current scene frame and the template frame; Based on the coordinates of the feature points combined with the multiple identical feature points, the mean value is calculated to obtain the translation matrix between the current scene frame and the template frame; By combining the rotation matrix and the translation matrix, the offset matrix between the current scene frame and the template frame is obtained.

8. A door unloading processing device based on material frame positioning, characterized in that, The device includes: The plane fitting processing module is used to collect the initial point cloud data of the current scene's material frame, convert the initial point cloud data of the material frame into point cloud data of the material frame in the robot coordinate system, and perform plane fitting processing on the point cloud data of the material frame to obtain multiple current slot feature points in the current scene. The feature point comparison module is used to obtain multiple template slot feature points corresponding to the pre-configured template material frame, and to perform feature point comparison based on the multiple template slot feature points and the multiple current slot feature points to obtain multiple combinations of the same-name feature points whose feature point distance is less than a preset distance threshold. The coordinate matrix construction module is used to construct, based on the combination of the multiple feature points with the same name, a scene coordinate matrix corresponding to the current scene frame and a template coordinate matrix corresponding to the template frame, respectively. The material frame pose difference data generation module is used to perform matrix operations based on the scene coordinate matrix and the template coordinate matrix to obtain the offset matrix between the current scene material frame and the template material frame, generate material frame pose difference data based on the offset matrix, and feed the material frame pose difference data back to the robot to instruct the robot to perform door unloading processing based on the material frame pose difference data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images