A target localization and task execution method for blasting scenarios, and a robot

By processing image information within the blasting area to obtain point cloud data of the target object, generating multiple sets of candidate poses and selecting the optimal grasping pose, the problem of inaccurate robot grasping is solved, and precise target positioning and task execution are achieved.

CN123103675APending Publication Date: 2026-11-17ENCOSMART TECHNOLOGY (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611525973.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-29
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing technologies, determining the position of a target object through image recognition cannot determine the actual posture of the target object in space, leading to inaccurate grasping by the robot.

Method used

By processing image information within the blasting area, target point cloud data of the target object is obtained. The point cloud data is used to characterize the three-dimensional contour and spatial distribution of the target object, generate multiple sets of candidate poses, and select the optimal grasping pose based on kinematic prior constraints to control the blasting robot to perform precise grasping.

Benefits of technology

This solves the problem of inaccurate grasping caused by lack of spatial posture information, ensuring that the robot grasps at a reasonable angle and with accurate positioning, avoiding grasping failures, and improving the safety and reliability of operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123103675A_ABST
    Figure CN123103675A_ABST
Patent Text Reader

Abstract

This application relates to the field of robotic blasting technology, and discloses a target localization and task execution method and a robot for blasting scenarios. The method includes: processing image information of a target object within the blasting area to obtain target point cloud data corresponding to the target object; wherein the target object indicates the part that the blasting robot needs to grasp; decomposing the target point cloud data to obtain the initial pose corresponding to the target object; generating multiple sets of candidate poses based on the initial pose and a preset rotation transformation library; filtering the multiple sets of candidate poses based on kinematic prior constraints to locate the target grasping pose of the target object; and controlling the blasting robot to grasp the target object to perform the blasting task based on the target grasping pose of the target object. This application can improve the accuracy of robot grasping.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of robotic blasting technology, and more specifically, to a target localization and task execution method and a robot for blasting scenarios. Background Technology

[0002] The catering industry is undergoing a transformative phase of deep integration with automation and artificial intelligence technologies, with smart kitchens becoming a core development direction. By using collaborative robotic arms to perform standardized, highly repetitive cooking operations such as high-temperature frying, it is possible to effectively improve food preparation efficiency, stabilize food quality, reduce labor costs, and enhance the safety of high-temperature operations in the kitchen.

[0003] In related technologies, for frying target objects, conventional image recognition methods are usually used to determine the location of the food, and then the control system drives the robot to perform the grasping and frying actions according to the preset path.

[0004] However, determining the position of a target object solely through image recognition cannot determine its actual orientation in space, leading to inaccurate grasping by the robot. Summary of the Invention

[0005] In view of this, this application provides a target localization and task execution method and a robot for blasting scenarios, in order to solve the problem that determining the position of a target object through image recognition cannot determine the actual posture of the target object in space, thus causing the robot to grasp it inaccurately.

[0006] Firstly, this application provides a target localization and task execution method for blasting scenarios. The method is applied to a blasting robot and includes: processing image information of a target object within the blasting area to obtain target point cloud data corresponding to the target object; wherein the target object indicates the part that the blasting robot needs to grasp; decomposing the target point cloud data to obtain the initial pose corresponding to the target object; generating multiple sets of candidate poses based on the initial pose and a preset rotation transformation library; filtering the multiple sets of candidate poses based on kinematic prior constraints to locate the target grasping pose of the target object; and controlling the blasting robot to grasp the target object to perform the blasting task based on the target grasping pose.

[0007] The target localization and task execution method for frying scenarios provided in this application obtains target point cloud data corresponding to the target object through image information processing. The target point cloud data is used to represent the three-dimensional contour and spatial distribution of the target object, breaking through the limitation of a single location point. Then, the target point cloud data is decomposed to obtain the initial posture of the target object. Through the decomposition operation of the target point cloud data, the rotation relationship and spatial orientation of the target object are calculated from the three-dimensional structure, enabling the robot to go from knowing only the position of the target object to knowing both the position and posture of the target object. Subsequently, multiple sets of candidate postures are generated based on the initial posture and a preset rotation transformation library to cope with possible posture errors and randomness in food placement, avoiding grasping failure due to inaccurate initial posture.

[0008] Next, based on kinematic prior constraints, multiple candidate postures are screened and the target grasping posture is located. The optimal posture that conforms to the actual posture of the target object and is adapted to the robot's motion capability is selected to further ensure that the grasping angle is reasonable and the position is accurate. Finally, based on the final determined target grasping posture, the blasting robot is controlled to perform grasping and blasting actions according to the precise posture, solving the problem of inaccurate grasping caused by the lack of spatial posture information.

[0009] In one possible implementation, the target point cloud data is decomposed to obtain the initial pose of the target object, including: centering the target point cloud data to obtain centralized point cloud data; performing singular value decomposition on the centralized point cloud data to obtain the corresponding singular vector matrix; determining the initial rotation matrix of the target object based on the singular vector matrix; determining the initial translation vector based on the centroid of the target point cloud data; and combining the initial rotation matrix and the initial translation vector to obtain the initial pose of the target object.

[0010] In one possible implementation, based on the initial pose and a preset rotation transformation library, multiple sets of candidate poses are generated, including: retrieving multiple sets of rotation transformation parameters for the target grabbing pose from a preset rotation transformation library adapted to the blasting scene; applying each set of rotation transformation parameters sequentially to the rotation matrix of the initial pose to obtain multiple sets of candidate rotation matrices; keeping the translation component of the initial pose unchanged, combining each set of candidate rotation matrices with the translation component of the initial pose to generate multiple sets of candidate poses.

[0011] In one possible implementation, the target grasping pose of the target object is located by screening multiple sets of candidate poses based on kinematic prior constraints. This includes: verifying multiple sets of candidate poses based on kinematic prior constraints, eliminating candidate poses that do not meet the kinematic prior constraints, and obtaining valid candidate poses; wherein, the kinematic prior constraints include at least one of the following: the joint limits of the blasting robot, the workspace range, the obstacle avoidance space, and the opening and closing stroke of the gripper of the blasting robot; and locating the target grasping pose of the target object based on the valid candidate poses.

[0012] In one possible implementation, the target grasping pose of the target object is located based on the effective candidate poses, including: selecting the pose with the smallest deviation from the initial pose or the smallest interference with the blasting area from the effective candidate poses as the target grasping pose.

[0013] In one possible implementation, based on the target grasping pose, the blasting robot is controlled to grasp the target object to perform the blasting task, including: performing inverse kinematics solution on the target grasping pose to obtain the target angles corresponding to each joint of the blasting robot; and sending the target object into the blasting area for blasting based on the target angles.

[0014] In one possible implementation, the target object is sent into the frying area for frying based on the target angle, including: immersing the target object in the frying oil in the frying area based on the target angle; controlling the immersion depth of the target object in the oil according to a preset frying time; and removing the target object from the frying area after frying is completed.

[0015] In one possible implementation, the image information of the target object within the blasting area is processed to obtain the target point cloud data corresponding to the target object, including: segmenting the image information of the target object within the blasting area using a multi-strategy segmentation method to extract the target object region; performing mask filtering on the depth image based on the target object region to obtain the original depth point cloud corresponding to the target object; wherein, the depth image is an image representing the spatial depth information of scene elements within the blasting area; and filtering and denoising the original depth point cloud to reconstruct the regularized target point cloud data.

[0016] In one possible implementation, the method further includes: during the blasting process of the target object, acquiring image information of the blasting area and updating the corresponding target point cloud data; based on the updated target point cloud data, detecting the pose offset between the current grasping pose of the target object and the target grasping pose; when the pose offset exceeds a preset pose offset threshold, re-executing the target object positioning process to correct the target grasping pose; and controlling the blasting operation based on the corrected target grasping pose.

[0017] Secondly, this application provides a robot for blasting scenarios, including: a processor and a memory for storing executable instructions, the memory and the processor being communicatively connected to each other, and the processor executing the executable instructions to perform the target localization and task execution method for blasting scenarios described in the first aspect or any corresponding embodiment.

[0018] Thirdly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the target localization and task execution method for a bombing scenario as described in the first aspect or any of its corresponding embodiments.

[0019] Fourthly, this application provides a computer program product, including computer instructions, which are used to cause a computer to execute the target positioning and task execution method for a blasting scenario described in the first aspect or any of its corresponding embodiments. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a target localization and task execution method for a blasting scenario according to an embodiment of this application; Figure 2 This is a flowchart illustrating another target location and task execution method for a blasting scenario according to an embodiment of this application; Figure 3 This is a flowchart illustrating another target location and task execution method for a blasting scenario according to an embodiment of this application; Figure 4 This is a block diagram of a robot for a frying scenario according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Based on relevant technologies, it is known that the catering industry is undergoing a profound transformation driven by automation and artificial intelligence, with smart kitchens becoming a core development direction. Utilizing robotic arms to perform standardized, highly repetitive cooking tasks, such as the high-temperature frying process focused on in this invention, is of great value in improving food preparation efficiency, ensuring consistent food quality, reducing labor costs, and enhancing kitchen safety.

[0024] The specific application scenario of this invention is: deploying a collaborative robotic arm, with the end effector adopting an eye-in-hand mode and equipped with an industrial-grade high dynamic range RGB-D camera, to perform fully unmanned operation on a frying basket containing food.

[0025] However, when applying laboratory-based robot vision technology to real, dynamic kitchen environments, especially under extreme conditions like deep frying, the technology has several shortcomings, as follows: Visual perception is prone to failure in extreme environments. Kitchen frying areas contain large amounts of high-temperature oil fumes and water vapor, which form scattering media and reduce image clarity. Simultaneously, the boiling oil surface and stainless steel cookware surfaces produce high-intensity, dynamically changing specular reflections and ambient light reflections. Algorithms relying on traditional edge detection, feature point matching, or some deep learning segmentation models are highly susceptible to failure when image quality is severely degraded, making it impossible to reliably extract target information.

[0026] High-precision pose estimation struggles to balance real-time performance, accuracy, and robustness. Existing 3D positioning technologies cannot simultaneously meet the needs of kitchen scenarios: SLAM (Simultaneous Localization and Mapping)-based solutions are computationally intensive and prone to losing positioning in kitchen environments with repetitive or missing textures, failing to achieve precise grasping in an absolute coordinate system; structured light or laser scanning-based solutions are costly, and the projected structured light patterns are easily interfered with by oil fumes and water vapor, resulting in large areas of voids or distortions in the depth data; solutions based on traditional markers (such as QR codes and April Tags) only achieve centimeter-level positioning accuracy, failing to meet the precise grasping requirements of frying basket handles, and significantly reducing decoding and positioning robustness when the target is occluded or the image is blurred.

[0027] The operation target exhibits pose ambiguity, resulting in insufficient operational safety. The pose estimation of the frying basket handle is a non-convex optimization problem, which is prone to getting trapped in local optima and causing pose ambiguity, such as confusing the front and back of the handle. In high-temperature oil frying scenarios, incorrect pose can directly lead to the failure of the robotic arm to grasp the basket, causing safety accidents such as the frying basket falling and hot oil splashing, making it difficult to meet the requirements of commercial applications.

[0028] Based on this, this application provides a target localization and task execution method for frying scenarios. This method obtains target point cloud data corresponding to the target object through image information processing, and uses the target point cloud data to represent the three-dimensional contour and spatial distribution of the target object, breaking through the limitation of a single location point. Then, the target point cloud data is decomposed to obtain the initial posture of the target object. Through the decomposition operation of the target point cloud data, the rotation relationship and spatial orientation of the target object are calculated from the three-dimensional structure, enabling the robot to know both the position and posture of the target object. Subsequently, multiple sets of candidate postures are generated based on the initial posture and a preset rotation transformation library to cope with possible posture errors and randomness in food placement, avoiding grasping failure due to inaccurate initial posture. Next, the multiple sets of candidate postures are screened and the target grasping posture is located based on kinematic prior constraints. The optimal posture that conforms to the actual posture of the target object and is adapted to the robot's motion capabilities is selected to further ensure that the grasping angle is reasonable and the position is accurate. Finally, based on the finally determined target grasping posture, the frying robot is controlled to perform grasping and frying actions according to the precise posture, solving the problem of inaccurate grasping caused by the lack of spatial posture information.

[0029] According to an embodiment of this application, a target localization and task execution method for a bombing scenario is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This embodiment provides a target localization and task execution method for blasting scenarios, which can be used for blasting robots. Figure 1 This is a flowchart of a target localization and task execution method for a blasting scenario according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: Step S101: Process the image information of the target object within the blasting area to obtain the target point cloud data corresponding to the target object; wherein, the target object indicates the part that the blasting robot needs to grasp.

[0031] Image information can be understood as RGB images, depth images, or RGB-D fused images of the frying area. The target object can be understood as the part of the food to be fryed that the robot can grip, is not easily damaged, and is not easily detached, such as the edges of the food, the handle, or the skeletal area. Target point cloud data can be understood as the set of points representing the three-dimensional spatial coordinates of the target object.

[0032] Specifically, the collected images of the frying area are segmented, filtered, and denoised to separate the target point cloud data corresponding to the target object from the scene and eliminate interference from oil, pot body, and background.

[0033] As an example, the acquired RGB image is preprocessed sequentially by grayscale conversion, adaptive threshold dehazing, and contrast enhancement to weaken oil mist and reflective interference. Then, it is combined with three strategies—edge detection segmentation, region growing segmentation, and threshold segmentation—to accurately locate the parts of the food suitable for robot grasping, and remove background interference areas such as the inner wall of the oil pan, oil, and the frame. Subsequently, a binary mask is generated for the target area and synchronously mapped to the depth image acquired from the same viewpoint. Invalid depth pixels other than those in the mask are removed, and only the depth pixel information corresponding to the target object is retained. Then, the two-dimensional depth pixels are converted into three-dimensional spatial coordinate points to obtain the target point cloud data.

[0034] Step S102: Decompose the target point cloud data to obtain the initial pose of the target object.

[0035] The initial attitude can characterize the approximate position and orientation of the target object in space.

[0036] Specifically, the point cloud is first centered, then the main direction of the point cloud is obtained through matrix decomposition, the initial rotation relationship of the target object is calculated accordingly, and the initial attitude is obtained by combining the point cloud centroid as the translation amount.

[0037] As an example, the target point cloud data is centered by calculating the mean of all 3D coordinate points in the target point cloud. The origin of the point cloud coordinate system is then translated to this mean point to eliminate the influence of the overall positional offset of the point cloud on the attitude calculation, resulting in centered point cloud data. Subsequently, a covariance matrix is ​​constructed from the centered point cloud data. The covariance matrix is ​​decomposed using singular value decomposition (SVD) to obtain three mutually orthogonal singular vectors. These singular vectors correspond to the three principal directions of the target object in space. The three singular vectors are arranged sequentially to form an initial rotation matrix, which is used to characterize the spatial orientation and rotation relationship of the target object. Simultaneously, the centroid coordinates of all coordinate points in the target point cloud are calculated and used as the initial translation vector to characterize the specific position of the target object in space. Finally, the initial rotation matrix and the initial translation vector are combined using a rigid body transformation to obtain the initial attitude of the target object.

[0038] As an example, the target point cloud data is preprocessed, and outlier noise points are further removed through statistical filtering to ensure the integrity of the point cloud contour. Then, principal component analysis (PCA) is used to extract principal components from the preprocessed point cloud data, calculate the covariance matrix of the point cloud, and solve for the eigenvalues ​​and eigenvectors of the covariance matrix. The eigenvector with the largest eigenvalue corresponds to the principal axis direction of the target object, the eigenvector with the second largest eigenvalue corresponds to the secondary axis direction of the target object, and the eigenvector with the smallest eigenvalue corresponds to the normal direction of the target object. The three extracted eigenvectors are used as the spatial principal directions of the target object and mapped to the initial rotation matrix to determine the spatial orientation of the target object. Then, the centroid of the point cloud is used as the initial translation vector to determine the spatial position of the target object. The combination of the two forms the initial pose of the target object.

[0039] Step S103: Based on the initial pose and the preset rotation transformation library, generate multiple sets of candidate poses.

[0040] The preset rotation transformation library can be understood as a pre-configured set of multiple small-angle rotations around the X / Y / Z axes or a combination of axes. Candidate poses can be understood as multiple feasible poses generated by fine-tuning rotations based on the initial pose.

[0041] Specifically, based on the initial posture, multiple sets of rotation parameters are retrieved from the rotation transformation library, and the rotation part of the initial posture is transformed in turn while keeping the position unchanged, generating multiple sets of similar candidate postures.

[0042] As an example, a safe angle range for candidate pose generation is defined. This range can be determined by considering the kinematic constraints of the frying robot, the spatial limitations of the frying area, and the stability requirements for food grasping. The safe angle, as the rotation angle in the small-angle fine-tuning sub-library, has a range of, for example, no more than -6 to +6 degrees, to avoid the rotation angle being too large, causing the pose to exceed the robot's workspace or interfere with the frying equipment. Based on this safe angle range, uniform random sampling or Gaussian random sampling is used to randomly sample a certain number of rotation angles around the X, Y, and Z axes, ensuring that the sampling angles for each axis are within the safe range. The number of samples can be adjusted according to actual needs (e.g., 20 to 50 sets). The rotation angles of the three axes obtained from each set of random sampling are combined into random rotation parameters and applied to the rotation matrix of the initial pose to generate multiple sets of candidate rotation matrices after random rotation. Keeping the initial translation vector unchanged, each set of candidate rotation matrices is combined with the translation vector to obtain the corresponding number of candidate poses.

[0043] Step S104: Based on kinematic prior constraints, multiple candidate poses are screened to locate the target grasping pose of the target object.

[0044] The target grasping pose can be understood as the optimal grasping pose that is determined after screening, which the blasting robot can stably execute and is adapted to the target object.

[0045] Specifically, each group of candidate poses is substituted into the constraints in turn for feasibility verification, infeasible poses are eliminated, and the best one is selected from the remaining valid poses as the final grasping pose.

[0046] As an example, the kinematic parameters of the frying robot are obtained in advance, including the three-dimensional coordinate range of the robot's workspace and the angular limits of each joint (e.g., the maximum forward rotation angle and maximum reverse rotation angle of the shoulder, elbow, and wrist joints). These kinematic parameters are preset as constraints. For each set of candidate postures, forward kinematics are solved based on the robot's kinematic model to calculate the joint angles and the spatial coordinates of the end effector corresponding to each set of candidate postures. The calculated joint angles are compared with the preset joint limits. If the angle of any joint exceeds the limit range, the candidate posture is determined to be infeasible and is eliminated. At the same time, the spatial coordinates of the end effector are compared with the preset workspace range. If the coordinates exceed the workspace (e.g., exceeding the graspable range above the frying area, below the bottom of the oil pan, or exceeding the frame limit range), it is also determined to be an infeasible posture and is eliminated. After this step of verification, only candidate postures whose joint angles and end effector positions meet the constraints are retained.

[0047] Step S105: Based on the target grasping pose, control the blasting robot to grasp the target object to perform the blasting task.

[0048] Specifically, inverse kinematics is solved based on the target grasping pose to obtain the angles of each joint of the robot, and the robot is controlled to grasp the target object in the optimal posture and complete the blasting task according to the preset process.

[0049] As an example, combining the kinematic model of the frying robot, inverse kinematics calculations are performed on the target grasping pose to obtain the target rotation angle, motion velocity, and acceleration parameters corresponding to each joint of the robot, ensuring that the joint limit and motion smoothness requirements of the frying robot are met. Based on the calculated motion parameters, the motion trajectory of the frying robot is planned. The trajectory design needs to be adapted to the characteristics of the frying scene, using a gentle curved trajectory to avoid sudden stops or turns that could cause food posture deviation or oil splashing. At the same time, the starting point of the trajectory (such as the grasping station), intermediate transition points (such as a preset position above the oil surface), and immersion endpoint (such as a preset depth position in the oil) are clearly defined. The robot is controlled to move smoothly according to the planned trajectory, first moving the grasped target to a preset position above the oil surface, etc.

[0050] In one possible implementation scenario, a depth camera captures image information of ingredients with handles in the feeding area, such as the outline and spatial position of the frying handle. The frying handle is the target object for grasping. After processing in steps S101 to S104, the target grasping pose of the frying handle is determined. Combining the kinematic model of the frying robot, the inverse kinematics calculation is performed on the target grasping pose of the frying handle to obtain the target rotation angle and acceleration parameters of the robot's shoulder joint, elbow joint, wrist joint, and end effector. Based on the calculated joint motion parameters, a smooth curved motion trajectory of the robot is planned, with the trajectory starting point clearly defined as the frying handle corresponding to the storage tray in the feeding area. The handle position and intermediate transition point are 1cm above the oil surface in the fryer, and the immersion endpoint is 2cm inside the oil. The robot moves smoothly along the planned trajectory, first moving to the collection tray, gripping the middle of the frying handle according to the target gripping posture, and after confirming that the grip is firm, it moves the chicken nugget with handle to 1cm above the oil surface, pauses for M1 seconds to confirm that the frying handle and chicken nugget posture are correct, and then slowly descends at a speed of M2cm / s to immerse the chicken nugget into the frying oil until the preset immersion depth of X3cm is reached. The frying handle is kept at 4cm above the oil surface. The whole process is without sudden stops or turns to avoid oil splashing and the frying handle posture deviation.

[0051] The target localization and task execution method for frying scenarios provided in this application obtains target point cloud data corresponding to the target object through image information processing. The target point cloud data is used to represent the three-dimensional contour and spatial distribution of the target object, breaking through the limitation of a single location point. Then, the target point cloud data is decomposed to obtain the initial posture of the target object. Through the decomposition operation of the target point cloud data, the rotation relationship and spatial orientation of the target object are calculated from the three-dimensional structure, enabling the robot to go from knowing only the position of the target object to knowing both the position and posture of the target object. Subsequently, multiple sets of candidate postures are generated based on the initial posture and a preset rotation transformation library to cope with possible posture errors and randomness in food placement, avoiding grasping failure due to inaccurate initial posture. Next, based on kinematic prior constraints, multiple candidate postures are screened and the target grasping posture is located. The optimal posture that conforms to the actual posture of the target object and is adapted to the robot's motion capability is selected to further ensure that the grasping angle is reasonable and the position is accurate. Finally, based on the final determined target grasping posture, the blasting robot is controlled to perform grasping and blasting actions according to the precise posture, solving the problem of inaccurate grasping caused by the lack of spatial posture information.

[0052] In one possible implementation, such as Figure 2 As shown, step S102 above includes the following steps: Step S2021: The target point cloud data is centrally processed to obtain centralized point cloud data.

[0053] Specifically, the target point cloud data of the blasting handle obtained in step S101 is extracted, the three-dimensional coordinates of all target point cloud data are traversed, and then the mean values ​​of the X, Y and Z coordinates are calculated respectively. The coordinate point composed of the three mean values ​​is taken as the mean point of the point cloud. Then, the mean value of the corresponding coordinate is subtracted from the three-dimensional coordinates of each point cloud, and the origin of the target point cloud is translated to the mean point to obtain the centered point cloud data.

[0054] Step S2022: Perform singular value decomposition on the centralized point cloud data to obtain the corresponding singular vector matrix.

[0055] Specifically, the covariance is obtained by calculating the covariance between the coordinate components of the centered point cloud. Then, singular value decomposition is performed on the constructed covariance matrix to obtain three matrices: the U matrix (left singular vector matrix), the S matrix (singular value diagonal matrix), and the V matrix (right singular vector matrix). Among them, the V matrix is ​​the required singular vector matrix, and its three column vectors are orthogonal to each other, corresponding to the three principal directions of the blasting handle in space. The magnitude of the singular values ​​in the singular value diagonal matrix S corresponds to the concentration of point cloud distribution in each principal direction. The larger the singular value, the more the corresponding principal direction reflects the contour features of the blasting handle (e.g., the length direction of the blasting handle corresponds to the column vector corresponding to the largest singular value).

[0056] Step S2023: Determine the initial rotation matrix of the target object based on the singular vector matrix.

[0057] The initial rotation matrix is ​​a 3×3 orthogonal matrix that characterizes the spatial rotation relationship of the frying handle and is used to describe the spatial orientation (such as tilt angle and rotation angle) of the frying handle relative to the world coordinate system.

[0058] Specifically, a singular vector matrix is ​​extracted, whose three column vectors correspond to the three principal spatial directions of the handle. The first column vector corresponds to the length direction of the handle, the second column vector corresponds to the width direction, and the third column vector corresponds to the thickness direction. Then, the V matrix is ​​transposed to obtain V0. T The matrix, initially used as the basis for the initial rotation matrix; further, V... T The matrix is ​​checked for orthogonality by checking whether det(V) is +1; if it is -1, a certain main direction is flipped to meet the requirements of the right-hand rotation matrix. The matrix elements are adjusted to ensure that the rotation matrix meets the orthogonality requirements and to avoid matrix deviation caused by point cloud noise. The final orthogonal matrix is ​​the initial rotation matrix for blasting the handle.

[0059] Step S2024: Determine the initial translation vector based on the centroid of the target point cloud data.

[0060] The centroid of the target point cloud data can be understood as the average position of all three-dimensional coordinate points of the target point cloud of the blasted handle, used to characterize the geometric center position of the handle.

[0061] Specifically, the target point cloud data of the blasting handle is retrieved, and the three-dimensional coordinates of all point clouds are traversed. The average values ​​of the X, Y, and Z coordinates of all point clouds are calculated, and the coordinate points (Xc, Yc, Zc) formed by the three average values ​​are used as the centroid coordinates of the blasting handle, where Xc is the mean of all X coordinates, Yc is the mean of all Y coordinates, and Zc is the mean of all Z coordinates. Then, the centroid coordinates (Xc, Yc, Zc) are constructed into a 3×1 column vector, which is the initial translation vector.

[0062] In one possible implementation, a weighted average method is used to calculate the centroid. Weights are assigned based on the confidence level of the point cloud (e.g., the point cloud in the middle of the handle has a high confidence level, while the point cloud at the edge has a low confidence level), and then the weighted average is calculated as the centroid.

[0063] Step S2025: Combine the initial rotation matrix and the initial translation vector to obtain the initial attitude of the target object.

[0064] Specifically, the initial rotation matrix is ​​embedded into the 3×3 region at the top left corner of the homogeneous transformation matrix, and the translation vector is filled into the first to third rows and fourth column. The fourth row is set to a fixed value, and the combination is completed to obtain the initial posture.

[0065] The target localization and task execution method for frying scenarios provided in this application embodiment effectively eliminates the interference of overall position offset caused by random placement of the frying handle through target point cloud centralization processing; and adopts the singular value decomposition algorithm to accurately extract the spatial main direction of the frying handle, adapting to interference factors such as oil mist noise and point cloud fluctuations in the frying scenario, ensuring the stability and accuracy of the main direction extraction.

[0066] Furthermore, the initial rotation matrix is ​​determined based on the singular vector matrix. Through orthogonality verification and correction, the rationality of the rotation matrix is ​​ensured, accurately representing the spatial orientation of the blasting handle and adapting to random posture conditions such as handle tilting and rotation. Then, the initial translation vector is determined based on the centroid of the point cloud, which can accurately locate the spatial position of the blasting handle and optimize the centroid calculation method according to the grasping requirements, meeting the accuracy requirements of robot grasping. Finally, the rotation matrix and translation vector are integrated through the homogeneous transformation matrix to obtain the complete initial posture, thereby improving the accuracy of target object position recognition.

[0067] In one possible implementation, such as Figure 3 As shown, step S103 above includes the following steps: Step S3031: Retrieve multiple sets of rotation transformation parameters for the target grabbing pose from a preset rotation transformation library adapted to the blasting scene.

[0068] The rotational variation parameter can be understood as the rotation angle of the target object.

[0069] Specifically, a rotation transformation library adapted to the frying scenario is pre-built. The rotation transformation parameters stored in the pre-built rotation transformation library have all been verified for scenario adaptation. The types of rotation transformation parameters can include rotation angles around a single axis of X, Y, and Z, as well as rotation angles of multi-axis combinations. The range of rotation angles can be set in combination with the shape of the frying handle (such as a long strip handle) and the grasping accuracy requirements. After obtaining the initial posture of the frying handle, multiple sets of adapted rotation transformation parameters are retrieved from the pre-built rotation transformation library according to the material and shape of the frying handle (such as cylindrical or cuboid) and the orientation of the current initial posture, to ensure that the retrieved parameters can cover the possible deviation range of the initial posture.

[0070] Step S3032: Apply each set of rotation change parameters sequentially to the rotation matrix of the initial attitude to obtain multiple sets of candidate rotation matrices.

[0071] For each set of rotational change parameters, first convert them into the corresponding rotation matrix. Then, perform matrix multiplication between the rotation matrix corresponding to each set of rotational change parameters and the initial rotation matrix. Apply the angle adjustment corresponding to each set of rotational change parameters to the initial rotation matrix to obtain multiple different rotation matrices, i.e., candidate rotation matrices.

[0072] Step S3033: Keep the translation component of the initial pose unchanged, combine the candidate rotation matrices and translation components to generate multiple candidate poses.

[0073] For each set of candidate rotation matrices, the standard combination rules of homogeneous transformation matrices are adopted to embed the candidate rotation matrices into the upper left 3×3 sub-matrix region of the 4×4 homogeneous transformation matrix to represent the current orientation of the handle; the initial translation vector is filled into the first to third rows and the fourth column of the homogeneous transformation matrix to represent the spatial position of the handle; through the above combination operation, each set of candidate rotation matrices and the initial translation vector are integrated to obtain multiple complete candidate poses.

[0074] The target localization and task execution method for blasting scenarios provided in this application takes adaptation to blasting scenarios as its core premise. Through the coordinated cooperation of preset rotation transformation library, rotation matrix operation and posture combination, it can generate diverse and highly adaptable candidate postures without additional complex operations. This not only avoids the spatial limitations of blasting scenarios and reduces the risk of equipment interference, but also accurately covers the initial posture error and the randomness of handle orientation.

[0075] In one possible implementation, step S104 above may include the following steps: Step S4041: Based on kinematic prior constraints, multiple candidate postures are verified, and candidate postures that do not meet the kinematic prior constraints are eliminated to obtain valid candidate postures; wherein, the kinematic prior constraints include at least one of the following: joint limit of the blasting robot, workspace range, obstacle avoidance space, and gripper opening and closing stroke of the blasting robot.

[0076] Joint limits can be understood as the preset maximum forward and reverse rotation angle range of each joint of the frying robot (such as the shoulder, elbow, and wrist joints). Workspace range can be understood as all spatial coordinate areas that the frying robot's end effector can reach. Obstacle avoidance space can be understood as preset areas in the frying scenario where the robot is prohibited from entering, mainly including areas containing equipment such as the frying pan, support, and oil pipes, to avoid collisions and interference between the robot and the equipment. Gripper opening and closing stroke can be understood as the range between the maximum and minimum opening degrees that the frying robot's end effector gripper can achieve.

[0077] Specifically, using multiple sets of candidate postures and pre-set kinematic prior constraint parameters, each candidate posture is sequentially verified based on each kinematic prior constraint. The verification order can be set according to the requirements of the blasting scenario; for example, first verify whether the spatial coordinates of the robot end effector corresponding to the candidate posture are within the preset workspace range; if they are outside, the candidate posture is directly eliminated; then verify whether the rotation angles of each joint of the robot corresponding to the posture meet the joint limit requirements; if any joint rotation angle exceeds the limit, it is eliminated; then verify whether the blasting handle and robot end effector corresponding to the candidate posture enter the obstacle avoidance space; if there is spatial overlap, it is eliminated; finally verify whether the opening and closing size of the gripper corresponding to the posture is within the gripper opening and closing stroke range; if it cannot be adapted, it is eliminated; after the above constraint verification, all candidate postures that meet the constraint conditions are retained as valid candidate postures.

[0078] Step S4042: Based on the valid candidate poses, locate the target grasping pose of the target object.

[0079] Specifically, a selection criterion for the target grasping pose can be set, and then based on the selection criterion, a comprehensive evaluation of each group of valid candidate poses can be performed to select the optimal pose as the target grasping pose. During the selection process, the gripping stability (such as the contact area between the gripper and the gripping part of the frying handle, and the distribution of gripping force), posture deviation (such as the amount of deviation from the initial posture, the smaller the deviation, the easier it is to execute), and frying adaptability (such as whether the posture is convenient for the handle to carry the food into the oil and drain the oil) are considered. After the selection is completed, a unique target grasping pose is output.

[0080] In a feasible scenario, the specific scenario adaptation is as follows: Within the frying station, the frying pot, support, temperature control equipment, etc. are arranged compactly, and the obstacle avoidance space is clearly defined (the obstacle avoidance area is within 10cm around the pot). The frying robot has preset joint limits (shoulder joint ±90°, elbow joint ±120°, wrist joint ±180°). The workspace covers the feeding area, frying area, and discharging area, and the gripper opening and closing stroke is adapted to the size of the frying handle. In step S4041, the multiple sets of candidate postures generated in step S3033 are verified, and invalid postures that exceed the workspace (such as exceeding the grasping range above the frying area), interfere with the pot or support, exceed joint limits, or cannot be adapted by the gripper are preferentially eliminated. In step S4042, a weighted comprehensive screening method is used to select the optimal target grasping posture.

[0081] Specifically, step S4042 includes: Step a1: Among the valid candidate poses, select the pose with the smallest deviation from the initial pose or the smallest interference with the blasting area as the target grasping pose.

[0082] Specifically, among the valid candidate postures, the posture with the smallest deviation from the initial posture or the smallest interference with the blasting area is selected as the target grasping posture.

[0083] As an example, for a centralized point cloud matrix P centered For an N×3 matrix, its SVD decomposition is: P centered =U∑V T Where U is the left singular vector matrix in SVD decomposition, V is a 3×3 right singular vector matrix, the column vectors of V correspond to the three principal spatial directions of the object, and the initial rotation matrix is ​​taken as R. initial =V, no transpose of V is needed. If det(V) = -1, then flip any column of V to obtain the corrected matrix V', making det(V') = +1. The initial translation vector is taken as the centroid of the original target point cloud: T initial =mean(P). P is the uncentered original target point cloud, and mean(P) represents the average value of all points in P along each coordinate dimension. T initial This indicates the initial position of the target object in space. (P) centered The mean of is zero, and it is not used to determine the initial translation vector.

[0084] Pre-build a rotation transformation library R lib This rotation transformation library can be used as a large-angle ambiguity resolution sub-library, containing all combinations of rotations of ±90° and ±180° around the X, Y, and Z axes. The initial rotation matrix R... initial With each element R in the rotation transformation library lib,i Perform matrix multiplication to generate a candidate pose set: {R} cand,i |Rcand,i =R initial ×R lib,i}

[0085] i is the index of the rotation transformation parameter in the large-angle ambiguity elimination sub-library. For each candidate rotation matrix R... cand,i Calculate its Euler angles Then, constraints are selected, that is, the initial pose with the smallest deviation, the smallest interference with the blasting area, or all Euler angles are less than the Euler angle threshold are selected from the candidate poses as the target grasping pose.

[0086] The target localization and task execution method for blasting scenarios provided in this application embodiment is based on kinematic prior constraints. Through the coordinated cooperation of candidate posture verification and target grasping posture localization, the method can achieve accurate screening and localization of target grasping posture without additional complex operations. This not only avoids safety risks such as robot joint damage and equipment collisions from the source, but also ensures that the localized target grasping posture is fully adapted to the robot's motion capabilities, the spatial limitations of the blasting scenario, and the grasping requirements of the blasting handle, thus balancing operational safety and efficiency.

[0087] In one possible implementation, step S105 above includes the following steps: Step S5051: Perform inverse kinematics solution on the target grasping pose to obtain the target angles corresponding to each joint of the blasting robot.

[0088] Obtain the inherent kinematic parameters of the blasting robot, including the link length, joint type, and joint limit range of each joint, and construct a robot kinematic model. Based on this robot kinematic model, perform inverse kinematics solution for the target grasping pose, for example, by using matrix transformation, iterative calculation, etc., to deduce the target angles corresponding to each joint of the robot in reverse.

[0089] Step S5052: Based on the target angle, the target object is sent into the blasting area for blasting.

[0090] Based on the target angles of each joint and the three-dimensional coordinate parameters of the frying area (oil level, frying position coordinates, obstacle avoidance area), the target angles of each joint are converted into control signals recognizable by the robot control system. This controls the robot's joints to move smoothly at preset speeds and accelerations, moving the frying handle and food held by the end effector along a preset trajectory. During the movement, the actual angles of each robot joint are collected in real time and compared with the target angles. If a deviation occurs, closed-loop feedback adjustments are immediately implemented to ensure joint movement accuracy. The joints are then controlled to move slowly, smoothly immersing the food in the frying oil to a preset depth, completing the operation of delivering the target object into the frying area.

[0091] Specifically, step S5052 includes: Step b1: Immerse the target object into the frying oil in the frying area based on the target angle.

[0092] By using the target angles of each joint of the robot and the preset parameters of the frying area, combined with the structural characteristics of the frying handle, the starting and ending positions for the immersion of the target object are determined. The target angles are converted into control signals that the control system can recognize, and the robot's joints are controlled to move at a preset slow speed, causing the frying handle and food held by the end gripper to descend smoothly from the transition position towards the oil. During the movement, the actual angles of each joint of the robot are collected and compared with the target angles in a closed-loop feedback. At the same time, the relative position of the food and the oil surface is monitored in real time by a depth camera. If there is a joint angle deviation or food posture deviation, the joint parameters are immediately fine-tuned to ensure that the food is vertically and smoothly immersed. When the food reaches the preset immersion depth and the frying handle is at a preset height above the oil surface, the joints are controlled to stop moving, completing the operation of immersing the target object in the frying oil.

[0093] Step b2: Control the immersion depth of the target object in the oil according to the preset frying time, and remove the target object from the frying area after frying is completed.

[0094] Retrieve the preset frying time parameters and, combined with the preset immersion depth determined in step b1, start the frying timer and real-time immersion depth monitoring. During frying, collect the actual immersion depth of the food in the oil and compare it with the preset immersion depth. If the actual depth deviates due to oil disturbance or slight joint displacement (e.g., the deviation exceeds 1cm), adjust the robot joint angle (based on target angle correction) to bring the food to the preset immersion depth, ensuring even heating. Simultaneously, the temperature control module monitors the oil temperature and compares it with the preset temperature (180-200°C). The robot operates in conjunction with the 0℃ temperature control system. When temperature fluctuations cause changes in oil density, the immersion depth is adjusted synchronously to compensate for the impact of temperature on the frying effect. When the frying timer reaches the preset duration, the food is determined to be fried. The robot then controls each joint to slowly lift at a preset speed, lifting the frying handle and food out of the oil and to a preset draining position above the frying area (10-15cm above the oil surface). The food stays there for a preset time to complete the initial draining. Then, the robot is transferred along a preset trajectory to move the target object out of the frying area and to the discharge area, completing the entire frying and removal operation.

[0095] The target localization and task execution method for frying scenarios provided in this application takes the target grasping pose as the core. Through the coordinated linkage of inverse kinematics solution and target feeding for frying, it can realize the fully automated control of the entire process from pose analysis to actual frying without additional complex parameter adjustments. This not only ensures the safety of the joint movement of the frying robot, but also ensures that the target object enters the frying area accurately and smoothly and completes the frying operation, avoiding risks such as oil splashing, food falling off, and equipment interference.

[0096] In one possible implementation, step S101 above includes the following steps: Step S6011: Use multi-strategy segmentation to segment the image information of the target object within the blasting area and extract the target object region.

[0097] Multi-strategy segmentation can be understood as combining multiple image segmentation algorithms to complement each other and leverage their respective strengths. It aims to achieve accurate separation of the target area from the background area, taking into account the image interference characteristics of frying scenes. The image information of the target object within the frying area can be understood as a color or grayscale image of the frying area captured by a depth camera. Specifically, it can include all scene elements such as the frying handle, the food to be fried, the frying pot, the oil, and the support.

[0098] Specifically, a multi-strategy segmentation scheme is adopted, combining the color, contour, and texture features of the fried handle with multiple segmentation algorithms to process the acquired image information. First, color threshold segmentation is used to initially segment the suspected target area based on the color difference between the fried handle and the background (oil, pot body), and the background with obvious color differences is removed. Then, contour segmentation algorithm is used to extract the contour features of the suspected area, and combined with the long strip contour characteristics of the fried handle, the area that matches the handle contour is selected. Finally, texture segmentation algorithm is used to distinguish the surface texture of the fried handle from the blurred texture of oil mist and oil, and further remove background interference. Through the above multi-strategy collaborative segmentation, the accurate target area is finally extracted.

[0099] Step S6012: Perform mask filtering on the depth image based on the target object region to obtain the original depth point cloud corresponding to the target object.

[0100] A depth image can be understood as an image captured by a depth camera that represents the spatial depth information of various scene elements within the blasting area. Each pixel corresponds to a spatial depth value, reflecting the distance between the element and the camera. Mask filtering can be understood as performing pixel-level matching between the binary mask of the target area and the depth image, retaining only the depth pixels corresponding to the target area in the mask, and removing the depth pixels corresponding to the background area. The original depth point cloud can be understood as the point cloud set formed by converting the depth pixels obtained after mask filtering into three-dimensional spatial coordinates (X, Y, Z).

[0101] Retrieve the binary mask of the target area obtained in step S6011 and the depth image of the blasted area simultaneously acquired by the depth camera, ensuring that the pixel coordinates of the two are completely aligned; perform pixel-level point-by-point matching between the binary mask and the depth image, filter out all depth pixels corresponding to a pixel value of 1 in the mask, and extract the depth values ​​corresponding to these depth pixels; based on the preset calibration parameters (intrinsic and extrinsic parameters) of the depth camera, convert the filtered depth pixels and their corresponding image coordinates into three-dimensional spatial coordinates (X, Y, Z), where the X and Y coordinates correspond to the spatial horizontal coordinates after the image plane coordinates are converted, and the Z coordinate corresponds to the depth value (distance from the camera); use the set of all converted three-dimensional coordinate points as the original depth point cloud corresponding to the target object.

[0102] Step S6013: Filter and denoise the original depth point cloud to reconstruct the normalized target point cloud data.

[0103] A multi-step filtering and denoising strategy is adopted. First, a statistical filtering algorithm is used to calculate the number and distance of neighboring points within a preset range around each point cloud, and discrete noise points with too few neighboring points or abnormal distances are removed. Then, a Gaussian filtering algorithm is used to smooth the remaining point cloud, eliminating point cloud fluctuations caused by depth acquisition errors and making the point cloud distribution more uniform. Subsequently, a pass-through filtering algorithm is used to remove redundant points that exceed the preset size range of the blasting handle. Finally, the filtered point cloud is reconstructed to fill sparse areas and optimize the integrity of the point cloud outline, resulting in regularized target point cloud data.

[0104] The target localization and task execution method for frying scenarios provided in this application embodiment, through the coordinated linkage of multi-strategy image segmentation, mask filtering, and multi-step filtering and denoising, progressively eliminates background interference such as oil mist, pot body, and support in the frying scenario, effectively overcoming problems such as image blurring and abnormal depth data caused by oil mist, ensuring that the extracted target point cloud data is complete, pure, and uniformly distributed, and can accurately represent the three-dimensional contour and spatial position of the frying handle, thereby improving the accuracy of locating the target object.

[0105] In one possible implementation, the above method also includes: Step c1: During the blasting process of the target object, image information of the blasting area is collected and the corresponding target point cloud data is updated.

[0106] After the target object is immersed in the frying oil and the frying timer is started, the depth camera is activated to enter real-time acquisition mode. Color and depth images of the frying area are acquired at a preset acquisition frequency (e.g., 10-20 frames / second). During the acquisition process, periods of oil splashing and high oil mist concentration are avoided to ensure image clarity. Based on the acquired real-time image information, following the process of steps S6011-S6013, the current target object area is first extracted through multi-strategy segmentation to remove background interference such as oil and pot body. Then, the depth image is masked according to the target object area to extract the real-time original depth point cloud corresponding to the target object. Finally, the original depth point cloud is filtered and denoised to reconstruct the updated target point cloud data corresponding to the current target object. The target point cloud data is updated synchronously once for each set of images acquired.

[0107] Step c2: Based on the updated target point cloud data, detect the pose offset between the current grasping pose of the target object and the target grasping pose.

[0108] Based on the updated target point cloud data, the current actual spatial attitude of the target object is obtained through the attitude calculation process of steps S2021-S2025, including the current position and orientation; the target grasping pose, i.e. the preset optimal grasping pose obtained by positioning in step S4042 (or step a1), is the ideal attitude that the target object should maintain during the blasting process; the pose offset refers to the spatial difference between the current grasping pose and the target grasping pose, which is divided into rotation offset (orientation deviation, calculated by the angle between the rotation matrix) and translation offset (position deviation, calculated by the Euclidean distance of the translation vector), used to characterize the degree of target object attitude offset.

[0109] Based on the updated target point cloud data, the initial pose calculation process is repeated to calculate the current grasp pose of the target object; then the pose offset between the current grasp pose and the target grasp pose is calculated, and the rotation offset and translation offset are calculated separately; the rotation offset and translation offset are integrated to obtain the pose offset.

[0110] Step c3: When the pose offset exceeds the preset pose offset threshold, the target object localization process is re-executed to correct the target grasping pose.

[0111] The preset pose offset threshold can be understood as a pre-set offset value. Specifically, the pose offset is compared with the preset threshold to determine whether it exceeds the threshold: if the overall pose offset does not exceed the preset threshold, it means that the target's pose offset is within the allowable range and no correction is needed; the original target grasping pose control and blasting operation continues. If the overall pose offset exceeds the preset threshold, it means that the target's pose offset is too large, which will affect the blasting quality or cause grasping instability. The correction process is immediately started, and the entire target positioning process is re-executed, that is, the target point cloud data is re-extracted, the initial pose is calculated, candidate poses are generated, valid candidate poses are selected, and a new target grasping pose is located to ensure that the new target grasping pose matches the current actual pose of the target and meets the kinematic prior constraints and blasting requirements. After the correction is completed, the new corrected target grasping pose is output.

[0112] Step c4: Control the blasting operation based on the corrected target grasping pose.

[0113] Then, based on the corrected target, the pose is captured to control the blasting operation.

[0114] The target localization and task execution method for frying scenarios provided in this application embodiment achieves full-process dynamic monitoring of the target object's grasping posture by real-time acquisition of images of the frying area and dynamic updating of target point cloud data. It accurately captures posture changes caused by factors such as oil disturbance and joint displacement. Through the coordinated linkage of posture displacement detection, threshold judgment, posture correction and closed-loop control of frying operation, posture displacement exceeding the preset threshold can be corrected in a timely manner, ensuring that the target object always maintains the optimal grasping posture that adapts to the frying requirements throughout the entire frying process, effectively avoiding the impact of posture displacement on frying quality.

[0115] This embodiment also provides a target positioning and blasting device for a blasting scenario. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0116] Figure 4 A module diagram of a robot for blasting scenarios is provided, such as... Figure 4As shown, the robot for the blasting scenario in this embodiment may include: a first processing module 401, used to process the image information of the target object within the blasting area to obtain the target point cloud data corresponding to the target object; wherein, the target object indicates the part that the blasting robot needs to grasp; a second processing module 402, used to decompose the target point cloud data to obtain the initial posture corresponding to the target object; a generation module 403, used to generate multiple sets of candidate postures based on the initial posture and a preset rotation transformation library; a positioning module 404, used to filter the multiple sets of candidate postures based on kinematic prior constraints to locate the target grasping posture of the target object; and a control module 405, used to control the target object to blast based on the target grasping posture of the target object.

[0117] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0118] In this embodiment, the robot in the blasting scenario is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0119] This application also provides a computer device having the above-described features. Figure 4 The robot shown is facing a blasting scenario.

[0120] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application, such as... Figure 5 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0121] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0122] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0123] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0124] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0125] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0126] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0127] The computer device also includes a communication interface for communicating with other devices or communication networks.

[0128] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0129] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0130] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A target localization and task execution method for blasting scenarios, characterized in that, The method is applied to a frying robot, and the method includes: The image information of the target object within the blasting area is processed to obtain the target point cloud data corresponding to the target object; wherein, the target object indicates the part that the blasting robot needs to grasp; The target point cloud data is decomposed to obtain the initial pose of the target object. Based on the initial pose and the preset rotation transformation library, multiple sets of candidate poses are generated; Based on kinematic prior constraints, the multiple candidate poses are filtered to locate the target grasping pose of the target object. Based on the target grasping pose, the blasting robot is controlled to grasp the target object to perform the blasting task.

2. The target localization and task execution method for blasting scenarios according to claim 1, characterized in that, The target point cloud data is decomposed to obtain the initial pose corresponding to the target object, including: The target point cloud data is centrally processed to obtain centralized point cloud data; Singular value decomposition is performed on the centralized point cloud data to obtain the corresponding singular vector matrix; Based on the singular vector matrix, determine the initial rotation matrix of the target object; Determine the initial translation vector based on the centroid of the target point cloud data; The initial rotation matrix and the initial translation vector are combined to obtain the initial pose of the target object.

3. The target localization and task execution method for blasting scenarios according to claim 1, characterized in that, Based on the initial pose and a preset rotation transformation library, multiple candidate poses are generated, including: From a preset rotation transformation library adapted to frying scenarios, retrieve multiple sets of rotation transformation parameters for the target grasping pose; Each set of rotation transformation parameters is applied sequentially to the rotation matrix of the initial posture to obtain multiple sets of candidate rotation matrices; Keeping the translation component of the initial pose unchanged, the candidate rotation matrices of each group and the translation component of the initial pose are combined to generate the multiple groups of candidate poses.

4. The target localization and task execution method for blasting scenarios according to claim 1, characterized in that, The step of filtering the multiple candidate poses based on kinematic prior constraints to locate the target grasping pose of the target object includes: Based on kinematic prior constraints, the multiple sets of candidate postures are verified, and candidate postures that do not meet the kinematic prior constraints are eliminated to obtain valid candidate postures; wherein, the kinematic prior constraints include at least one of the following: the joint limit of the blasting robot, the workspace range, the obstacle avoidance space, and the gripper opening and closing stroke of the blasting robot. Based on the valid candidate poses, the target grasping pose of the target object is located.

5. The target localization and task execution method for blasting scenarios according to claim 4, characterized in that, Based on the valid candidate poses, the target grasping pose of the target object is located, including: Among the valid candidate postures, the posture with the smallest deviation from the initial posture or the smallest interference with the blasting area is selected as the target grasping posture.

6. The target localization and task execution method for blasting scenarios according to claim 1, characterized in that, Based on the target grasping pose, the blasting robot is controlled to grasp the target object to perform the blasting task, including: The target grasping pose is solved by inverse kinematics to obtain the target angles corresponding to each joint of the blasting robot; Based on the target angle, the target object is sent into the frying area for frying.

7. The target localization and task execution method for blasting scenarios according to claim 6, characterized in that, Based on the target angle, the target object is sent into the frying area for frying, including: The target object is immersed in the frying oil in the frying area based on the target angle. The immersion depth of the target object in the oil is controlled according to the preset frying time, and the target object is removed from the frying area after frying is completed.

8. The target localization and task execution method for blasting scenarios according to claim 1, characterized in that, The process of processing the image information of the target object within the blasting area to obtain the target point cloud data corresponding to the target object includes: A multi-strategy segmentation method is used to segment the image information of the target object within the blasting area and extract the target object region; The depth image is masked and filtered according to the target object region to obtain the original depth point cloud corresponding to the target object; wherein, the depth image is an image representing the spatial depth information of scene elements within the blasting area; The original depth point cloud is filtered and denoised to reconstruct the normalized target point cloud data.

9. The target localization and task execution method for a blasting scenario according to any one of claims 1-8, characterized in that, The method further includes: During the blasting process of the target object, image information of the blasting area is collected and the corresponding target point cloud data is updated; Based on the updated target point cloud data, the pose offset between the current grasping pose of the target object and the target grasping pose is detected; When the pose offset exceeds the preset pose offset threshold, the target object localization process is re-executed to correct the target grasping pose. The frying operation is controlled based on the corrected target grasping pose.

10. A robot designed for blasting scenarios, characterized in that, include: processor; Memory used to store executable instructions; The processor is configured to execute the executable instructions to implement the target localization and task execution method for bombing scenarios as described in any one of claims 1-9.