Automatic cutting method and device, electronic equipment and computer program product

By identifying historical cut marks and generating cutting trajectories, the problem of difficulty in balancing accuracy and efficiency in traditional cutting methods is solved, efficient and accurate automated cutting is achieved, and production efficiency and automation level are improved.

CN118968057APending Publication Date: 2024-11-15AUTOMOTIVE WALKING TECHNOLOGY (BEIJING) CO LTD

Patent Information

Application Number
CN202410976106.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Traditional automated cutting methods are inefficient when pursuing high precision and are difficult to balance accuracy and speed. Especially in agriculture and industry, manual judgment of cutting position efficiency is low, resulting in limited production costs and output.

Method used

By obtaining images of color information and spatial position information of the surface of the object to be cut, the pre-trained historical cut mark recognition model recognizes the historical cut mark area, determines the cutable area, and generates the motion trajectory of the cutting tool to avoid repeated cutting and improve cutting Efficiency and accuracy.

Benefits of technology

It achieves efficient and precise cutting in industrial and agricultural cutting, reduces manual intervention, reduces the risk of errors, and improves the level of production automation and cutting efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968057A_ABST
    Figure CN118968057A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an automatic cutting method and device, electronic equipment and a computer program product, two images of the surface of a to-be-cut object are obtained, one image contains color information (a first image), and the other image contains spatial position information (a second image). The two images are aligned at the pixel level, the first image is analyzed through a pre-trained historical cut mark recognition model, whether previous cut marks exist on the surface of the object or not is recognized, repeated cutting in an existing damaged area can be avoided in the recognition process, and according to the recognized position of the historical cut marks, the damage area of the object can be recognized. A new region suitable for cutting is defined in a second image containing spatial information. And finally, based on the determined area capable of being cut and the spatial position information of the area capable of being cut, the motion trail of the cutting tool is automatically generated, it is ensured that cutting is conducted on the ideal and undamaged surface every time, and cutting errors are reduced. And meanwhile, the cutting path is automatically planned, the cutting tool is guided to cut based on the cutting path, and the cutting efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of cutting technology, and in particular to an automatic cutting method, device, electronic equipment and computer program product. Background Art

[0002] Cutting is widely used in industrial and agricultural activities, and its efficiency and accuracy directly affect the cost and output of production. With the continuous development and progress of science and technology, automated cutting has become an important way to improve production efficiency.

[0003] For example, in the agricultural activity of rubber tree tapping, automated tapping needs to be able to accurately detect the location of the rubber tree rubber mark, so as to determine the appropriate new cutting position, and control the cutting tool to complete the cutting based on the new cutting position, so as to ensure the accuracy and efficiency of the tapping process. However, the traditional tapping method relies on manual judgment and manual cutting, which is less efficient.

[0004] For example, in the automated cutting of industrial products, humans are usually required to determine the appropriate cutting position and control the cutting tools for cutting. It is difficult to balance accuracy and efficiency. Specifically, while pursuing high accuracy, a certain cutting speed may be sacrificed, and vice versa. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an automatic cutting method, device, electronic device and computer program product to achieve the technical effect of improving cutting accuracy and efficiency.

[0006] A first aspect of an embodiment of the present application provides an automatic cutting method, the method comprising:

[0007] Acquire a first image carrying color information and a second image carrying spatial position information of the surface of the object to be cut; align pixel positions of the first image and the second image;

[0008] Inputting the first image into a trained historical cut mark recognition model to obtain a historical cut mark recognition result; the historical cut mark recognition result indicates that there are historical cut marks on the surface of the object to be cut;

[0009] Determine, in the second image, a cuttable area on the surface of the object to be cut according to the historical cut mark area in the first image indicated by the historical cut mark recognition result; the cuttable area is an area on the surface of the object to be cut excluding the historical cut mark area;

[0010] A motion trajectory is generated based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the object to be cut along the motion trajectory.

[0011] In the above implementation process, two images of the surface of the object to be cut are obtained, one containing color information (first image) for identifying the surface features of the object (such as historical cut marks); the other containing spatial position information (second image) for precise positioning. The two images are aligned at the pixel level. The first image is analyzed using a pre-trained historical cut mark recognition model, which can identify whether there are previous cutting marks on the surface of the object. This recognition process can avoid repeated cutting in the damaged area. According to the identified historical cut mark position, a new area suitable for cutting is defined in the second image containing spatial information. Finally, based on the determined cuttable area and its spatial position information, the motion trajectory of the cutting tool is automatically generated. By accurately identifying and avoiding historical cut marks, it is ensured that each cut is made on an ideal, undamaged surface, reducing cutting errors. At the same time, the cutting path is automatically planned and the cutting tool is guided to cut based on this, thereby improving cutting efficiency.

[0012] Furthermore, the historical cut mark recognition model is trained by the following steps:

[0013] Performing semantic segmentation on the training image using a semantic segmentation algorithm to determine a recognition area in the training image;

[0014] Acquiring semantic information of the identified area;

[0015] The historical cut mark recognition model is supervisedly trained using the training image carrying the semantic information to obtain the trained historical cut mark recognition model.

[0016] In the above implementation process, the semantic segmentation algorithm is used to process the training images, dividing the images into multiple meaningful regions, each corresponding to a specific category or object part. The semantic information of these identified regions is further extracted. The semantic information refers to the feature description related to the region category, which helps the model understand which features correspond to historical cuts. The labeled semantic information extracted from the segmented image is used as a supervisory signal to train the historical cut recognition model. In this process, the model learns how to accurately distinguish and identify the historical cuts in the image based on these features. After multiple rounds of iterative optimization, a trained model that can effectively identify historical cuts is finally obtained. Through semantic segmentation preprocessing, the model can focus on areas with specific meanings, significantly improving the accuracy and robustness of identifying historical cuts.

[0017] Further, determining the cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result includes:

[0018] According to the pixel position alignment relationship between the first image and the second image, projecting the historical cut mark area onto the second image to obtain spatial position information of the historical cut mark area;

[0019] Based on the spatial position information of the historical cut mark area, the cuttable area in the second image is determined.

[0020] In the above implementation process, the historical cut mark area identified from the first image is mapped to the second image using the known pixel alignment relationship between the first image (containing color information) and the second image (containing spatial position information). The mapping involves converting the 2D image recognition result into 3D spatial coordinates to determine the specific position of the cut mark on the surface of the actual object. According to the spatial position of the historical cut mark projected in the second image, the cuttable area on the surface of the object to be cut is further determined. By projecting the historical cut mark position into three-dimensional space, the position information of the cuttable area can be determined more accurately, thereby improving the accuracy of cutting.

[0021] Further, the second image is a point cloud image; the spatial position information includes point cloud information; and the step of determining the cuttable area in the second image based on the spatial position information of the historical cut mark area includes:

[0022] Preprocessing the point cloud information of the historical cut mark area in the second image; the preprocessing includes filtering and / or cut mark surface clustering;

[0023] The cuttable area in the second image is determined based on the preprocessed point cloud information.

[0024] In the above implementation process, the point cloud information of the identified historical cut mark area is first preprocessed, wherein the preprocessing includes filtering processing to remove noise points and abnormal points, and / or cut mark surface clustering, that is, the point cloud in the cut mark area is grouped according to its spatial distribution characteristics, so as to more clearly identify the cut mark structure. Based on the preprocessed point cloud information, the cuttable area in the second image (i.e., the point cloud image) is further located and divided. By preprocessing the point cloud data, especially filtering and clustering, the recognition accuracy of the boundary of the historical cut mark area can be significantly improved, and interference information can be filtered out.

[0025] Further, determining the cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result includes:

[0026] Based on the historical cut mark area in the first image, an initial position area is determined in the first image; the initial position area is an area excluding the historical cut mark area;

[0027] According to the pixel position alignment relationship between the first image and the second image, the initial position area is projected into the second image to obtain the cuttable area in the second image.

[0028] In the above implementation process, the initial position area is determined in the first image, that is, the area excluding the historical cut mark area, and the image registration technology is used to ensure that the pixel positions between the first image and the second image are accurately aligned. Subsequently, the determined initial position area is mapped from the first image to the second image, so as to locate the cuttable area in the second image. Through the collaborative analysis and projection across images, the accuracy of determining the cuttable area is improved.

[0029] Furthermore, the historical cutting mark recognition result indicates that the surface of the object to be cut does not include the historical cutting mark; and the method further includes:

[0030] The second image is determined as the cuttable area.

[0031] In the above implementation process, if it is determined through analysis that there is no historical cutting mark on the surface of the object to be cut, the entire surface of the object to be cut is regarded as a region suitable for cutting. A method for automatically cutting the surface of the object to be cut without historical cutting marks is provided.

[0032] Furthermore, the first image includes an RGB image; the second image includes a point cloud image; the point cloud image is generated by converting a depth image; the RGB image and the depth image are acquired by an RGB-D camera at the same time, and the pixel positions of the first image and the second image are aligned based on the depth image.

[0033] In the above implementation process, an RGB-D camera is used to simultaneously obtain the RGB image and depth image of the surface of the object to be cut, and the depth image is further processed and converted into a point cloud image. Ensure that the pixel positions of the RGB image and the point cloud image are accurately aligned based on the depth image, and each RGB pixel corresponds to a three-dimensional space point in the point cloud, and establish a mapping relationship between two-dimensional color information and three-dimensional space structure. By combining the color information of the RGB image with the depth information of the point cloud image, it plays an important role in subsequent tasks such as object recognition, segmentation, positioning, and path planning.

[0034] A second aspect of an embodiment of the present application provides a device, the device comprising:

[0035] An acquisition module, used to acquire a first image carrying color information and a second image carrying spatial position information of the surface of the object to be cut; the pixel positions of the first image and the second image are aligned;

[0036] A processing module, used for inputting the first image into a trained historical cut mark recognition model to obtain a historical cut mark recognition result;

[0037] The processing module is further used to determine a cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result; the cuttable area is an area on the surface of the object to be cut excluding the historical cut mark area;

[0038] The operation module is used to generate a motion trajectory based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the surface of the object to be cut along the motion trajectory.

[0039] In the above implementation process, the device obtains image information of the surface of the object to be cut. This includes an image carrying color information, which is used to identify the surface features and historical cutting marks of the object; and another image carrying spatial position information, which is used to accurately locate the three-dimensional spatial structure of the surface of the object. The first image carrying color information is input into a pre-trained historical cutting mark recognition model, which can recognize the traces left by previous cutting on the surface of the object. The identified historical cutting mark area is then mapped to the second image carrying spatial position information, so as to determine the cuttable area on the surface of the object to be cut, that is, the uncut area remaining after removing the historical cutting marks. Finally, according to the determined cuttable area and its spatial position information, the motion trajectory of the cutting tool is automatically generated. By intelligently identifying historical cutting marks and avoiding these areas, the cutting path can be controlled more accurately, reducing miscutting and overcutting, and improving the overall quality and efficiency of the cutting operation. Secondly, through the above steps, the fully automated processing from image acquisition to cutting path planning is realized, which reduces manual intervention, improves the level of production automation, reduces the risk of errors, and improves efficiency.

[0040] According to a third aspect of the present application, an electronic device is provided, the electronic device comprising:

[0041] processor;

[0042] a memory for storing processor-executable instructions;

[0043] Wherein, when the processor calls the executable instruction, any method described in the first aspect is implemented.

[0044] A fourth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a process flow of an automatic cutting method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of another automatic cutting method provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of another automatic cutting method provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of another automatic cutting method provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the structure of a device provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0053] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0054] In current automated cutting activities, it is necessary to manually determine the cutting position and control the cutting tool to perform cutting, and it is difficult to balance accuracy and efficiency.

[0055] In response to any of the above-mentioned problems, the present application provides an automatic cutting method, referring to Figure 1 , Figure 1 A schematic flow chart of an automatic cutting method provided in an embodiment of the present application.

[0056] In this embodiment, the method includes:

[0057] Step S10: acquiring a first image carrying color information and a second image carrying spatial position information of the surface of the object to be cut; aligning pixel positions of the first image and the second image;

[0058] The execution subject of this embodiment can be a host computer, or it can be an embedded system, a smart camera, a robot controller, an edge computing device, or other device with data analysis and processing capabilities. This embodiment and the following embodiments will be described using the host computer as an example.

[0059] It should be noted that the first image carrying color information may be an RGB image, a CMYK image, an HSV image, a Lab image, or other image that can display various color information of the surface of an object.

[0060] The second image carrying the spatial position information may be an image that can provide three-dimensional spatial coordinate information, such as a depth image, a point cloud image, a stereo image pair, or an RGB-D image.

[0061] Spatial position information refers to the position information of the object to be cut in the real world. For example, the spatial position information may be the spatial position information of the object to be cut relative to the acquisition device. Each pixel in the second image carries the spatial position information, and thus can be mapped to the coordinates of the real world.

[0062] The first image and the second image are acquired at the same time by the same acquisition device, or by different acquisition devices at the same time. The first image and the second image are acquired from the same perspective or from different perspectives. The acquisition device can be an RGB-D (i.e., red, green, blue + depth) camera, or a monochrome or color camera, an infrared camera, a binocular stereo camera, or other equipment. The acquisition device can be one or more.

[0063] It is understandable that the pixel positions of the two images need to be accurately aligned through an image processing algorithm, that is, to establish a one-to-one correspondence between color information and spatial position information, which usually involves image registration technology or image alignment technology to ensure that the features identified in the color image match the corresponding spatial positions in the spatial information image. The pixel position alignment operation of the first image and the second image can be completed in the acquisition device or in the host computer.

[0064] Among them, image registration technology includes but is not limited to:

[0065] Feature-based method: detect key points in the image and describe the features, then match the features between different images, and finally align the images through a transformation model.

[0066] Intensity-based methods: directly compare images at the pixel intensity level and use similarity metrics such as mutual information and NCC to optimize the alignment between images.

[0067] Transformation-based method: Assuming that there is a certain geometric transformation relationship between images, the transformation parameters are estimated through optimization algorithms, such as rigid body transformation, affine transformation, etc.

[0068] Template matching based methods: Search for templates or patterns in the image and match them to achieve alignment.

[0069] Frequency domain based methods: use the spectral characteristics of images for registration, such as phase correlation.

[0070] Deep learning-based methods: use neural networks to automatically learn feature representation and transformation parameters, suitable for nonlinear deformation and complex scenes.

[0071] Image alignment techniques include but are not limited to:

[0072] D2C (Depth to Color) hardware solution: A technology designed specifically for RGB-D (red, green, blue + depth) cameras that aims to achieve real-time, efficient pixel-level spatial alignment of depth images and color images, so that depth information can be accurately mapped to every pixel of the color image. The D2C hardware solution function is built directly into the camera's hardware, usually implemented by a dedicated chip or integrated circuit.

[0073] Feature-based methods: Feature extraction and matching techniques such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Orientation FAST and Rotation BRIEF) are used to find stable corresponding feature points in different images, and then calculate the transformation parameters through these corresponding points to align the images.

[0074] Direct geometric transformation: Apply simple geometric transformations such as translation, rotation, scaling, or affine transformations to directly adjust image positions or shapes so that they are spatially aligned. This can be achieved by manually setting or calculating the optimal transformation parameters.

[0075] Intensity-based registration: Use image grayscale values ​​or color histograms for matching, such as mutual information (MI), normalized cross-correlation (NCC) and other similarity metrics to find the best image alignment method.

[0076] Template matching based: Search for a template or pattern in one image for another image, align the images through template matching technology, suitable for scenarios with clear templates.

[0077] RANSAC (Random Sample Consensus): used to remove outliers from a large number of matches, using only inliers to estimate the transformation model, often combined with other feature matching methods to improve the robustness of the alignment.

[0078] Iterative Closest Point (ICP): It is particularly suitable for the alignment of 3D point clouds or surfaces. It finds the closest point pair through iteration and optimizes the transformation parameters to reduce the distance between point clouds.

[0079] Optimization-based methods: Use optimization algorithms (such as gradient descent, least squares method) to minimize a certain metric (such as the difference in overlapping areas) to find the optimal alignment parameters.

[0080] Deep learning method: deep neural network is used to automatically learn feature representation and registration transformation, which can handle complex nonlinear deformation and large-scale data sets.

[0081] Optical flow method: In video sequences, the movement of objects is estimated by analyzing the movement of pixels between consecutive frames (optical flow field), thereby achieving alignment between images or sequences. It is particularly suitable for dynamic scene analysis.

[0082] Step S20: inputting the first image into a trained historical cut mark recognition model to obtain a historical cut mark recognition result; the historical cut mark recognition result indicates that there are historical cut marks on the surface of the object to be cut;

[0083] It should be noted that the cut marks in this embodiment and the following embodiments generally refer to damage, cut marks or other marks on the surface of the object to be cut.

[0084] Historical cut mark recognition models can be built using a variety of techniques to automatically detect damage, cuts, or other marks on the surface of an object. Historical cut mark recognition models include, but are not limited to, the following types:

[0085] (1) Machine Learning Model:

[0086] Support Vector Machine (SVM): Train an SVM model to classify whether a pixel or region in an image is a cut or not.

[0087] Decision trees and random forests: Build rule-based models to determine whether there is a cut through the combination of multiple features.

[0088] K Nearest Neighbor (KNN): Predicts the presence of cuts in unknown images based on the nearest neighbor samples in the training set.

[0089] (2) Deep Learning Model:

[0090] Convolutional Neural Network (CNN): CNN is particularly suitable for image recognition tasks. It can extract features through multiple convolutional layers and learn complex image patterns. It is suitable for directly classifying or locating cuts in the entire image.

[0091] Fully Convolutional Network (FCN): used for image semantic segmentation, which can classify each pixel and accurately mark the location of all cuts in the image.

[0092] U-Net: A special FCN architecture that is often used for tasks that require precise boundary positioning and can also be effectively applied to fine-grained recognition and segmentation of cut marks.

[0093] YOLOv5: An efficient object detection model that can be trained to identify cracks, scratches, and damage in images, including historical cuts.

[0094] (3) Hybrid model: Combining multiple technologies mentioned above, such as using traditional image processing technology for preprocessing, and then inputting the processed image into a deep learning model for further analysis.

[0095] It should be understood that the result of historical cut mark recognition can be in various forms, depending on the recognition technology used. For example, the recognition result form is:

[0096] Semantic segmentation mask: Semantic segmentation can classify each pixel in the image and mark the areas belonging to different categories. In the task of identifying historical cuts, by using deep learning models such as fully convolutional networks (FCN), U-Net, DeepLab series, etc., a model can be trained to identify and distinguish which pixels in the image belong to the cut part and which do not. The output semantic segmentation mask will be a two-dimensional matrix of the same size as the original image, in which each pixel is assigned a category label, usually represented by different colors or grayscale values.

[0097] Binary classification results: The most basic recognition result is to determine whether there are historical cuts on the surface of the object to be detected, which is usually expressed as a binary classification label of "with cuts" or "without cuts".

[0098] Probability score: The model not only determines whether there is a cut, but also gives the confidence of this judgment. For example, it expresses the probability that the model believes there is a cut in percentage form.

[0099] Positioning and size information: In addition to determining whether a cut exists, the model can also output detailed information such as the specific location coordinates of the cut in the image, the size of the cut, and the shape of the cut.

[0100] Segmentation mask: In the image segmentation task, the model assigns a category label to each pixel in the image, generates a segmentation mask, clearly marks all cut areas, and provides an accurate cut distribution map.

[0101] Damage severity assessment: Advanced recognition models may also be able to assess the severity of cuts, such as shallow scratches, deep cuts, or cracks, providing a basis for subsequent maintenance or treatment decisions.

[0102] 3D reconstruction and depth information: In some cases, if a 3D image or depth camera is used, the recognition result may also include the 3D geometric information of the cut.

[0103] Statistical reporting: In batch processing or production line applications, the model may summarize the identification results and generate a report containing information such as quantity statistics, average damage level, etc.

[0104] Step S30: determining a cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result; the cuttable area is an area on the surface of the object to be cut excluding the historical cut mark area;

[0105] It should be noted that the cuttable area may be a whole area consisting of all the cuttable positions on the surface of the object to be cut, or may be a plurality of areas subdivided by all the cuttable positions.

[0106] Optionally, the cuttable area may be all remaining areas in the second image except the historical cut mark area.

[0107] Optionally, the cut area may be a remaining area of ​​the second image except for the historical cut mark area.

[0108] It is understandable that, in order to determine the cuttable area in the second image according to the historical cut mark area in the first image, the following methods can be used:

[0109] Detection and mapping of historical cut areas: In the first image, the historical cut areas are identified and marked by image segmentation or machine learning models (such as deep learning convolutional neural networks (CNNs). These marked areas are mapped to the aligned second image to form a mask of the uncut area. A logical operation is performed on the second image (for example, a "logical negation" operation is performed on the second image and the mask of the uncut area) to obtain a mask of the cuttable area.

[0110] Deep learning method: Train a deep learning model, directly input the second image and historical cut area information, and output an accurate prediction of the cuttable area.

[0111] Step S40: generating a motion trajectory based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the object to be cut along the motion trajectory.

[0112] It can be understood that since the second image carries spatial position information, that is, each pixel position in the second image can correspond to a certain spatial position in the real world. Therefore, after determining the cuttable area from the second image, the spatial position information of the cuttable area can be determined based on the pixel area of ​​the cuttable area in the second image, thereby obtaining the cuttable area carrying spatial position information. Then, the cuttable area carrying spatial position information can be used to guide the generation of the motion trajectory of the cutting tool in the real world.

[0113] It should be understood that the reason why the cuttable area in the second image can be used to guide the generation of the motion trajectory is mainly because the second image not only contains the appearance information of the object, but also provides additional depth information and exact spatial position information. This information can be used to plan a precisely positioned motion path. Specifically, once the cuttable areas in the second image are identified, the three-dimensional coordinates of these areas in the real world can be directly obtained. The three-dimensional coordinates of the key points or representative points that can define the contour of the cutting path can be determined using the three-dimensional coordinates of the cuttable areas. In this way, each cutting point has clear X, Y, and Z coordinates. Based on these cutting points with spatial position information, motion planning can be performed. This involves how to effectively move the cutting tool to cover all cutting points, while considering factors such as speed, acceleration limits, and obstacle avoidance to ensure that the cutting process is both efficient and safe.

[0114] In a specific implementation, to generate a motion trajectory based on the spatial position information of the determined cuttable area to guide the cutting tool to operate, the following steps can be followed: extract the contour of the cuttable area and refine the boundary, and design one or more continuous, non-crossing, and efficient cutting paths according to the extracted contour. Among them, the cutting path can be planned by a path planning algorithm, which can be a greedy algorithm, an A* search algorithm, or other heuristic path planning methods, and the path planning algorithm can be used to minimize the cutting stroke and reduce the cutting time. The preliminary planned cutting path is smoothed to remove unnecessary sharp corners and fluctuations to ensure a smooth cutting process, which can be achieved by spline interpolation (such as Bézier curves, spline curves) or path smoothing algorithms. At the same time, the physical properties of the cutting tool (such as minimum turning radius, maximum acceleration, etc.) and working environment restrictions must also be considered to adjust the path to ensure practical feasibility. Finally, a motion instruction is generated to convert the optimized motion trajectory into a motion control instruction that can be understood by the cutting tool (such as a robot, a CNC machine tool, or a robotic arm carrying a cutting tool), which usually includes setting parameters such as cutting speed, feed rate, and tool direction.

[0115] In this embodiment, first, two images of the surface of the object to be cut are obtained, one carrying color information (first image) for identifying features such as historical cut marks; the other carrying spatial position information (second image) for precise positioning to ensure that the two images are accurately aligned at the pixel level. The first image is processed using a pre-trained cut mark recognition model, which can identify and mark the historical cut mark area on the surface of the object. According to the identified historical cut mark position, the area where the historical cut mark is located is correspondingly determined in the second image carrying spatial position information, and the remaining part is the cuttable area. Based on the determined cuttable area and its spatial position information, the motion trajectory of the cutting tool is generated. This trajectory avoids all historical cut mark areas to ensure a safe and effective cutting process. By accurately identifying historical cut marks and avoiding these areas, the accuracy of cutting is significantly improved. At the same time, automated trajectory planning reduces the need for manual intervention, speeds up the preparation process before cutting, and improves overall work efficiency.

[0116] Based on any of the above embodiments, the historical cut mark recognition model is as follows: Figure 2 Steps S21 to S23 shown are used for training:

[0117] Step S21: performing semantic segmentation on the training image using a semantic segmentation algorithm to determine a recognition area in the training image;

[0118] It should be noted that the training image is an image carrying color information that is aligned with the pixel position of the image carrying spatial position information. For example, the image carrying spatial position information is a point cloud image of the object surface, and the image carrying color information is an RGB image of the object surface. The RGB image is aligned with the point cloud image to obtain the aligned RGB image, which is the training image. The training image can be an image of the surface of any object, and can be one or more images.

[0119] Optionally, the training images are labeled with the category to which each pixel belongs, so that the algorithm can learn the association between each pixel-level feature and the category, thereby more accurately adjusting the model parameters during training to distinguish different types of regions.

[0120] Preferably, taking the rubber tree as an example, when the object to be cut is a rubber tree, the glue mark of the rubber tree is the above-mentioned cut mark, and the training images may include images taken during the day and at night. The training images may also include images with the following three forms of glue marks: images with clear glue marks, images with blurred or intermittent glue marks, and images without glue marks.

[0121] It should be understood that the semantic segmentation algorithm can be any of the following:

[0122] Fully Convolutional Networks (FCN): The fully connected layer is replaced by a convolutional layer, so that the network can accept input of any size and output a segmentation map of the same resolution. FCN introduces upsampling techniques (such as deconvolution or transposed convolution) to restore the resolution of the feature map for pixel-level classification.

[0123] U-Net: A classic encoder-decoder architecture, which is characterized by a U-shaped network structure. The encoder part downsamples to extract features, and the decoder part upsamples and combines the skip connection structure to restore details. It is very suitable for processing data with fine-grained structures.

[0124] SegNet: It is also an encoding-decoding network, similar to U-Net, but adopts a parameter-free upsampling strategy to restore the feature map by storing the maximum pooling index, thereby reducing the number of model parameters.

[0125] DeepLab series: including DeepLabv1 to DeepLabv3+, introduced technologies such as dilated convolution and conditional random field (CRF) to improve the segmentation accuracy and boundary accuracy.

[0126] PSPNet (Pyramid Scene Parsing Network): proposes a pyramid pooling module that can capture multi-scale contextual information and enhance the model's ability to recognize objects of different sizes.

[0127] Mask R-CNN: Mainly classified as an instance segmentation model, but can also be used for some semantic segmentation tasks, especially when it is necessary to distinguish different individuals in the same category. It extends Faster R-CNN and adds a pixel-level segmentation branch.

[0128] U-Net++: A variant of U-Net that enhances feature fusion by adding skip paths and dense connection strategies, improving segmentation accuracy and model generalization ability.

[0129] EfficientNet & EfficientDet: Although mainly designed for image classification and object detection, the efficient network structure of the EfficientNet series can also be adjusted for semantic segmentation tasks, such as EfficientSeg.

[0130] SegFormer: With MiT (Mix Transformer) as the backbone network, it combines the attention mechanism and lightweight design of Transformer to provide efficient and accurate semantic segmentation.

[0131] ENet (Encoders for Real-Time Semantic Segmentation): A lightweight real-time semantic segmentation network designed for resource-constrained environments that reduces computational burden through depth-wise separable convolutions and early downsampling.

[0132] Segment-Anything (SAM) algorithm: The main feature of SAM is its interactivity. Users can guide the model to focus on specific parts of the image by drawing a small number of prompt points or box selection areas. The model then generates high-quality segmentation masks for these areas. SAM is able to work without specific category training, which means that it does not perform segmentation based on predefined categories, but responds more flexibly to user input and is suitable for various segmentation tasks, including but not limited to semantic segmentation. SAM contains a pre-trained large visual model (such as ViT-Large) to generate image features, and a mask decoder for generating the final segmentation mask. Through manually specified semantic information (such as prompts entered by the user), it can achieve semantic understanding and segmentation of specific objects or regions in the image.

[0133] This embodiment and the following embodiments will take the Segment-Anything (SAM) algorithm as an example to illustrate the semantic segmentation algorithm.

[0134] Specifically, the identification area in the training image may include a cut area, an uncut area, and the like.

[0135] Step S22: Acquire semantic information of the identified area;

[0136] Exemplarily, the semantic information is cut marks and uncut surfaces.

[0137] Specifically, the semantic information can be input by the user or obtained through other deep learning models or annotation tools.

[0138] Step S23: Perform supervised training on the historical cut mark recognition model using the training image carrying the semantic information to obtain the trained historical cut mark recognition model.

[0139] It is understandable that in supervised learning, the model learns by analyzing the input (image) and the corresponding correct output (such as a label or category). In this embodiment, using these images annotated with semantic information as input, the model attempts to learn the mapping relationship from image pixels to predefined categories (such as different types of historical cuts). During the training process, the model will go through multiple iterations. In each cycle, the model will try to predict the label of the training image and adjust the internal parameters according to the difference between its prediction and the actual label (loss function). Through the back propagation algorithm, the model gradually optimizes its weights to reduce the prediction error and thus improve the recognition accuracy. After multiple rounds of training, when the performance of the model on the validation set is no longer significantly improved or reaches the preset performance indicators, the model is considered to have been fully trained. The model at this time is recorded as a "trained historical cut recognition model", which can be used to automatically detect and identify historical cuts in new and unseen images.

[0140] In this embodiment, a semantic segmentation algorithm is used to process training images, divide the images into multiple regions, and obtain semantic information corresponding to each region. The images carrying the above semantic information are used to train the historical cut mark recognition model. The model learns how to identify and classify historical cut marks from image data. After training, a trained historical cut mark recognition model is obtained, which can accurately identify whether there are historical cut marks in new images. By accurately dividing image regions through semantic segmentation and obtaining detailed semantic information, the model can learn more detailed features and significantly improve the accuracy and precision of historical cut mark recognition.

[0141] On the basis of any of the above embodiments, the method of determining the cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result includes:

[0142] According to the pixel position alignment relationship between the first image and the second image, projecting the historical cut mark area onto the second image to obtain spatial position information of the historical cut mark area;

[0143] Specifically, for each cut mark pixel point identified in the first image, its corresponding coordinate position in the second image is calculated according to the pixel alignment relationship between the two images. These converted coordinate points are projected on the second image, and the corresponding pixels or areas are marked to form a mapped historical cut mark area mask (or historical cut mark area mask). This mask (or mask) reflects the corresponding positions of all historical cut marks identified in the first image in the second image.

[0144] Based on the spatial position information of the historical cut mark area, the cuttable area in the second image is determined.

[0145] Specifically, a mask of a "cuttable area" is generated on the second image by using a mask of the historical cut mark area through image processing techniques such as logical operations. Specifically, a "logical negation" operation can be performed on the second image (i.e., the area not marked as a historical cut mark is retained) to obtain a mask that only displays the area where a cut operation can be performed.

[0146] Preferably, the generated cuttable area mask can be further processed, including removing small isolated areas, smoothing edges, or adjusting the cuttable area according to the physical limitations of the actual cutting tool (such as the minimum turning radius) to ensure that the subsequent cutting path planned according to the cuttable area is feasible and efficient.

[0147] In this embodiment, the pixel alignment relationship between the first image and the second image is used to project the known historical cut marks into the second image, thereby obtaining the relative positions of these cut marks in the actual space; then, based on these spatial position information, the area in the second image suitable for cutting operation, i.e., the cuttable area, is identified and defined. The combination of image processing and spatial analysis technology significantly improves the accuracy of locating and planning the cuttable area.

[0148] Based on any of the above embodiments, the second image is a point cloud image; the spatial position information includes point cloud information; and determining the cuttable area in the second image based on the spatial position information of the historical cut mark area includes:

[0149] Preprocessing the point cloud information of the historical cut mark area in the second image; the preprocessing includes filtering and / or cut mark surface clustering;

[0150] It should be noted that since point cloud data may contain noise, filtering is one of the necessary preprocessing steps. It can include downsampling (removing overly dense points, reducing the amount of data, and improving the efficiency of subsequent processing) and smoothing filtering (reducing random noise in point cloud data, making the surface look smoother, and facilitating subsequent feature extraction and analysis). Exemplarily, downsampling and smoothing filtering can be: voxel grid filtering, statistical filtering (such as mean filtering, median filtering), and surface fitting filtering.

[0151] The step of cutting surface clustering aims to identify and separate the point set representing the historical cutting marks from the point cloud. Clustering algorithms can group points in the point cloud based on distance, density or other features. For example, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a widely used density-based clustering algorithm that can effectively identify the compact point cloud clusters formed by the cutting area while excluding noise points and outliers. Through clustering, it is possible to clearly distinguish which points belong to the historical cutting area and which belong to the cuttable area.

[0152] Specifically, the preprocessing includes downsampling filtering, global discrete filtering, and cut surface clustering. Specifically, firstly, the point cloud information of the historical cuts is simplified by voxelization to generate a simplified point cloud composed of representative points (representative points are, for example, the centroid of all points in each non-empty voxel, the point closest to the centroid of the voxel in each voxel, randomly selected representative points in each voxel containing multiple points, and representative points selected according to certain importance indicators, such as curvature, color difference, etc.); secondly, the isolated points and abnormal points in the simplified point cloud are removed to obtain a point cloud data set; finally, a cluster analysis of the historical cuts is performed based on the point cloud data set to obtain the point cloud information of the processed historical cuts, and the preprocessing is completed.

[0153] The cuttable area in the second image is determined based on the preprocessed point cloud information.

[0154] Specifically, each point cloud cluster after clustering processing is analyzed to identify the areas that are directly related to the historical cutting marks and suitable for cutting.

[0155] In this embodiment, the point cloud data of the historical cutting mark area in the second image is first preprocessed to facilitate subsequent analysis and distinction. After the preprocessing is completed, the processed point cloud information is used to accurately locate and divide the cuttable area in the second image. On the one hand, the automated and precise point cloud processing technology significantly speeds up the preliminary data analysis and makes the entire cutting preparation process more efficient. On the other hand, for the subsequent cutting path planning, the cutting path determined based on high-quality point cloud data can adapt to more complex cutting requirements.

[0156] Based on any of the above embodiments, the step of determining the cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result includes: Figure 3 Steps S31-S32 shown:

[0157] Step S31: Based on the historical cut mark area in the first image, an initial position area is determined in the first image; the initial position area is an area excluding the historical cut mark area;

[0158] It should be noted that the initial position area is consistent with the above-mentioned cuttable area range.

[0159] Step S32: According to the pixel position alignment relationship between the first image and the second image, the initial position area is projected into the second image to obtain the cuttable area in the second image.

[0160] It is understandable that projecting the initial position area in the first image into the second image using the pixel position alignment relationship means mapping the identified initial position area onto an image containing spatial position information, thereby associating visual features with spatial coordinates.

[0161] In this embodiment, based on the historical cut mark area in the first image, the area other than the historical cut mark area in the first image, i.e., the initial position area, is determined, and then the area corresponding to the initial position area in the second image is obtained through projection, which is defined as the cuttable area. The initial position area is first determined in the first image carrying color information, and then the cuttable area in the second image is determined through projection, which improves the speed of determining the cuttable area.

[0162] Based on any of the above embodiments, the historical cutting mark recognition result indicates that the surface of the object to be cut does not include the historical cutting mark; the method further includes:

[0163] The second image is determined as the cuttable area.

[0164] It should be noted that the surface of the object to be cut includes two situations: with cut marks and without cut marks. This embodiment describes how to complete automatic cutting in the situation without cut marks.

[0165] In a specific implementation, if the historical cut mark recognition result indicates that the surface of the object to be cut does not include any historical cut marks, the entire surface of the object to be cut represented by the second image is used as the cuttable area, and a motion trajectory is generated based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the object to be cut along the motion trajectory.

[0166] In this embodiment, a method for completing automatic cutting when there are no cut marks on the surface of the object to be cut is provided, which realizes personalized adjustment according to different states of the object to be cut and improves the adaptability of automatic cutting.

[0167] Based on any of the above embodiments, the first image includes an RGB image; the second image includes a point cloud image; the point cloud image is generated by converting a depth image; the RGB image and the depth image are acquired by an RGB-D camera at the same time, and the pixel positions of the first image and the second image are aligned based on the depth image.

[0168] It should be noted that the RGB-D camera can simultaneously obtain the RGB image of the surface of the object to be cut and the corresponding depth image and the point cloud image converted from the depth image. The RGB image and the depth image are aligned through the D2C hardware solution function of the RGB-D camera. It can be seen that the depth image and the point cloud image have a one-to-one correspondence between pixels, so accurate registration from the RGB image to the point cloud image can be achieved.

[0169] In this embodiment, an RGB-D camera is used to synchronously capture the RGB image and depth image of the surface of the object to be cut. Based on the captured depth image, it is converted into a point cloud image through a corresponding algorithm. Using the depth image as a bridge, pixel-level alignment between the RGB image and the point cloud image is achieved. Combining the color information of the RGB image and the spatial structure information of the point cloud image, the target area can be identified more accurately, providing detailed three-dimensional spatial guidance for subsequent cutting path planning and reducing errors. At the same time, through pixel-level image alignment technology, accurate data correspondence is ensured, making planning more efficient.

[0170] In addition, the present application also provides another automatic cutting method, which is applied to rubber tree tapping, such as Figure 4 As shown, Figure 4 The schematic flow diagram of another kind of automatic cutting method provided for the present embodiment. The specific implementation process of the automatic cutting method applied to rubber tree tapping is as follows:

[0171] 1. The robotic arm carries the RGBD camera to a location close to the cut marks on the rubber tree.

[0172] 2. Obtaining RGB image and depth image information of rubber tree cuts: The image information obtained by the RGBD camera includes the RGB image of the rubber tree surface and the corresponding depth image and the point cloud image converted from the depth image. The RGB image provides color information, while the depth image and point cloud image provide the distance information of each pixel from the camera.

[0173] 3. Alignment of RGB image and point cloud image: Align the RGB image and the depth image through D2C hardware solution. This process ensures that the pixels in the RGB image and the corresponding pixels in the depth image have the same spatial position. Based on the one-to-one correspondence between the depth image and the point cloud image, accurate registration from the RGB image to the point cloud image can be achieved.

[0174] 4. Image segmentation and manual labeling of glue marks: On the obtained RGB image, the SegmentAnything algorithm is used to perform semantic segmentation and manually determine semantic information, such as glue marks, cut surfaces, uncut surfaces, etc. The rubber tree glue marks are labeled on the RGB image. This step is essentially object recognition and segmentation of the image, which helps the labeler quickly locate and identify the area where the glue marks are located.

[0175] 5. Neural network training: The annotated glue mark information is passed into the mmdetection-based neural network for training to build a model that can identify glue marks. Through neural network training, the model can learn the characteristics of glue marks, improving the accuracy and robustness of glue mark recognition.

[0176] 6. Real-time reasoning: After the model is trained, in actual applications, proceed to step 3, obtain the registered image of the RGB image and the point cloud image, and use the trained model in step 5 to perform reasoning to obtain the corresponding relationship between the actual glue mark in the point cloud image, so as to obtain the point cloud information of the glue mark.

[0177] 7. Processing of glue mark point cloud: Since the obtained glue mark point cloud information is relatively messy, it is first necessary to perform a series of point cloud processing such as downsampling filtering, global discrete filtering, and cutting mark surface clustering to obtain the spatial position information required for rubber tree tapping, that is, the spatial position information of the cutting area.

[0178] 8. Robotic arm trajectory planning: Based on the rubber tapping spatial position information obtained in step 7, plan the motion trajectory of the robotic arm carrying the rubber tapping tools. In this process, avoid the problem of multiple solutions and singular points of the robotic arm.

[0179] 9. The robotic arm performs rubber tapping: The robotic arm drives the rubber tapping tools to complete automatic rubber tapping according to the motion trajectory.

[0180] The automatic cutting method provided in this embodiment can be applied to rubber tree tapping operations to automate the tapping process, thereby improving tapping efficiency and accuracy while enhancing intelligence.

[0181] like Figure 5 As shown, Figure 5 A schematic diagram of a structure of a device 500 provided in an embodiment of the present application, the device comprising:

[0182] The acquisition module 501 is used to acquire a first image carrying color information and a second image carrying spatial position information of the surface of the object to be cut; the pixel positions of the first image and the second image are aligned;

[0183] A processing module 502 is used to input the first image into a trained historical cut mark recognition model to obtain a historical cut mark recognition result;

[0184] The processing module 502 is further configured to determine a cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result; the cuttable area is an area on the surface of the object to be cut excluding the historical cut mark area;

[0185] The operation module 503 is used to generate a motion trajectory based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the surface of the object to be cut along the motion trajectory.

[0186] Optionally, the above device further includes:

[0187] A first processing module, configured to perform semantic segmentation on the training image using a semantic segmentation algorithm to determine a recognition area in the training image;

[0188] A first acquisition module, used to acquire semantic information of the identified area;

[0189] The training module is used to perform supervised training on the historical cut mark recognition model using the training image carrying the semantic information to obtain the trained historical cut mark recognition model.

[0190] Optionally, the above device further includes:

[0191] A projection module, configured to project the historical cut mark area onto the second image according to the pixel position alignment relationship between the first image and the second image, so as to obtain spatial position information of the historical cut mark area;

[0192] The second processing module is used to determine the cuttable area in the second image based on the spatial position information of the historical cut mark area.

[0193] Optionally, the above device further includes:

[0194] A preprocessing module, used for preprocessing the point cloud information of the historical cut mark area in the second image; the preprocessing includes filtering processing and / or cut mark surface clustering;

[0195] The third processing module is used to determine the cuttable area in the second image based on the preprocessed point cloud information.

[0196] Optionally, the above device further includes:

[0197] A fourth processing module, configured to determine an initial position region in the first image based on the historical cut mark region in the first image; the initial position region is a region excluding the historical cut mark region;

[0198] The first projection module is used to project the initial position area into the second image according to the pixel position alignment relationship between the first image and the second image to obtain the cuttable area in the second image.

[0199] Optionally, the above device further includes:

[0200] A fifth processing module is used to determine the second image as the cuttable area.

[0201] The device provided in the embodiment of the present application obtains two images of the surface of the object to be cut, one of which carries color information (first image) for identifying features such as historical cut marks; the other carries spatial position information (second image) for precise positioning, while ensuring that the two images are accurately aligned at the pixel level. The first image is processed using a pre-trained cut mark recognition model, which can identify and mark the historical cut mark area on the surface of the object. According to the identified historical cut mark location, the area where the historical cut mark is located is correspondingly determined in the second image carrying spatial position information, and the remaining part is the cuttable area. Based on the determined cuttable area and its spatial position information, the motion trajectory of the cutting tool is generated, which significantly improves the cutting accuracy. At the same time, the automated trajectory planning reduces the need for manual intervention, speeds up the preparation process before cutting, and improves overall work efficiency.

[0202] Based on the method described in any of the above embodiments, the present application also provides Figure 6 A schematic diagram of the structure of an electronic device is shown in FIG. Figure 6 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the method described in any of the above embodiments.

[0203] Based on the method described in any of the above embodiments, the present application also provides a computer program product, which includes one or more computer programs or instructions. The computer program or instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. When the computer program is executed by a processor, the method described in any of the above embodiments is implemented.

[0204] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0205] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0206] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0207] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0208] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0209] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An automatic cutting method, It is characterized in that The method comprises: Acquire a first image carrying color information and a second image carrying spatial position information of the surface of the object to be cut; align pixel positions of the first image and the second image; Inputting the first image into a trained historical cut mark recognition model to obtain a historical cut mark recognition result; the historical cut mark recognition result indicates that there are historical cut marks on the surface of the object to be cut; Determine, in the second image, a cuttable area on the surface of the object to be cut according to the historical cut mark area in the first image indicated by the historical cut mark recognition result; the cuttable area is an area on the surface of the object to be cut excluding the historical cut mark area; A motion trajectory is generated based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the object to be cut along the motion trajectory.

2. The method according to claim 1, It is characterized in that The historical cut mark recognition model is trained by the following steps: Performing semantic segmentation on the training image using a semantic segmentation algorithm to determine a recognition area in the training image; Acquiring semantic information of the identified area; The historical cut mark recognition model is supervisedly trained using the training image carrying the semantic information to obtain the trained historical cut mark recognition model.

3. The method according to claim 1, It is characterized in that The step of determining the cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result comprises: According to the pixel position alignment relationship between the first image and the second image, projecting the historical cut mark area onto the second image to obtain spatial position information of the historical cut mark area; Based on the spatial position information of the historical cut mark area, the cuttable area in the second image is determined.

4. The method according to claim 3, It is characterized in that The second image is a point cloud image; the spatial position information includes point cloud information; and determining the cuttable area in the second image based on the spatial position information of the historical cut mark area includes: Preprocessing the point cloud information of the historical cut mark area in the second image; the preprocessing includes filtering and / or cut mark surface clustering; The cuttable area in the second image is determined based on the preprocessed point cloud information.

5. The method according to claim 1, It is characterized in that The step of determining the cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result comprises: Based on the historical cut mark area in the first image, an initial position area is determined in the first image; the initial position area is an area excluding the historical cut mark area; According to the pixel position alignment relationship between the first image and the second image, the initial position area is projected into the second image to obtain the cuttable area in the second image.

6. The method according to claim 1, It is characterized in that The historical cutting mark recognition result indicates that the surface of the object to be cut does not include the historical cutting mark; the method further includes: The second image is determined as the cuttable area.

7. The method according to claim 1, It is characterized in that The first image includes an RGB image; the second image includes a point cloud image; the point cloud image is generated by converting a depth image; the RGB image and the depth image are acquired by an RGB-D camera at the same time, and the pixel positions of the first image and the second image are aligned based on the depth image.

8. A device, It is characterized in that The device comprises: An acquisition module, used to acquire a first image carrying color information and a second image carrying spatial position information of the surface of the object to be cut; the pixel positions of the first image and the second image are aligned; A processing module, used for inputting the first image into a trained historical cut mark recognition model to obtain a historical cut mark recognition result; The processing module is further used to determine a cuttable area on the surface of the object to be cut in the second image according to the historical cut mark area in the first image indicated by the historical cut mark recognition result; the cuttable area is an area on the surface of the object to be cut excluding the historical cut mark area; The operation module is used to generate a motion trajectory based on the spatial position information carried by the cuttable area, so that the cutting tool cuts the surface of the object to be cut along the motion trajectory.

9. An electronic device, It is characterized in that The electronic device comprises: processor; a memory for storing processor-executable instructions; Wherein, when the processor calls the executable instruction, the method described in any one of claims 1-7 is implemented.

10. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Systems and methods for automatically verifying correct die removal from film frames

    CN105408990A

  • Automatic cerebrospinal fluid cell analysis system based on artificial intelligence

    CN112595655A

  • Automatic semiconductor partitioning method based on panoramic segmentation

    CN113902765A

  • Apparatus and method for detecting hot rolled bar

    KR1020010060947A

Cited By

  • Laser cutting machine track searching path planning optimization method and system

    CN119347779A

  • Laser cutting machine track-seeking path planning optimization method and system

    CN119347779B

  • High-precision plasma cutting method and system in radioactive environment

    CN119566484A

  • Three-axis positioning rubber tapping control method and three-axis positioning rubber tapping device

    CN120077921A

  • Three-axis positioning rubber tapping control method and three-axis positioning rubber tapping device

    CN120077921B