Chinese segmentation method, apparatus, device, and medium

Abstract

The application discloses a middle square segmentation method, device and equipment and medium, and relates to the technical field of food processing. The method comprises the following steps: acquiring an original image of a middle square to be segmented by using a preset camera, and obtaining a target semantic segmentation graph of the original image; planning a segmentation track based on the target semantic segmentation graph, and determining depth information of the segmentation track, so that a mechanical arm controls a cutting tool to segment the middle square to be segmented according to the segmentation track and the depth information. According to the above scheme, only the original image of the middle square to be segmented is acquired by using the preset camera to obtain the target semantic segmentation graph, and it is not necessary to acquire redundant information of the middle square to be cut, so that the efficiency of the middle square segmentation is improved, and the segmentation complexity is reduced. The segmentation track and the depth information are planned, so that the mechanical arm can control the cutting tool to perform more accurate segmentation according to the segmentation track and the depth information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of food processing, in particular to a middle side cutting method, device, equipment and medium. BACKGROUND

[0002] In the fine cutting process of pork, the six-segmented body of the split pig carcass needs to be cut first. The middle segment in the six-segmented body will be cut into a big side with skin and spine (No. 3 meat) and a middle side according to customer demand. The middle side will be cut into rib and five flower parts according to customer demand. The rib part includes thoracic rib, rib cartilage, sternum, and 9-11 rib bones in total. The protruding sternum part needs to be trimmed. The rib side with meat thickness should not be exposed to bone (i.e. the phenomenon that the rib side is cut by the cutter to expose the bone during the cutting process), the Luo gap meat strip is less than 0.5 cm, the rib side skirt width is controlled within 1-5 cm according to customer order demand, the rib cartilage with meat thickness is 1.5-2 cm, there is no obvious broken plate oil on the surface and inner cavity, there is no hard bone, periosteum, broken plate oil, lymph residue, etc. on the surface of the five flower part.

[0003] The current domestic slaughter plant cuts the middle side with skin and bone manually, which has low cutting efficiency and requires high experience of the operator. In addition, the breed, size and quality of the pig half carcass need to be obtained to determine the corresponding grade, so as to plan the cutting path, which has low cutting efficiency and complicated process.

[0004] In summary, how to improve the efficiency of middle side cutting and reduce the complexity of cutting is a problem to be solved in the field. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a middle side cutting method, device, equipment and medium to improve the efficiency of middle side cutting and reduce the complexity of cutting. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a middle side cutting method, comprising:

[0007] An original image of a middle side to be cut is collected by a preset camera, and a target semantic segmentation map of the original image is obtained;

[0008] A cutting trajectory is planned based on the target semantic segmentation map, and depth information of the cutting trajectory is determined, so that a mechanical arm controls a cutting tool to cut the middle side to be cut according to the cutting trajectory and the depth information.

[0009] Optionally, the segmentation trajectory is planned based on the target semantic segmentation map, and depth information of the segmentation trajectory is determined, so that the robot arm controls the cutting tool to segment the to-be-segmented medium according to the segmentation trajectory and the depth information, comprising:

[0010] The segmentation trajectory is planned based on the target semantic segmentation map, and depth information of the segmentation trajectory is determined;

[0011] The first control command corresponding to the segmentation trajectory and the depth information is generated, and the first control command is sent to the robot arm, so that the robot arm performs coordinate system conversion on the first control command to obtain a second control command, and controls the cutting tool to segment the to-be-segmented medium according to the second control command.

[0012] Optionally, the robot arm performs coordinate system conversion on the first control command to obtain a second control command, comprising:

[0013] The coordinate system conversion relationship between the first coordinate system of the robot arm and the second coordinate system of the cutting tool is determined;

[0014] The first control command is converted in the coordinate system by using the coordinate system conversion relationship to obtain a second control command.

[0015] Optionally, the segmentation trajectory is planned based on the target semantic segmentation map, comprising:

[0016] The target segmentation area is determined based on the target semantic segmentation map, and a plurality of first feature points on the boundary overlapping the target segmentation area and the target semantic segmentation map are determined;

[0017] The arc line between the corresponding first feature points is planned, and a plurality of second feature points on the arc line are determined, so that the corresponding segmentation trajectory is obtained based on the first feature points and the second feature points.

[0018] Optionally, the robot arm controls the cutting tool to segment the to-be-segmented medium according to the segmentation trajectory and the depth information, comprising:

[0019] The robot arm controls the cutting tool to segment the to-be-segmented medium according to the tangent direction of the first feature point, the tangent direction of the second feature point, and the depth information.

[0020] Optionally, the robot arm controls the cutting tool to segment the to-be-segmented medium according to the tangent direction of the first feature point, the tangent direction of the second feature point, and the depth information, comprising:

[0021] The mechanical arm adjusts an included angle between a cutting edge surface of a cutting tool and a cutting surface of the to-be-cut medium to be a preset acute angle, and then controls the cutting tool to cut the to-be-cut medium according to a tangent direction of the first feature point, a tangent direction of the second feature point and the depth information.

[0022] Optionally, the target semantic segmentation map of the original image comprises:

[0023] The original semantic segmentation map of the original image is obtained by using a semantic segmentation model based on a DeepLabV3 algorithm.

[0024] The original semantic segmentation map is subjected to noise removal operation and slicing operation to obtain the target semantic segmentation map.

[0025] In a second aspect, the present application discloses a medium cutting device, and has the characteristics that the medium cutting device comprises:

[0026] A semantic segmentation map acquisition module is configured to acquire an original image of a to-be-cut medium by using a preset camera, and acquire a target semantic segmentation map of the original image.

[0027] A cutting module is configured to plan a cutting track based on the target semantic segmentation map, determine depth information of the cutting track, and control a mechanical arm to cut the to-be-cut medium according to the cutting track and the depth information.

[0028] In a third aspect, the present application discloses an electronic device, and has the characteristics that the electronic device comprises:

[0029] A memory is configured to save a computer program.

[0030] A processor is configured to execute the computer program to realize the steps of the medium cutting method disclosed above.

[0031] In a fourth aspect, the present application discloses a computer readable storage medium configured to store a computer program, wherein the computer program is executed by a processor to realize the steps of the medium cutting method disclosed above.

[0032] The application has the beneficial effects that: the preset camera is used to collect an original image of a to-be-segmented middle piece and obtain a target semantic segmentation image of the original image; a segmentation trajectory is planned based on the target semantic segmentation image, and depth information of the segmentation trajectory is determined, so that a cutting tool controlled by a mechanical arm performs segmentation on the to-be-segmented middle piece according to the segmentation trajectory and the depth information. As can be seen, the application only needs to collect an original image of a to-be-segmented middle piece by using a preset camera to obtain a target semantic segmentation image, without collecting redundant information of the to-be-cut middle piece, thereby improving the efficiency of middle piece segmentation and reducing the complexity of segmentation; the segmentation trajectory and its depth information are planned, so that the mechanical arm can control the cutting tool to perform more accurate segmentation according to the segmentation trajectory and the depth information. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0034] Figure 1 A middle piece segmentation method flow chart disclosed by the present application;

[0035] Figure 2 A specific cutting system schematic diagram disclosed by the present application;

[0036] Figure 3 A specific segmentation trajectory plane schematic diagram disclosed by the present application;

[0037] Figure 4 A specific segmentation trajectory front side schematic diagram disclosed by the present application;

[0038] Figure 5 A specific segmentation trajectory right side schematic diagram disclosed by the present application;

[0039] Figure 6 A specific coordinate system schematic diagram disclosed by the present application;

[0040] Figure 7 A specific tool rotation schematic diagram disclosed by the present application;

[0041] Figure 8 A specific middle piece segmentation method flow chart disclosed by the present application;

[0042] Figure 9 A specific DeepLabV3 semantic segmentation model schematic diagram disclosed by the present application;

[0043] Figure 10A specific original semantic segmentation map disclosed by the present application;

[0044] Figure 11 A specific semantic segmentation result map disclosed by the present application;

[0045] Figure 12 A specific target semantic segmentation map disclosed by the present application;

[0046] Figure 13 A specific feature point planning map disclosed by the present application;

[0047] Figure 14 A specific cutting plane schematic diagram disclosed by the present application;

[0048] Figure 15 Another specific Chinese segmentation method flow chart disclosed by the present application;

[0049] Figure 16 A specific segmentation trajectory schematic diagram disclosed by the present application;

[0050] Figure 17 A Chinese segmentation device structure schematic diagram disclosed by the present application;

[0051] Figure 18 An electronic device structure diagram disclosed by the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0053] In the fine segmentation process of pork, the first step is to segment the six-segment of the split pig carcass. According to customer demand, the middle segment in the six-segment will be segmented into skin-on and bone-in loin (No. 3 meat) and middle side. The middle side will be segmented into rib and five flower parts according to customer demand. The rib part includes rib bones, rib cartilage, sternum cartilage, and 9-11 rib bones in total. The protruding sternum cartilage part needs to be trimmed. The rib meat thickness should not be exposed to bone (i.e., the phenomenon of rib being cut by the knife to expose the bone during cutting), the rib gap meat strip is less than 0.5 cm, the rib skirt edge width is controlled within 1-5 cm according to customer order demand, the rib cartilage meat thickness is 1.5-2 cm, and there is no obvious broken plate oil on the surface and inner cavity. The five flower part has no serious white exposure (i.e., the phenomenon of the five flower part exposing too much fat part during cutting, with an area of no more than 3 cm x 3 cm), no hard bone, periosteum, broken plate oil, and lymph residue.

[0054] Currently, the segmentation of skin-on and bone-in middle side in domestic slaughter plants is performed manually, which has low segmentation efficiency and requires high experience of the operator. Alternatively, the breed, size, and quality of the pig half carcass need to be obtained to determine the corresponding grade, so as to plan the cutting path, which has low segmentation efficiency and complicated process.

[0055] Therefore, the present application provides a middle side segmentation scheme to improve the segmentation efficiency of the middle side and reduce the segmentation complexity.

[0056] Referring to Figure 1 The embodiment of the present application discloses a middle side segmentation method, which comprises:

[0057] Step S11: acquiring the original image of the middle side to be segmented by using a preset camera, and obtaining the target semantic segmentation map of the original image.

[0058] In this embodiment, for example Figure 2 As shown in a specific cutting system schematic diagram, an operation table, a mechanical arm, a cutting tool, and a preset camera are installed, and the middle side to be cut is placed in the operation table. The original image of the middle side to be cut is acquired by using the preset camera, and the target semantic segmentation map of the original image is obtained.

[0059] Step S12: planning a segmentation trajectory based on the target semantic segmentation map, and determining the depth information of the segmentation trajectory, so that the mechanical arm controls the cutting tool to segment the middle side to be segmented according to the segmentation trajectory and the depth information.

[0060] Next, taking the plane, the front side, and the right side as examples, the segmentation is described as follows:

[0061] I. For example Figure 3 As shown in a specific segmentation trajectory plane schematic diagram,

[0062] (1)Point QADP in the figure represents the area of the middle side to be cut, and the area surrounded by point OAD is the area of the ribbed side to be separated from the middle side;

[0063] (2) E is a point on the blade, and the straight lines FF' and GG' are parallel, representing the position of the blade cutting meat, E is the position of the blade tip, and the area surrounded by points EF' FG G' is the cutting blade, and the direction indicated by the arrow is the cutting direction of the blade;

[0064] (3) Points H and M are respectively located at the intersection points of the fat and red meat on the right side of the middle side and the right side, and the area OHMD surrounded by points HM and OD is the area of the ribbed side skirt;

[0065] (4) Lines OB, OC, OD, etc. represent the trajectory lines of the blade when the blade cuts the ribbed side along the cutting direction of the blade, which are arc-shaped, and are the simplified and re-planned trajectory lines of the blade tip during manual cutting of the middle side. The number of trajectory lines and the positional relationship between the trajectory lines are determined by the middle side and the cutting trajectory planning. Here, the blade cuts 4 times as an example;

[0066] According to the cutting direction of the blade, the area swept by the blade is the area surrounded by point OAD. Due to the different sizes of the middle side to be cut and the ribbed side, and different product requirements, the size of the area surrounded by point OAD is also different, and the specific position is determined by the width of the skirt edge.

[0067] II. For example Figure 4 a specific cutting trajectory is shown in a specific cutting trajectory right side view,

[0068] (1) Points O and A are known, and points A' and O' are respectively points O and A vertically downward a certain distance (the actual position is inside the middle side); the arrow direction is the cutting direction of the blade, and the cutting position and blade position are shown in Figure 4 ;

[0069] (2) Point A' is the contact surface of the blade tip cutting meat, and O'A' is the trajectory line of the blade right side cutting, and the length of AA' is greater than the length of OO';

[0070] (3) O'A'AO is the range of the ribbed side area to be separated. The blade tip position cuts along the cutting direction of the blade, starting at point A' along the trajectory line of A'O', and ending at point O. The blade tip position needs to cover the trajectory line range composed of points A'O'O, so as to separate the required ribbed side on the right side. The lengths of AA', OO', etc. represent the depth of the ribbed side to be cut on the right side, and the standard is: the ribbed side does not expose the bone, and the white part is exposed.

[0071] III. For exampleFigure 5 One specific segmentation trajectory right side view is shown,

[0072] (1) Points O, A, D, A' are known, and point D' is a point vertically downward from point D by a certain distance (the actual position is inside the middle square); under this view, the tool cutting direction is represented by the direction of line AO according to the actual object diagram;

[0073] (2) Point A' is the contact surface of the tool edge cutting meat, and A'D' is the trajectory line of the right side surface cutting of the tool edge;

[0074] (3) AA'DD' is the right side surface area range of the separated rib row. The tool edge position is along the tool cutting direction, and the tool edge part needs to cover the trajectory line range composed of points A'D' by starting cutting at point A' along the A'D' trajectory line and ending at point D', so as to separate the required rib row in the right side surface part. The lengths of the line segments AA', DD', etc. represent the depth of the rib row that needs to be cut in the right side surface, and the standard is that the rib row does not expose the bone, exposes white, and the five-color part does not expose white.

[0075] From the above Figures 3 to 5 It can be seen that the segmentation trajectory can be simply divided into horizontal cutting and vertical cutting, such as segmentation trajectory OB, segmentation trajectory OC, and segmentation trajectory OD, and AA', OO' represent the corresponding depth information. The vertical cutting segmentation trajectory is between OD and HM. According to different cutting requirements, the specific vertical cutting trajectory is determined, such as OD. After horizontal cutting and vertical cutting, the rib row can be segmented, that is, the segmentation of the middle square is completed.

[0076] In this embodiment, the segmentation trajectory is planned based on the target semantic segmentation map, and the depth information of the segmentation trajectory is determined, so that the mechanical arm controls the cutting tool to segment the middle square to be segmented according to the segmentation trajectory and the depth information, which includes: planning a segmentation trajectory based on the target semantic segmentation map, and determining the depth information of the segmentation trajectory; generating a first control command corresponding to the segmentation trajectory and the depth information, and sending the first control command to the mechanical arm, so that the mechanical arm performs coordinate system conversion on the first control command to obtain a second control command, and controls the cutting tool to segment the middle square to be segmented according to the second control command. It can be understood that the mechanical arm needs to obtain the cutting command of how to cut, that is, the first control command. Because the coordinate systems of the mechanical arm and the cutting tool are not completely consistent, the first control command needs to be converted to obtain the second control command for controlling the cutting tool to segment the middle square to be segmented according to the segmentation trajectory and the depth information.

[0077] In the embodiment, the mechanical arm performs coordinate system conversion on the first control command to obtain a second control command, including: determining a coordinate system conversion relationship between a first coordinate system of the mechanical arm and a second coordinate system of the cutting tool; and performing coordinate system conversion on the first control command by using the coordinate system conversion relationship to obtain a second control command.

[0078] It can be understood that the embodiment is based on the mechanical arm and the cutting tool to complete the cutting function, that is, a coordinate system needs to be determined as a world coordinate system first to specify the actual position of the planning point. Before this, Euler angles and rotation angles in the mechanical arm coordinate system need to be introduced, which are represented by OD, A'O'O, A'D' and corresponding RZ, RX and RY to represent the specific posture of the tool at a certain trajectory or point. In addition, according to the tool position (workpiece position), a tool coordinate system is established with the plane surrounded by the tool EE'FG as the XOY plane to plan the posture of the tool alone, such as a specific coordinate system diagram shown in Figure 6

[0079] (1) Tool Euler angles: Euler angles are a mechanism for creating general rotation based on three relatively simple rotational motions (called pitch, roll and yaw). For example, a specific tool rotation diagram shown in Figure 7

[0080] (2) Mechanical arm rotation angle: the embodiment adopts a 6-axis mechanical arm, in the mechanical arm world coordinate system, according to the angles of rotation around the X, Y and Z axes by a certain number of degrees, defined as RX, RY and RZ, for example, to represent the tool posture.

[0081] Matching of tool coordinate system and mechanical arm coordinate system: in actual situations, in order to make the pitch, yaw and roll angles of the tool correspond to RY, RZ and RX in the mechanical arm coordinate system one by one, and the pitch, yaw and roll angles of the tool are represented by RY, RZ and RX respectively, the calculation results of RY, RZ and RX in the world coordinate system will also represent the pitch, yaw and roll angles of the tool. The initial posture of the tool and the world coordinate system under the tool coordinate system are needed to ensure that the calculated posture value and the actual posture are basically consistent, and to ensure that the conveying belt (operation table) plane is parallel or coincides with the XOY plane in the mechanical arm world coordinate system at this time:

[0082] ​​Operation (1) Matching of X, Y, and Z axes: First, determine that when the robot arm is assembled, its world coordinate system is perpendicular to the horizontal plane. At this time, the Z axis of the tool and the robot arm are unified. Make the cutting edge (FF' or GG') parallel or coincident with the X axis of the robot arm coordinate system (in this embodiment, the cutting edge FF' or GG' is parallel to the X axis of the robot arm world coordinate system). At this time, the X and Y axes of the tool can be matched with the X and Y axes of the robot arm, and the RZ matching of the tool and the robot arm coordinate system is completed, with a value of 0.

[0083] Operation (2) Matching Euler angles and pitch angles: Based on "Operation (1)", place the level on the tool surface and adjust RX and RY to 0. This will match the RX and RY of the tool and the robot arm coordinate system (and the RX and RY values ​​will be 0).

[0084] In summary, this design completes the setting and matching of the coordinate systems and corresponding angles of the tool and the robotic arm.

[0085] The beneficial effects of this application are as follows: It utilizes a preset camera to acquire the original image of the Chinese side to be segmented and obtains a target semantic segmentation map of the original image; based on the target semantic segmentation map, it plans a segmentation trajectory and determines the depth information of the segmentation trajectory, so that the robotic arm can control the cutting tool to segment the Chinese side according to the segmentation trajectory and the depth information. Therefore, this application only needs to use a preset camera to acquire the original image of the Chinese side to be segmented to obtain the target semantic segmentation map, without needing to acquire redundant information about the Chinese side to be segmented, thus improving the efficiency of Chinese side segmentation and reducing segmentation complexity; by planning the segmentation trajectory and its depth information, the robotic arm can control the cutting tool to perform more precise segmentation according to the segmentation trajectory and depth information.

[0086] See Figure 8 As shown in the figure, this application discloses a specific Chinese segmentation method, including:

[0087] Step S21: Use a preset camera to acquire the original image of the Chinese side to be segmented.

[0088] Step S22: Obtain the original semantic segmentation map of the original image using a semantic segmentation model based on the DeepLabV3 algorithm; perform noise removal and slicing operations on the original semantic segmentation map to obtain the target semantic segmentation map.

[0089] In this embodiment, for subsequent trajectory segmentation, semantic segmentation is first used to divide the rib cage into the thoracic cavity region and the skirt edge region, for example... Figure 9 The diagram illustrates a specific DeepLabV3 semantic segmentation model. Using a DeepLabV3-based semantic segmentation model, the original image is input into the model, and the output is, for example... Figure 10The image shown is a specific original semantic segmentation graph. Figure 11 For example, to provide a specific semantic segmentation result image, in order to plan trajectories more clearly, Figure 12 The example shown is a specific target semantic segmentation map, which can then undergo noise removal and slicing operations.

[0090] Step S23: Plan a segmentation trajectory based on the target semantic segmentation map and determine the depth information of the segmentation trajectory so that the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the segmentation trajectory and the depth information.

[0091] In this embodiment, planning the segmentation trajectory based on the target semantic segmentation map includes: determining a target segmentation region based on the target semantic segmentation map, and determining a plurality of first feature points on the boundary where the target segmentation region overlaps with the target semantic segmentation map; planning an arc between the corresponding first feature points, and determining a plurality of second feature points on the arc, so as to obtain the corresponding segmentation trajectory based on the first feature points and the second feature points. It is understood that a preset number of feature points need to be obtained before planning the segmentation trajectory, for example... Figure 13 The diagram shows a specific feature point planning map, containing feature points 1 to 18. Points 18, 19, 15, and 17 are the four vertices of the target semantic segmentation map, thus obtaining the vertices and boundaries of the region to be segmented. The first feature points include points 1, 15, 14, 10, and 6; the second feature points include points 2 to 5, 7 to 9, and 11 to 13. That is, the first feature points lie on the boundary where the target segmentation region (the ribbed area) overlaps with the target semantic segmentation map, and the second feature points lie on the arcs between the corresponding first feature points. It should be noted that, depending on the specific situation, point 16 can also be determined and replaced with point 14. The specific process of determining the first and second feature points can be shown below:

[0092] Point 1: Based on the target semantic segmentation map, target segmentation region a and target segmentation region b are determined. According to the corresponding features, the X coordinate of point 1 is located at a fixed distance to the left of the left boundary of the skirt edge, and the Y coordinate of point 1 is located at the upper boundary point of the rib segmentation surface. After finding the leftmost position of the target semantic segmentation map through the corresponding algorithm, the X coordinate of point 1 is determined by shifting it to the left by a certain position according to the empirical parameters. The corresponding Y coordinate can be roughly determined by the coordinates of the lowest part of the target semantic segmentation map, and fine calibration will be performed later.

[0093] Point 2: Point 2 is determined by moving point 1 vertically upward by a fixed distance. This fixed distance is determined by the process parameters, that is, by the cutting requirements.

[0094] The determination of points 3 to 5 is based on experience. These positions are at a fixed distance from the edge of the skirt, and the three points are evenly distributed. Taking point 3 as an example, first calculate the width of the skirt slice image. According to the evenly distributed proportion, find the corresponding two rectangular areas on the skirt slice image, and use the coordinates of the upper left corner of these two rectangles as positioning reference points. Construct a straight line equation from these two points and calculate its normal equation. Find a position on the normal equation that is a certain distance from the midpoint of the line connecting the two points as the coordinates of point 3. The same applies to points 4 and 5.

[0095] Point 6: Point 6 is located at the upper right corner of the rib. To determine this position, the outline of the rib should first be found. The search method involves obtaining the outline of the rib by means of image data channel extraction, background subtraction, morphological processing, and noise reduction. Based on the closed area of ​​the rib, the coordinates of point 6 can be obtained by finding its upper right corner.

[0096] Points 7 to 9 and 11 to 13: These points are implemented based on the thoracic cavity region. First, the binarized image of the segmented thoracic cavity region is sliced. This operation facilitates the identification of different regions within the rib cage. Essentially, all six points are determined by the location of the thoracic cavity; point 7 is used as an example. First, the coordinates of each small rectangle after slicing are calculated and arranged from left to right. Point 7 is located at 35% of the x-axis from left to right in the thoracic cavity. Based on this, the rectangle number corresponding to this point is determined, thus determining the X-coordinate of the point. The coordinate is determined by referencing the upper left corner of the rectangle where the point is located. The other five points follow a similar approach.

[0097] Points 10 and 14: Points 10 and 14 are obtained by offsetting points 10 and 13 to the right, respectively. The offset amount depends on the right boundary of the rib. The method for determining the boundary point is the same as that for point 6.

[0098] Point 15: The method for determining point 15 is the same as that for point 6. Its characteristic is that it is a protruding point on the right side of the rib boundary area.

[0099] As Figure 14 Taking a specific cutting plane diagram as an example, the relationship between the first feature point, the second feature point, and the segmentation trajectory is explained: Figure 14 The line segment between points O and H in the diagram represents Figure 13 The range of midpoint 1, the line segment between points M and D, represents Figure 13 The range of midpoint 6 can be explained, for example, by taking point 1 as point O and point 6 as point D. Figure 13 The arc formed by points 1, 2, 3, 4, 5, and 6 is Figure 14 The segmented trajectory OD in Figure 13The arc formed by points 1, 7, 8, 9, and 10 is Figure 14 The segmented trajectory OC in Figure 13 The arc formed by points 1, 11, 12, 13, and 14 is Figure 14 The segmented trajectory OB in the image. Wherein, Figure 14 The cutting tool EF'FGG' on the left represents vertical cutting of the center, i.e., operation 1. The cutting tool EF'FGG' on the right represents horizontal cutting of the center, i.e., operation 2. The arrows indicate one of the cutting directions of the tool, OD is another cutting direction, EF'F is the cutting part of the tool edge, OD is one of the cutting trajectories of the tool edge, X, Y, and Z are the coordinate axes of the world coordinate system, and the direction pointed by the arrow is the positive direction.

[0100] In this embodiment, the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the segmentation trajectory and the depth information, including: the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the tangent direction of the first feature point, the tangent direction of the second feature point, and the depth information. The robotic arm controls the cutting tool to complete operation 1 and operation 2, wherein there is no sequential order between operation 1 and operation 2; for example, operation 1 is performed first, followed by operation 2.

[0101] Operation 1: The robotic arm controls the cutting tool to cut along the dividing trajectory OD. During cutting, it is necessary to cut according to the tangent direction of points 1, 2, 3, 4, 5, and 6.

[0102] Operation 2: The robotic arm controls the cutting tool to cut along the segmentation trajectory OB. During cutting, the cutting must be performed according to the tangent directions of points 1, 11, 12, 13, and 14. Then, the robotic arm controls the cutting tool to cut along the segmentation trajectory OC. During cutting, the cutting must be performed according to the tangent directions of points 1, 7, 8, 9, and 10. Finally, the robotic arm controls the cutting tool to cut along the segmentation trajectory OD. During cutting, the cutting must be performed according to the tangent directions of points 1, 2, 3, 4, 5, and 6. It should be noted that in Operation 2, the cutting is performed horizontally along the segmentation trajectory OD, which is different from the vertical cutting along the segmentation trajectory OD in Operation 1.

[0103] Ideally, the cutting trajectories of Operation 1 and Operation 2 should completely overlap, with the intersection only occurring at the depths of OO' and DD'. In reality, the cutting surface will deform to varying degrees during the cutting process, so the cutting trajectories of Operation 1 and Operation 2 will not completely overlap. The trajectory of Operation 1 is relatively stable. To achieve the intersection of the cutting trajectories, in practical planning, the DO trajectory of Operation 2 is proportionally expanded by approximately 5mm towards point P, using the original line as a reference.

[0104] As can be seen, this application uses a preset camera to collect images to plan the segmentation trajectory, that is, it uses visual recognition of feature points, and through spatial calculation, plans a segmentation trajectory that fully includes the feature point position information. The robotic arm carries the cutting tool to precisely run on the points, so as to achieve precise cutting and separation of the ribs and the Chinese side of the product.

[0105] See Figure 15 As shown in the embodiments of this application, another specific Chinese segmentation method is disclosed, including:

[0106] Step S31: Use a preset camera to acquire the original image of the Chinese side to be segmented, and obtain the target semantic segmentation map of the original image.

[0107] Step S32: Determine the target segmentation region based on the target semantic segmentation map, and determine a number of first feature points on the boundary where the target segmentation region overlaps with the target semantic segmentation map; plan the arc between the corresponding first feature points, and determine a number of second feature points on the arc, so as to obtain the corresponding segmentation trajectory based on the first feature points and the second feature points.

[0108] Step S33: Determine the depth information of the segmentation trajectory so that the robotic arm adjusts the angle between the cutting edge of the cutting tool and the cutting surface of the square to be segmented to a preset acute angle, and then controls the cutting tool to segment the square to be segmented according to the tangent direction of the first feature point, the tangent direction of the second feature point and the depth information.

[0109] In this embodiment, for example Figure 16The diagram illustrates a specific segmentation trajectory, including the front and side cutting trajectories and the right side cutting trajectories. In other words, OO' and AA' represent the corresponding depth information. Because there are horizontal and vertical segments in the area to be segmented, there are also horizontal and vertical cutting surfaces. The robotic arm adjusts the angle between the cutting tool's blade edge and the vertical and horizontal cutting surfaces in the area to be segmented to a preset acute angle, mimicking the tilted posture of the tool during manual cutting. This acute angle is adjusted according to the smoothness of the cutting process. Since the ribs have a certain degree of curvature, the tool plane needs to be tangent to the curved part of the rib bone during cutting. In this embodiment, the slope value of the tangent line segment AOO' is used to represent RX to ensure the effectiveness of the front and side cutting. It is important to note that, in this embodiment, the tool pitch, yaw, and roll angles are mapped one-to-one with the RY, RZ, and RX coordinates of the robotic arm, achieving a one-to-one numerical correspondence. The slope is introduced here to simplify the calculation of the corresponding values. However, Euler angles and tangent slope are not the same concept. The slope is used to visualize the corresponding numerical value of Euler angles, meaning there is a corresponding positive correlation, but they are not equivalent. From this perspective, as... Figure 16 Taking a relatively curved section on the cutting area A'O' as an example, draw its tangent line L1. The RX value of L1 at this time is the tool RX value. In actual practice, different numbers of tangent lines need to be drawn on the A'O' trajectory line to ensure that the tool tip is always tangent to A'O'. This L1 is just an example. RX, RY, and RZ are only examples to illustrate a typical value describing the tool posture from the perspective of the corresponding diagram, but each point in each cutting trajectory will have corresponding RX, RY, and RZ. Divide the A'D' trajectory line into multiple curved sections. In actual practice, different numbers of tangent lines need to be drawn on the A'D trajectory line to ensure that the tool tip is always tangent to A'D'. This L1, L2, and L3 are examples. At this time, the angles formed by tangent lines L1, L2, and L3 with the X-axis of the world coordinate system are positively correlated with the pitch angle, denoted by RY. In summary, the entire cutting process involves simultaneous cutting of the planar part, the front side part, and the right side part. Precise control of the cutting area and the cutting trajectory is essential to complete the cutting.

[0110] As can be seen, this application not only uses a preset camera to accurately plan the segmentation trajectory and depth information, but also controls the angle between the blade edge of the cutting tool and the cutting surface to be segmented to a preset acute angle, so as to simulate the tilted posture of the tool when cutting manually, and to segment along the tangent direction of the segmentation trajectory, so that the width of the skirt edge and the amount of meat on the rib can be precisely controlled without exposing the bone or white part of the cut.

[0111] See Figure 17 As shown in the figure, this application discloses a Chinese segmentation device, including:

[0112] The semantic segmentation map acquisition module 11 is used to acquire the original image of the Chinese side to be segmented using a preset camera, and to acquire the target semantic segmentation map of the original image;

[0113] The segmentation module 12 is used to plan a segmentation trajectory based on the target semantic segmentation map and determine the depth information of the segmentation trajectory, so that the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the segmentation trajectory and the depth information.

[0114] The beneficial effects of this application are as follows: It utilizes a preset camera to acquire the original image of the Chinese side to be segmented and obtains a target semantic segmentation map of the original image; based on the target semantic segmentation map, it plans a segmentation trajectory and determines the depth information of the segmentation trajectory, so that the robotic arm can control the cutting tool to segment the Chinese side according to the segmentation trajectory and the depth information. Therefore, this application only needs to use a preset camera to acquire the original image of the Chinese side to be segmented to obtain the target semantic segmentation map, without needing to acquire redundant information about the Chinese side to be segmented, thus improving the efficiency of Chinese side segmentation and reducing segmentation complexity; by planning the segmentation trajectory and its depth information, the robotic arm can control the cutting tool to perform more precise segmentation according to the segmentation trajectory and depth information.

[0115] Furthermore, embodiments of this application also provide an electronic device. Figure 18 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0116] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the Chinese segmentation method performed by the electronic device as disclosed in any of the foregoing embodiments.

[0117] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0118] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0119] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0120] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the Chinese segmentation method disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0121] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the method steps executed by the Chinese side during the segmentation process disclosed in any of the foregoing embodiments.

[0122] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The foregoing has provided a detailed description of a Chinese segmentation method, apparatus, device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for dividing the Chinese side, characterized in that, include: The original image of the Chinese part to be segmented is acquired using a preset camera, and the target semantic segmentation map of the original image is obtained; Based on the target semantic segmentation map, a segmentation trajectory is planned, and the depth information of the segmentation trajectory is determined, so that the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the segmentation trajectory and the depth information; The segmentation trajectory includes a horizontal segmentation trajectory and a vertical segmentation trajectory; Wherein, obtaining the target semantic segmentation map of the original image includes: The original semantic segmentation map of the original image is obtained using a semantic segmentation model based on the DeepLabV3 algorithm; noise removal and slicing operations are performed on the original semantic segmentation map to obtain the target semantic segmentation map. The step of planning the segmentation trajectory based on the target semantic segmentation map includes: Based on the target semantic segmentation map, a target segmentation region is determined, and several first feature points are determined on the boundary where the target segmentation region overlaps with the target semantic segmentation map. An arc is planned between the corresponding first feature points, and several second feature points are determined on the arc, so that a corresponding segmentation trajectory is obtained based on the first feature points and the second feature points. The target segmentation region is a rib area, which includes the thoracic cavity region and the skirt edge region. Some of the first feature points are determined based on the boundary where the skirt edge region overlaps with the target semantic segmentation map, and some of the second feature points are determined based on the thoracic cavity region. The robotic arm controls the cutting tool to segment the area to be segmented according to the segmentation trajectory and the depth information, including: The robotic arm adjusts the angle between the cutting edge of the cutting tool and the cutting surface of the square to be segmented to a preset acute angle, and then controls the cutting tool to segment the square to be segmented according to the tangent direction of the first feature point, the tangent direction of the second feature point, and the depth information.

2. The Chinese segmentation method according to claim 1, characterized in that, The step of planning a segmentation trajectory based on the target semantic segmentation map and determining the depth information of the segmentation trajectory, so that the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the segmentation trajectory and the depth information, includes: Based on the target semantic segmentation map, a segmentation trajectory is planned, and the depth information of the segmentation trajectory is determined; A first control command corresponding to the segmentation trajectory and the depth information is generated and sent to the robotic arm so that the robotic arm can perform coordinate system transformation on the first control command to obtain a second control command, and control the cutting tool to segment the area to be segmented according to the second control command.

3. The Chinese segmentation method according to claim 2, characterized in that, The robotic arm performs coordinate system transformation on the first control command to obtain a second control command, including: Determine the coordinate system transformation relationship between the first coordinate system of the robotic arm and the second coordinate system of the cutting tool; The first control command is transformed using the coordinate system transformation relationship to obtain the second control command.

4. A Chinese segmentation device, characterized in that, include: The semantic segmentation map acquisition module is used to acquire the original image of the Chinese side to be segmented using a preset camera, and to acquire the target semantic segmentation map of the original image; The segmentation module is used to plan a segmentation trajectory based on the target semantic segmentation map and determine the depth information of the segmentation trajectory, so that the robotic arm controls the cutting tool to segment the Chinese side to be segmented according to the segmentation trajectory and the depth information; The semantic segmentation map acquisition module is specifically used for: The original semantic segmentation map of the original image is obtained using a semantic segmentation model based on the DeepLabV3 algorithm; noise removal and slicing operations are performed on the original semantic segmentation map to obtain the target semantic segmentation map. The segmentation module is specifically used for: Based on the target semantic segmentation map, a target segmentation region is determined, and several first feature points are determined on the boundary where the target segmentation region overlaps with the target semantic segmentation map. An arc is planned between the corresponding first feature points, and several second feature points are determined on the arc, so that a corresponding segmentation trajectory is obtained based on the first feature points and the second feature points. The target segmentation region is a rib area, which includes the thoracic cavity region and the skirt edge region. Some of the first feature points are determined based on the boundary where the skirt edge region overlaps with the target semantic segmentation map, and some of the second feature points are determined based on the thoracic cavity region. The segmentation module is specifically used for: The robotic arm adjusts the angle between the cutting edge of the cutting tool and the cutting surface of the square to be segmented to a preset acute angle, and then controls the cutting tool to segment the square to be segmented according to the tangent direction of the first feature point, the tangent direction of the second feature point, and the depth information.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the Chinese segmentation method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the Chinese segmentation method as described in any one of claims 1 to 3.

Images