A robot intelligent slaughtering and cutting method, device and storage medium

Abstract

The present invention relates to a robot intelligent slaughtering and segmentation method, device and storage medium, the method comprising: obtaining 3D point cloud data of a meat block to be slaughtered and cut, or obtaining 3D point cloud data and image data of a meat block to be slaughtered and cut; identifying a target meat block cutting area from the acquired data to determine the 3D point cloud data of the target meat block cutting area; determining a cut point and a plurality of reference points according to the 3D point cloud data of the target meat block cutting area, wherein the cut point and the plurality of reference points constitute segmentation trajectory information for slaughtering and segmentation; planning the path information of the robot to the target meat block according to the position of the target meat block and the position of the robot; after controlling the robot to reach the target meat block according to the path information, slaughtering and segmenting the target meat block according to the segmentation trajectory information. The method of the present invention can improve work efficiency and increase the quality and standardization of cut products.

Description

Technical Field

[0001] The present invention generally relates to the field of intelligent slaughtering, and in particular to a robot intelligent slaughtering and cutting method, device and storage medium. Background Art

[0002] In the field of pig slaughtering, carcass cutting is a key link in pig slaughtering and one of the key links to support the profit and revenue of slaughterhouses. The existing slaughtering and cutting process is mainly based on labor-intensive slaughtering methods; the more advanced online cutting method also uses the carcass to be hung on the assembly line and then cut by hand with a knife.

[0003] The limitations of the existing pig slaughtering and cutting methods are:

[0004] (1) Different operators use different forms of hand-held tool cutting, and there is no unified measurement standard for cutting quality and effect;

[0005] (2) There are high requirements for the technical proficiency of slaughter workers;

[0006] (3) The efficiency of slaughtering and cutting is low, and there is a high risk of accidental injury to workers during operation.

[0007] To sum up, it is necessary to develop a more intelligent slaughtering and cutting system and method. Summary of the invention

[0008] The present invention provides a robot intelligent slaughtering and segmenting method based on 3D vision guidance, which is used to at least solve the problems of high requirements for slaughter workers, low slaughtering and segmenting efficiency, and accidental injuries in the prior art, so as to improve the quality and standardization of slaughtering and segmenting. At the same time, the present invention also provides a device and a computer-readable storage medium for implementing the above method.

[0009] According to a first aspect of the present invention, a robot intelligent slaughtering and segmentation method based on 3D vision guidance is provided, comprising: acquiring 3D point cloud data of a piece of meat to be slaughtered and cut, or acquiring 3D point cloud data and image data of the piece of meat to be slaughtered and cut; identifying a target piece of meat cutting area from the acquired data to determine the 3D point cloud data of the target piece of meat cutting area; determining a cutting point and a plurality of reference points according to the 3D point cloud data of the target piece of meat cutting area, wherein the cutting point and the plurality of reference points constitute segmentation trajectory information for slaughtering and segmentation; planning path information from the robot to the target piece of meat according to the position of the target piece of meat and the position of the robot; and after controlling the robot to reach the target piece of meat according to the path information, slaughtering and segmenting the target piece of meat according to the segmentation trajectory information.

[0010] In one embodiment, identifying the target meat cutting area includes: identifying the target meat cutting area based on depth information in the 3D point cloud data of the meat to be slaughtered and cut; or identifying the target meat cutting area from the image data.

[0011] In one embodiment, the method further includes preprocessing the 3D point cloud data of the target meat cutting area: filtering the 3D point cloud data of the target meat cutting area to remove noise points, and downsampling to reduce the point cloud density.

[0012] In one embodiment, the meat to be slaughtered and cut is medium meat; the method further includes: after the preprocessing, constructing a covariance matrix for each point in the 3D point cloud data of the target meat cutting area, and calculating eigenvalues; according to the eigenvalues, selecting feature points from the 3D point cloud data of the target meat cutting area.

[0013] In one embodiment, the cutting point and multiple reference points are determined based on the 3D point cloud data of the target meat cutting area, including: filtering out the feature points located at the edge of the target meat cutting area, and dividing the remaining feature points into an upper and lower part; clustering the feature points in the upper part to determine the cutting point; performing straight line fitting on the feature points in the lower part, and determining the multiple reference points based on the fitted straight line.

[0014] In one embodiment, clustering the feature points of the upper part to determine the cutting point includes: clustering according to the feature values ​​of the feature points of the upper part, selecting the mean value of the X coordinate of the largest cluster as the X coordinate of the cutting point, and determining the Y coordinate and Z coordinate of the cutting point according to empirical values;

[0015] In one embodiment, determining the multiple reference points according to the fitting straight line includes: uniformly selecting multiple points on the fitting straight line as the reference points.

[0016] In one embodiment, straight line fitting is performed on the feature points, including: determining whether the middle piece of meat is on the left or right side according to the angle between the fitting straight line and the Y-axis; detecting the feature points of the lower half in response to the angle between the fitting straight line and the Y-axis being less than a preset angle; determining that the middle piece of meat is on the right side in response to the number of feature points in the lower right area of ​​the lower half being greater than the number of feature points in the lower left area, and determining that the middle piece of meat is on the left side in response to the number of feature points in the lower right area of ​​the lower half being less than the number of feature points in the lower left area.

[0017] According to the second aspect of the present invention, the present invention also provides a robot intelligent slaughtering and cutting device based on 3D vision guidance, characterized in that it includes a robot and a 3D camera for slaughtering and cutting, the robot has a processor and a memory, and when the processor executes the computer program stored in the memory, it implements the robot intelligent slaughtering and cutting method based on 3D vision guidance as described in any one of the first aspect of the present invention.

[0018] According to the third aspect of the present invention, the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the robot intelligent slaughtering and cutting method based on 3D vision guidance as described in any one of the first aspect of the present invention is implemented.

[0019] The embodiment of the present invention provides a robot intelligent slaughtering and segmentation method based on 3D vision guidance, which collects 3D point cloud data information through a 3D vision system, converts it into robot motion trajectory data through a specific robot planning algorithm and sends it to a robot controller, thereby guiding the robot to drive a special tool to cut the target object, which has the following beneficial effects:

[0020] (1) The present invention can solve many cutting application conditions in current slaughterhouses, replacing manual cutting methods. In scenarios where multiple cutters are required to complete the cutting, the present invention can even complete the cutting with just one cutter, thereby reducing the intensity of manual labor and improving work efficiency.

[0021] (2) Increase the quality and standardization of cutting products;

[0022] (3) The intelligence level of slaughterhouse equipment has been improved, and the process of intelligent upgrading and transformation of slaughterhouses has been promoted. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0024] Figure 1 is a flow chart of a robot intelligent slaughtering and segmentation method based on 3D vision guidance according to an embodiment of the present invention;

[0025] Figure 2 is a flow chart of a method for determining segmentation trajectory information in step S3;

[0026] Figure 3 is an instance of point cloud data;

[0027] Figure 43D vision-guided robot intelligent slaughtering and segmenting control system according to an embodiment of the present invention;

[0028] Figure 5 is a structural diagram of a robot intelligent slaughtering and cutting device based on 3D vision guidance according to an embodiment of the present invention;

[0029] in, Figure 4 In the figure, 1-3D vision camera dark box assembly; 2-meat to be cut; 3-conveyor belt; 4-industrial robot body; 5-special cutting tool; 6-robot base. DETAILED DESCRIPTION

[0030] Embodiments will now be described with reference to the accompanying drawings. It should be understood that for simplicity and clarity of explanation, reference numerals may be repeated in the accompanying drawings to indicate corresponding or similar elements where deemed appropriate. In addition, the present application sets forth many specific details in order to provide a thorough understanding of the embodiments described herein. However, it will be appreciated by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other cases, well-known methods, processes, and components are not described in detail to avoid blurring the embodiments described herein. Moreover, the description should not be construed as limiting the scope of the embodiments described herein.

[0031] Figure 1 A robot intelligent slaughtering and segmentation method based on 3D vision guidance according to an embodiment of the present invention is presented, comprising:

[0032] Step S1, obtaining 3D point cloud data of the meat to be slaughtered and cut, or obtaining 3D point cloud data and image data of the meat to be slaughtered and cut.

[0033] Step S2, identifying a target meat cutting area from the acquired data to determine 3D point cloud data of the target meat cutting area;

[0034] Step S3, determining a cutting point and a plurality of reference points according to the 3D point cloud data of the target meat cutting area, wherein the cutting point and the plurality of reference points constitute segmentation trajectory information for slaughtering and segmentation;

[0035] Step S4, planning path information from the robot to the target piece of meat according to the position of the target piece of meat and the position of the robot;

[0036] Step S5, after controlling the robot to reach the target piece of meat according to the path information, slaughter and cut the target piece of meat according to the cutting trajectory information.

[0037] In step S1, 3D point cloud data of the meat to be slaughtered and cut can be obtained. The 3D point cloud data of the meat to be slaughtered and cut includes not only the 3D point cloud data of the target meat cutting area, but also the point cloud data around the target meat cutting area. Therefore, it is necessary to first identify the target meat cutting area. In step S1 and step S2, in order to identify the target meat cutting area, two methods are included. The first method is to identify the target meat cutting area based on the depth information in the 3D point cloud data of the meat to be slaughtered and cut. The second method is to identify the target meat cutting area from the image data.

[0038] In the first method, the target meat cutting area can be identified only by relying on 3D point cloud data. In actual application scenarios, the target meat is placed on a conveyor belt (e.g. Figure 5 As shown, the depth information of the point cloud data of the target piece of meat (indicating the height of the piece of meat) is significantly different from the depth information of the point cloud data around the target piece of meat (indicating the height of the conveyor belt). Through the depth information of the point cloud data, for example, when the depth information of the point cloud data suddenly changes, it is considered to be the boundary area between the target piece of meat and the conveyor belt.

[0039] In the second method, it is necessary to use the plane image data. In the plane image data, it is easy to identify the target meat block with the help of various existing recognition algorithms, that is, to obtain the target meat block cutting area, and then the 3D point cloud data belonging to the target meat block cutting area can be determined in the 3D point cloud data of the meat block to be slaughtered and cut.

[0040] Among steps S3 to S5, step S4 belongs to a relatively mature trajectory planning algorithm, and step S5 also belongs to a mature technology, so it will not be described in detail. The following mainly describes step S3.

[0041] Figure 2 A schematic flow chart of the method for determining segmentation trajectory information in step S3 is shown.

[0042] In step S31, the 3D point cloud data of the target meat cutting area is first preprocessed, including: filtering the 3D point cloud data of the target meat cutting area to remove noise points, and downsampling to reduce the point cloud density.

[0043] The filtering algorithm may adopt various filtering algorithms for point cloud data in the prior art, and the present invention does not limit this; for example, the point cloud data is projected onto three coordinate axes to filter out protruding and not smooth point cloud data. The purpose of downsampling is to reduce the point cloud density and reduce the complexity of subsequent operations. For example, one data point is sampled for every 100 data points, thereby greatly reducing the point cloud density.

[0044] In step S32, feature points are selected from the point cloud data, so as to reduce the data dimension and retain only the information of the dimension of interest, thereby reducing the amount of calculation and improving the calculation efficiency.

[0045] In an actual application scenario, the meat to be slaughtered and cut is middle meat, which refers to a piece of meat with ribs, for example, the middle meat may belong to the left side of the pig, with left ribs, or the middle meat may belong to the right side of the pig, with right ribs.

[0046] Figure 3 An example of point cloud data is shown. The corresponding middle meat belongs to the left side of the pig, which has the left ribs. Figure 3 In the figure, the area where B3 is located indicates the left ribs. A1 to A8 are the track points in the segmentation track. The robot can separate the ribs from the middle meat according to the track points. After segmentation, the middle meat includes two pieces, one is the ribs and the other is the part without the ribs (such as pork belly).

[0047] Specifically, step S32 includes: after the preprocessing, constructing a covariance matrix for each point in the 3D point cloud data of the target meat cutting area, and calculating eigenvalues ​​and eigenvectors; and selecting feature points from each point in the 3D point cloud data of the target meat cutting area according to the eigenvalues. For example, the eigenvalues ​​of each point are sorted, and if the ratio of adjacent eigenvalues ​​is greater than a set threshold, the corresponding point is selected as the feature point.

[0048] Among them, the contents of constructing the covariance matrix, calculating the eigenvalues ​​and eigenvectors, etc., all belong to the existing technology, and thus will not be described in detail here.

[0049] In step S33, the feature points located at the edge of the target meat cutting area are filtered out, and the remaining feature points are divided into an upper part and a lower part.

[0050] For example, since the feature points at the edge of the target meat cutting area represent the information of the meat edge, this part of information is irrelevant to the segmentation trajectory information (e.g. Figure 3 The feature points of the B1 part in the image are white points, so the feature points at the edge are first filtered out. For the remaining feature points, these feature points can be equally divided in the X-axis direction into the upper half and the lower half. Of course, in other embodiments, it is also possible not to equally divide them, for example, the upper part can be smaller than the lower part, or the upper part can be larger than the lower part. For ease of understanding, Figure 3 In the figure, the X axis is the up-down direction, and the Y axis is the left-right direction.

[0051] In step S34, the feature points of the upper part are processed to obtain the cutting point A1. In this embodiment, in addition to obtaining the cutting point A1, reference points A2 and A3 can also be obtained.

[0052] The cutting point A1 is where the robot cuts. The robot cuts at the cutting point A1, and the blade cuts downward. The cutting depth is equal to the Z coordinate of the cutting point A1. Then the blade moves downward and passes through reference points A2 to A8 in sequence, thereby separating the ribs. Figure 3 A7 and A8 are outside the target meat block and are used to control the robot to return to its original position.

[0053] Specifically, in step S34, the feature values ​​of the feature points in the upper part are clustered according to the Euclidean distance, and the mean value of the X coordinate of the largest class is used as the X coordinate value of the cutting point A1. The Y coordinate value and Z coordinate value of the cutting point can be assigned according to empirical values. For example, the Y coordinate value of the cutting point A1 determines the position of the cutting point A1 on the target piece of meat in the left and right directions, and the left and right length (width) of the target piece of meat is relatively fixed, so its Y coordinate value can be determined according to empirical values. The Z coordinate value of the cutting point A1 determines the cutting depth, which can also be determined according to empirical values. In other embodiments, the Y coordinate value of the cutting point A1 can also be determined according to the actual width of the target piece of meat, and the Z coordinate value of the cutting point A1 can also be determined according to the actual depth of the target piece of meat. As an example, if the width of the target piece of meat is a (which can be obtained according to the width of the target piece of meat indicated by the point cloud data), a is calculated (for example, a certain ratio is obtained) to obtain the Y coordinate value of the cutting point A1. If the depth of the target meat block is b (which can be obtained based on the depth information of the point cloud data of the target meat block), b is calculated (for example, a certain ratio is calculated) to obtain the Z coordinate value of the cutting point A1.

[0054] After obtaining the cutting point A1, in this embodiment, the reference points A2 and A3 can be obtained based on fixed parameters. For example, the fixed parameters can be the relative position of A2 and A1, and the relative position of A3 and A2; the reference points A2 and A3 can also be obtained by offsetting according to a certain proportion of the target meat piece width.

[0055] In step S35, straight line fitting is performed on the feature points of the lower portion, and the plurality of reference points are determined according to the fitting straight line.

[0056] As an example, the RANSAC algorithm can be used to fit a straight line. The equation of the fitted line is expressed as Y=Kx+b. According to the slope K, it can be determined whether the middle meat is on the left or right side (of the pig carcass). Intuitively, the slope K represents the angle between the fitted line and the Y axis. Figure 3 As shown, for example: if the slope of the fitting line is positive, the corresponding middle meat is on the left, and if the slope of the fitting line is negative, the corresponding middle meat is on the right.

[0057] Depending on whether the middle meat is on the left or right side, the b value of the corresponding fitting straight line equation is different, and thus the final fitting straight line is different. The b value can be an empirical value, for example, if the middle meat is on the left side, the b value is assigned to be equal to b1, and if the middle meat is on the right side, the b value is assigned to be equal to b2.

[0058] As an example, multiple points can be evenly selected on the fitting line to obtain reference points A4 to A7. For example, a bounding box of the feature points of the lower part is constructed (i.e., the length and width of the lower part are calculated), and several X values ​​are obtained by evenly dividing them in the X-axis direction, and then the corresponding Y coordinate value is obtained by substituting the obtained fitted line equation, that is, the corresponding reference point is obtained (the Z coordinate of the reference point can be equal to the Z coordinate value of the lower cutting point A1). In other embodiments, the reference points can be selected by not evenly dividing them, but by unequally dividing them, and the present invention is not limited to this.

[0059] In some cases, the slope K of the fitting line may not be able to accurately determine whether the middle meat is on the left or right side. For example, when the angle between the fitting line and the Y axis is close to 90°, it is difficult to determine whether the middle meat is on the left or right side.

[0060] Based on this, when it is difficult to determine whether the middle meat is on the left or right side based on the slope K, the present invention further detects the feature points of the lower part, that is, performs a secondary judgment. For example: in response to the number of feature points in the lower right area of ​​the lower part being greater than the number of feature points in the lower left area, it is determined that the middle meat is on the right side; in response to the number of feature points in the lower right area of ​​the lower half being less than the number of feature points in the lower left area, it is determined that the middle meat is on the left side. Figure 3 As shown, there are more feature points in the lower right area (part B2), which means that the corresponding middle meat is on the left.

[0061] In addition, the point cloud processing method that can be used in this application can refer to the linear regression least squares method, gradient descent method, RANSAC algorithm, ICP algorithm, PCL surface clustering segmentation algorithm, region growing algorithm, Euclidean clustering algorithm and / or bounding box algorithm, etc.

[0062] In summary, after step S31 to step S35, the cutting point A1 and reference points A2 to A8 are obtained, that is, the segmentation trajectory information is obtained. The robot performs a cutting operation according to the segmentation trajectory information to separate the ribs from the middle meat.

[0063] In step S5, the method for planning path information can be: according to the cutting point A1 of the target meat cutting area and the position of the robot, the shortest path information from the robot to the target meat piece is planned by a trajectory planning algorithm, and the robot is controlled to reach the cutting point A1 according to the shortest path information.

[0064] As an example, the robot is a mechanical arm, and a standard cutting knife is connected to the end of the mechanical arm, and the standard cutting knife slaughters and cuts the target meat according to the cutting trajectory information. Of course, the form of the robot is not limited to the mechanical arm, and other possible forms are within the protection scope of the present invention as long as they can adapt to the slaughtering and cutting environment.

[0065] According to the second aspect of the present invention, the present invention further provides a robot intelligent slaughtering and cutting device based on 3D vision guidance, which includes a robot control system. As an example, the underlying architecture of the robot control system is implemented as follows:

[0066] (1) Mechanism layer: including the transmission mechanism and the electrical mechanism. The transmission mechanism is used to complete the transmission of the meat to be cut in the form of a workbench assembly line, for example, and the positioning and image and point cloud data collection of the 3D vision device. The electrical mechanism uses conventional electrical components and control logic to provide power for the transmission mechanism and coordinate the process.

[0067] (2) Perception layer: mainly includes IO devices and cameras, which complete the perception and acquisition of the final image and point cloud data;

[0068] (3) Execution layer: mainly includes industrial robots, which are used to execute the final cutting action;

[0069] (4) The Window / Linux system that the system relies on completes the software architecture, which mainly includes:

[0070] A. System layer: including camera processing module, business processing module, algorithm module and communication module;

[0071] B. Control layer: complete functions such as point cloud display, equipment operation status monitoring, and user operation interface interaction.

[0072] Based on the above architecture and Figure 5 The 3D vision-guided robot intelligent slaughtering and cutting device and its peripheral equipment shown in the figure can realize the above Figure 1 The robot intelligent slaughtering and segmentation method based on 3D vision guidance is shown.

[0073] Figure 5A robot intelligent slaughtering and cutting device based on 3D vision guidance according to an embodiment of the present invention is shown, comprising: a 3D vision dark box component 1, a workbench assembly line, an industrial robot and a special cutting tool 5, wherein: the 3D vision dark box component 1 is composed of a 3D camera, a light source, a fixing frame and a shell box body shielding external interference, the 3D camera and the light source are arranged in the shell box body, the fixing frame is supported below the shell box body, the 3D vision dark box component 1 is arranged upstream of the workbench assembly line, the shell box body is used to shield external light source interference, and at the same time provides a stable visual system data acquisition environment, which can stably collect plane images and 3D point cloud data information, thereby obtaining the stability of the later image and point cloud processing algorithm. The workbench assembly line has a conveyor belt 3, on which the meat block 2 to be cut (i.e., the square meat) is placed. The industrial robot is composed of an industrial robot body 4 and an industrial robot base 6, and the industrial robot body 4 is arranged above the industrial robot base 6; the special cutting tool 5 is fixed at the free end of the industrial robot body 4, and the special cutting tool 5 is specially designed for specific different cutting products and objects to implement different cutting effects and improve cutting efficiency.

[0074] For example, the special cutting tool 5 is bow-shaped or arc-shaped. For example, the special cutting tool 5 is a non-powered or powered cutting tool. It is determined whether the cutting tool itself needs to have a driving force according to the hardness and speed of the object to be cut. If the hardness of the meat block is high and the cutting speed is required to be fast, the tool itself needs to provide an auxiliary driving force source other than the power of the industrial robot through a power device such as a micro motor.

[0075] like Figure 5 As shown, the assembly line moves from left to right. The control system of the robot includes a processor and a memory; a trigger sensor (such as a laser sensor) is provided in the 3D visual dark box component 1, which is used to trigger when the meat piece arrives and feed back to the control system. The control system controls the 3D camera to take pictures, thereby obtaining the above-mentioned segmentation trajectory information and path information. Since the running speed of the assembly line is known, the control system can calculate when the meat piece will reach the position of the industrial robot body 4, and the control system can manipulate the industrial robot body 4 to act according to the path information and segmentation trajectory information after the time arrives. For example, at time t0, a piece of meat arrives, the trigger sensor is triggered, and the control system calculates that the piece of meat arrives at the industrial robot body 4 at time t1, so the control system manipulates the industrial robot body 4 to act according to the path information and segmentation trajectory information at time t1. In other words, the control system needs to calculate the path information between time t0 and time t1, and the path information is calculated by the control system based on the position of the industrial robot body 4 and the position of the target piece of meat at time t1.

[0076] According to the above description, when the special cutting tool 5 is at the cutting point, the blade faces downward, and when it reaches a certain depth (for example, the Z coordinate of the cutting point), the blade faces inward, thereby separating the ribs.

[0077] The processor may include a CPU, which may be a general-purpose CPU, a dedicated CPU, or other execution units for information processing and program operation, and the processor executes a computer program to implement the robot intelligent slaughtering and segmentation method based on 3D vision guidance as described in any embodiment of the first aspect of the present invention. Further, a large-capacity memory and a read-only memory ROM may be included, wherein the large-capacity memory may be configured to store various types of data, including image and point cloud test data, intermediate data, path planning results, and various programs required for path planning, and the ROM may be configured to store data required for power-on self-test of the robot intelligent slaughtering and segmentation device based on 3D vision guidance, initialization of various functional modules in the system, basic input / output drivers of the system, and booting the operating system.

[0078] It is understandable that although a variety of hardware platforms or components are shown in the 3D vision-guided robotic intelligent slaughtering and cutting equipment, these are merely exemplary and not restrictive, and those skilled in the art can add or remove corresponding hardware according to actual needs.

[0079] The 3D vision-guided robotic intelligent slaughtering and cutting device of the present invention also includes a communication interface, so that it can be connected to a local area network / wireless local area network (LAN / WLAN) through the communication interface, and then connected to a local server or to the Internet ("Internet") through the LAN / WLAN. Alternatively or additionally, the 3D vision-guided robotic intelligent slaughtering and cutting equipment of the present invention can also be directly connected to the Internet or cellular network through the communication interface based on wireless communication technology, such as wireless communication technology based on the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") . In some application scenarios, the 3D vision-guided robotic intelligent slaughtering and cutting equipment of the present invention can also access servers and possible databases of external networks as needed to obtain various known path planning algorithms, data and modules, and can remotely store various path planning data.

[0080] The peripheral devices of the robot intelligent slaughtering and cutting equipment based on 3D vision guidance may include a display device, an input device and a data transmission interface. In one embodiment, the display device may include, for example, one or more speakers and / or one or more visual displays, which are configured to provide voice prompts and / or image video displays for the operation process or final results of the robot intelligent slaughtering and cutting equipment based on 3D vision guidance of the present invention. The input device may include, for example, a keyboard, a mouse, a microphone, a gesture capture camera, or other input buttons or controls, which are configured to receive input of test data or user instructions. The data transmission interface may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a FireWire ("FireWire"), a PCI Express and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present invention, the data transmission interface can receive original images, point cloud data or type-converted data for path planning, and transmit various types of data and results to the robot intelligent slaughtering and cutting equipment based on 3D vision guidance.

[0081] The CPU, mass storage, read-only memory ROM, TPU, GPU, FPGA, MLU and communication interface of the 3D vision-guided robot intelligent slaughtering and cutting device of the present invention can be interconnected through a bus, and data interaction with peripheral devices can be achieved through the bus. In one embodiment, through the bus, the CPU can control other hardware components and peripheral devices in the 3D vision-guided robot intelligent slaughtering and cutting device.

[0082] During operation, the processor CPU of the robot intelligent slaughtering and cutting device based on 3D vision guidance of the present invention can receive test data through an input device or a data transmission interface, and retrieve computer program instructions or codes (such as codes related to path planning) stored in a memory to perform feature extraction on the received data to obtain path planning data for the industrial robot. At the same time, the robot intelligent slaughtering and cutting device based on 3D vision guidance can retrieve the algorithm in the memory and use the GPU to execute the path planning data to obtain the action path of the robot arm, etc., and can also display the response results on the display device or output them by voice prompts. In addition, the robot intelligent slaughtering and cutting device based on 3D vision guidance can also upload the results to a network, such as a remote database, through a communication interface.

[0083] It should also be understood that any module, unit, component, server, computer, terminal or device that executes instructions of the present invention examples may include or otherwise access computer-readable media, such as storage media, computer storage media or data storage devices (removable) and / or non-removable) such as disks, optical disks or tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.

[0084] According to the third aspect of the present invention, the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the robot intelligent slaughtering and cutting method based on 3D vision guidance as described in any embodiment of the first aspect of the present invention is implemented.

[0085] The computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0086] It should be understood that the possible terms "first" or "second" etc. in the claims, specifications and drawings disclosed in the present invention are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims disclosed in the present invention indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0087] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in the disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in the disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0088] Although the embodiments of the present invention are as described above, the contents are only examples used to facilitate understanding of the present invention, and are not intended to limit the scope and application scenarios of the present invention. Any technician in the technical field of the present invention can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed by the present invention, but the scope of patent protection of the present invention shall still be based on the scope defined by the attached claims.

Claims

1. A robot intelligent slaughtering and segmentation method based on 3D vision guidance, characterized in that: include: Acquire 3D point cloud data of the meat to be slaughtered and cut, or acquire 3D point cloud data and image data of the meat to be slaughtered and cut; Identifying a target meat cutting area from the acquired data to determine 3D point cloud data of the target meat cutting area; Determining a cutting point and a plurality of reference points according to the 3D point cloud data of the target meat cutting area, wherein the cutting point and the plurality of reference points constitute segmentation trajectory information for slaughtering and segmentation; Planning path information from the robot to the target piece of meat according to the position of the target piece of meat and the position of the robot; After controlling the robot to reach the target piece of meat according to the path information, slaughter and segment the target piece of meat according to the segmentation trajectory information; The method further includes preprocessing the 3D point cloud data of the target meat cutting area: filtering the 3D point cloud data of the target meat cutting area to remove noise points, and Downsampling is performed to reduce the density of the point cloud; Wherein, the meat to be slaughtered and cut is Chinese meat; The method further comprises: constructing a covariance matrix for each point in the 3D point cloud data of the target meat cutting area after the preprocessing, and calculating an eigenvalue; and selecting a feature point from the 3D point cloud data of the target meat cutting area according to the eigenvalue; Wherein, the cutting point and multiple reference points are determined according to the 3D point cloud data of the target meat cutting area, including: Filter out the feature points located at the edge of the target meat cutting area, and divide the remaining feature points into an upper part and a lower part; Clustering the feature points of the upper portion to determine the cutting point; and Performing straight line fitting on the feature points of the lower portion, and determining the plurality of reference points according to the fitted straight line; Wherein, determining the multiple reference points according to the fitting straight line includes: uniformly selecting multiple points on the fitting straight line as the reference points; Among them, performing straight line fitting on the characteristic points includes: Determine whether the square meat is on the left or right side according to the angle between the fitting straight line and the Y axis; In response to the angle between the fitting straight line and the Y axis being less than a preset angle, detecting a feature point of a lower portion; and In response to the fact that the number of feature points in the lower right area of ​​the lower part is greater than the number of feature points in the lower left area, it is determined that the middle meat is on the right side. In response to the fact that the number of feature points in the lower right area of ​​the lower part is less than the number of feature points in the lower left area, it is determined that the middle meat is on the left side.

2. The robot intelligent slaughtering and cutting method according to claim 1 is characterized in that: Identify the target meat cutting areas including: Identifying the target meat cutting area according to the depth information in the 3D point cloud data of the meat to be slaughtered and cut; or A target meat cutting area is identified from the image data.

3. The robot intelligent slaughtering and cutting method according to claim 2 is characterized in that: Clustering the feature points of the upper part to determine the cutting point includes: Clustering is performed according to the characteristic values ​​of the characteristic points in the upper part, and the mean value of the X coordinate of the largest cluster is selected as the X coordinate of the cutting point, and the Y coordinate and Z coordinate of the cutting point are determined according to empirical values.

4. A robot intelligent slaughtering and cutting device based on 3D vision guidance, characterized in that: The invention comprises a robot for slaughtering and cutting and a 3D camera, wherein the robot has a processor and a memory, and when the processor executes a computer program stored in the memory, the robot intelligent slaughtering and cutting method based on 3D vision guidance as claimed in any one of claims 1 to 3 is implemented.

5. A computer-readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed, implements the robot intelligent slaughtering and segmenting method based on 3D vision guidance according to any one of claims 1 to 3.

Images