A multi-adaptive intelligent beef cutting method and device

Abstract

This application discloses a multi-adaptive intelligent beef cutting method and apparatus, relating to the field of intelligent meat cutting. When the beef to be cut on the beef cutting production line reaches the freshness detection position, a first RGB image of the beef to be cut is acquired; the freshness of the beef to be cut is identified based on the first RGB image, and a freshness identification result is obtained; when qualified beef to be cut on the beef cutting production line reaches the cutting position, a second RGB image and a depth image of the qualified beef to be cut are acquired; the cutting trajectory and cutting parameters of the qualified beef to be cut are determined based on the second RGB image and the depth image; the beef cutting production line is controlled based on the cutting parameters and the cutting trajectory. This application improves the quality and accuracy of beef cutting.

Description

Technical Field

[0001] This application relates to the field of intelligent meat cutting, and in particular to a multi-adaptive intelligent beef cutting method and apparatus. Background Technology

[0002] Beef is rich in nutrients and widely enjoyed. Reports indicate that global beef production and consumption will grow by 9.27% ​​and 10.12% respectively over the next 10 years. Therefore, there is an urgent need to improve the efficiency and stability of the meat processing industry.

[0003] Currently, the livestock meat processing industry relies heavily on manual labor, resulting in low production efficiency and inconsistent cutting quality. Furthermore, the meat cutting industry suffers from poor working conditions, high labor intensity, and a high risk of injury, leading to recruitment difficulties. Therefore, the shift from manual labor to mechanization and automation in the meat cutting industry is an inevitable trend. However, beef exhibits diverse shapes and sizes, with inconsistent dimensions and thicknesses, and different cuts possess varying physical properties, increasing the difficulty of automated beef cutting. In addition, the unstructured meat cutting environment further complicates beef identification.

[0004] All of the above factors will seriously affect the quality and precision of beef cutting, making it impossible to meet the requirements of automated production. Summary of the Invention

[0005] The purpose of this application is to provide a multi-adaptive intelligent beef cutting method and apparatus that can improve the cutting quality and precision of beef.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] In a first aspect, this application provides a multi-adaptive intelligent beef cutting method, which is applied to a beef cutting production line, and the cutting method includes:

[0008] When the beef to be cut on the beef cutting production line reaches the freshness detection position, the first RGB image of the beef to be cut is acquired;

[0009] The freshness of the beef to be cut is identified based on the first RGB image, and a freshness identification result is obtained; the freshness identification result is: fresh, acceptable, or spoiled;

[0010] When qualified beef to be cut on the beef cutting production line reaches the cutting position, a second RGB image and a depth image of the qualified beef to be cut are acquired; the qualified beef to be cut is beef whose freshness identification result is fresh or acceptable; the cutting position is located behind the freshness detection position.

[0011] The cutting trajectory and cutting parameters of qualified beef to be cut are determined based on the second RGB image and the depth image;

[0012] The beef slitting production line is controlled according to the cutting parameters and the cutting trajectory.

[0013] Optionally, the freshness of the beef to be sliced ​​is identified based on the first RGB image to obtain a freshness identification result, specifically including:

[0014] The first RGB image is corrected based on the intrinsic parameters and distortion parameters of the USB camera to obtain the corrected first RGB image; the USB camera is the camera that captures the first RGB image.

[0015] Based on the corrected first RGB image, an image classification deep learning algorithm is used to identify the freshness of the beef to be cut, and the freshness identification result is obtained.

[0016] Optionally, the cutting trajectory and cutting parameters of the qualified beef to be sliced ​​are determined based on the second RGB image and the depth image, specifically including:

[0017] The second RGB image and the depth image are aligned and registered to obtain the aligned and registered RGB image and depth image;

[0018] Based on the aligned and registered RGB images, an instance segmentation deep learning algorithm is used to obtain the category of qualified beef to be cut and the pixel coordinate information of the beef mask.

[0019] The cutting trajectory is determined based on the pixel coordinate information of the beef mask and the depth image after alignment and registration.

[0020] Determine the cutting parameters based on the category of qualified beef to be cut.

[0021] Optionally, the cutting trajectory is determined based on the pixel coordinate information of the beef mask and the aligned depth image, specifically including:

[0022] Based on the pixel coordinate information of the beef mask, the minimum bounding rectangle of the beef mask region is drawn using image processing methods.

[0023] Based on the cutting task, determine the cutting points on the two long sides of the minimum bounding rectangle; the cutting task includes equal-width cutting and fixed-width cutting.

[0024] Determine the midpoint of the corresponding dividing points on the two long sides as the cutting point;

[0025] Determine the pixel coordinates and cutting angle of each cutting point in the second RGB image; the cutting angle is the angle between the short side of the smallest bounding rectangle and the transmission direction of the conveyor belt of the beef slitting production line, and the x-axis direction is the transmission direction of the beef slitting production line.

[0026] The pixel coordinates of each cutting point in the second RGB image are converted into spatial plane coordinates; the spatial plane coordinates are the coordinates in the XOY plane of the world coordinate system, the XOY plane of the world coordinate system is the upper surface of the conveyor belt of the beef slicing production line, the X-axis of the world coordinate system is the transmission direction of the conveyor belt, the Y-axis of the world coordinate system is the direction perpendicular to the X-axis in the XOY plane, and the Z-axis of the world coordinate system is the direction perpendicular to the XOY plane.

[0027] Based on the aligned and registered depth image, determine the spatial height coordinates of each cutting point; the spatial height coordinates are the Z-axis coordinates of the world coordinate system.

[0028] Based on the spatial plane coordinates, spatial height coordinates, and cutting angle of each cutting point, a cutting trajectory is generated in ascending order of spatial plane coordinates.

[0029] Secondly, this application provides a multi-adaptive intelligent beef cutting device, the cutting device comprising: a beef cutting production line, an industrial control computer, a freshness detection module, and a target detection and positioning module;

[0030] The freshness detection module is mounted on the upper part of the beef cutting production line and is located at the freshness detection position of the beef cutting production line. The target detection and positioning module is set on one side of the beef cutting production line and is located at the cutting position of the beef cutting production line.

[0031] Both the freshness detection module and the target detection and positioning module are connected to the industrial control computer.

[0032] The freshness detection module is used to acquire the first RGB image of the beef to be cut on the beef cutting production line; the target detection and positioning module is used to acquire the second RGB image and depth image of the qualified beef to be cut on the beef cutting production line.

[0033] The industrial control computer is connected to the beef slicing production line;

[0034] The industrial control computer is used to control the beef cutting production line using the above-mentioned cutting method.

[0035] Optionally, the freshness detection module includes: a first profile, a matte acrylic sheet, a first photoelectric sensor, a circular light source, and a USB camera;

[0036] The first profile is installed on the beef cutting production line and is located at the freshness detection position of the beef cutting production line;

[0037] The matte acrylic sheet is wrapped around the outside of the profile. The USB camera and the circular light source are both installed on the inside of the top matte acrylic sheet, with the lens of the USB camera facing the beef slicing production line and the circular light source located on the outside of the USB camera.

[0038] The transmitter and receiver of the first photoelectric sensor are installed on both sides of the beef cutting production line and are located at the freshness detection position of the beef cutting production line.

[0039] Both the first photoelectric sensor and the USB camera are connected to the industrial control computer;

[0040] The industrial control computer is used to control the USB camera to capture a first RGB image when it receives the photoelectric signal from the first photoelectric sensor.

[0041] Optionally, the target detection and positioning module includes: a second profile, a camera mounting plate, a 3D camera, and a second photoelectric sensor;

[0042] The second profile is disposed on one side of the beef slitting production line and is located at the slitting position of the beef slitting production line;

[0043] The 3D camera is mounted on the second profile via the camera mounting plate;

[0044] The transmitting end and receiving end of the second photoelectric sensor are respectively installed on both sides of the beef slicing production line and located at the slicing position of the beef slicing production line;

[0045] Both the second photoelectric sensor and the 3D camera are connected to the industrial control computer;

[0046] The industrial control computer is used to control the 3D camera to capture a second RGB image and a depth image when it receives the photoelectric signal from the second photoelectric sensor, and to control the conveyor of the beef slicing production line to stop.

[0047] Optionally, the beef slicing production line includes: a conveyor, a conveyor belt, and a cutting task execution device;

[0048] The conveyor belt is mounted on the conveyor, and the conveyor is used to drive the conveyor belt.

[0049] The target detection and positioning module is mounted on the other side of the conveyor, and the cutting task execution device is mounted on the other side of the conveyor, and the cutting task execution device is located at the cutting position of the beef cutting production line;

[0050] The industrial control computer is connected to the control terminal of the conveyor and the control terminal of the cutting task execution device, respectively.

[0051] Optionally, the cutting task execution device includes: a six-axis robotic arm, a force sensor, a flange, a flat-end cutter, a pressure device, a spring, a fixing component, and a universal ball.

[0052] The six-axis robotic arm is mounted on the other side of the conveyor, the force sensor is located at the end of the six-axis robotic arm, and the six-axis robotic arm is connected to the handle of the flat-head cutter through the flange;

[0053] The pressure fixing device is located on one side of the six-axis robotic arm. One end of the spring is connected to the pressure fixing device, and the other end of the spring is connected to the top surface of the fixing member. The universal ball is located on the bottom surface of the fixing member. When the spring is in its natural state, the position of the universal ball is lower than the position of the blade of the flat-head cutter.

[0054] The force sensor is connected to the industrial computer, and the industrial computer is connected to the control end of the six-axis robotic arm;

[0055] The industrial control computer is used to control the six-axis robotic arm based on the cutting trajectory, cutting parameters, and feedback signals from the force sensor.

[0056] Optionally, in controlling the six-axis robotic arm based on the cutting trajectory, cutting parameters, and feedback signals from the force sensor, the industrial computer is specifically used for:

[0057] Based on the spatial position of the current cutting point, the cutting angle, and the cutting parameters, the six-axis robotic arm is controlled to drive the flat-head cutter to cut the current cutting point, and the feedback signal from the force sensor is monitored in real time during the cutting process; the spatial position of the current cutting point includes the spatial plane coordinates and spatial height coordinates of the current cutting point;

[0058] When the feedback signal is greater than the preset pressure threshold, the six-axis robotic arm is controlled to drive the flat-head cutter to stop cutting, and the six-axis robotic arm is controlled to drive the flat-head cutter to lift to a preset height, the preset height being not less than the depth of the current cutting point;

[0059] Determine whether the cutting points described in the cutting trajectory have been successfully cut;

[0060] If not completed, the next cutting point in the cutting trajectory will be taken as the current cutting point, and the process will return to the step of "controlling the six-axis robotic arm to drive the flat-head cutter to cut the current cutting point according to the spatial position, cutting angle and cutting parameters of the current cutting point, and monitoring the feedback signal of the force sensor in real time during the cutting process".

[0061] If completed, the conveyor is then controlled to operate.

[0062] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0063] This application provides a multi-adaptive intelligent beef cutting method and apparatus. The cutting method detects freshness by acquiring a first RGB image at the freshness detection position of the beef cutting production line to avoid cutting rotten beef and improve the quality of the cut beef. By acquiring a second RGB image and a depth image at the cutting position, and setting cutting parameters and cutting trajectory based on the second RGB image and the depth image, combined with RGB information and depth information, the accuracy of beef cutting is improved.

[0064] This application also provides a multi-adaptive intelligent beef cutting device that applies the above-mentioned cutting method to a beef cutting production line to achieve automated cutting. By setting a freshness detection module and a target detection and positioning module at the freshness detection position and the cutting position of the beef cutting production line, the device acquires a first RGB image at the freshness detection position and a second RGB image and depth image at the cutting position. The industrial control computer then uses the above-mentioned beef cutting method to achieve automatic control of the beef cutting production line, thus meeting the requirements of automated production. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a flowchart illustrating a multi-adaptive intelligent beef cutting method according to an embodiment of this application. Figure 2 A schematic diagram illustrating the determination of cutting points according to an embodiment of this application.

[0067] Figure 3 This is a schematic diagram of the structure of a multi-adaptive intelligent beef cutting device provided in an embodiment of this application. Figure 4 This is a schematic diagram of the structure of a cutting task execution device provided in an embodiment of this application.

[0068] Figure 5 This is a schematic diagram illustrating the control principle of a multi-adaptive intelligent beef slicing device according to an embodiment of this application.

[0069] Figure 6 This is a flowchart of a freshness detection process provided in an embodiment of this application.

[0070] Figure 7 This is a flowchart of a target detection and localization process provided in an embodiment of this application.

[0071] Figure 8 This is a flowchart illustrating the cutting task execution process provided in one embodiment of this application.

[0072] Figure 9 This is a schematic diagram of the interface of an industrial control computer provided in one embodiment of this application.

[0073] Explanation of reference numerals in the attached figures:

[0074] 1-1 Material Transfer Module; 1-2 Freshness Detection Module; 1-3 Target Detection and Positioning Module; 1-4 Cutting Task Execution Module; 1. Conveyor; 2. Conveyor Belt; 3. Beef to be Cut; 4. Matte Acrylic Sheet; 5. First Photoelectric Sensor; 6. Circular Light Source; 7. USB Camera; 8. Industrial Computer; 9. Camera Mounting Plate; 10. 3D Camera; 11. Second Photoelectric Sensor; 12. Six-Axis Robotic Arm; 13. Force Sensor; 14. Flange; 15. Flat-Ended Knife; 16. Pressure Fixing Device; 17. Fixing Component; 18. Spring; 19. Omnidirectional Ball. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0077] In one exemplary embodiment, such as Figure 1 As shown, a multi-adaptive intelligent beef cutting method is provided. The cutting method is applied to a beef cutting production line and includes steps 101-105.

[0078] Step 101: When the beef to be cut on the beef cutting production line reaches the freshness detection position, acquire the first RGB image of the beef to be cut.

[0079] Step 102: Identify the freshness of the beef to be cut based on the first RGB image to obtain a freshness identification result; the freshness identification result is: fresh, acceptable, or spoiled.

[0080] Step 103: When the qualified beef to be cut on the beef cutting production line reaches the cutting position, acquire the second RGB image and depth image of the qualified beef to be cut; the qualified beef to be cut is the beef whose freshness identification result is fresh or acceptable; the cutting position is located behind the freshness detection position.

[0081] Step 104: Determine the cutting trajectory and cutting parameters of qualified beef to be cut based on the second RGB image and the depth image.

[0082] Step 105: Control the beef slitting production line according to the cutting parameters and the cutting trajectory.

[0083] In another exemplary embodiment of this application, step 102 may be implemented using the following steps:

[0084] The first RGB image is corrected based on the intrinsic parameters and distortion parameters of the USB camera to obtain the corrected first RGB image; the USB camera is the camera that captures the first RGB image.

[0085] Based on the corrected first RGB image, an image classification deep learning algorithm is used to identify the freshness of the beef to be sliced, obtaining a freshness identification result. In some embodiments, the steps for using an image classification deep learning algorithm to identify the freshness of the beef to be sliced ​​and obtain a freshness identification result can be: setting a classification loss function, constructing and training a deep neural network model, and using the trained deep neural network model for freshness identification. The specific training process and the setting of the binary loss function can be set as needed, and will not be elaborated here.

[0086] The correction step can be completed by the host computer. The USB camera takes a picture of the beef to be cut to obtain the first RGB image and sends it to the host computer. The host computer performs image correction on the captured image according to the intrinsic parameters and distortion parameters of the USB camera. After image correction, the industrial control computer uses an image classification deep learning algorithm to identify the freshness of different parts of the beef in the corrected image, saves the identification results in the memory of the industrial control computer, counts the beef of different freshness, and finally displays the identification and counting results on the human-computer interaction interface of the industrial control computer.

[0087] The aforementioned camera intrinsic parameters include: camera lens focal length parameter f. x and f y Principal point parameter C x and C y The distortion parameters include radial distortion parameters k1, k2, and k3, and tangential distortion parameters p1 and p2. The specific camera intrinsic parameter matrix is ​​Camera. intrinsics and distortion parameter matrix Camera distortion for:

[0088]

[0089] Camera distortion = [k1, k2, p1, p2, k3] (2)

[0090] In another exemplary embodiment of this application, step 104 of this application may be implemented using the following steps.

[0091] The second RGB image and the depth image are aligned and registered to obtain aligned and registered RGB images and depth images. Based on the aligned and registered RGB images, an instance segmentation deep learning algorithm is used to obtain the category of qualified beef to be sliced ​​and the beef mask pixel coordinate information. The cutting trajectory is determined based on the beef mask pixel coordinate information and the aligned and registered depth image. Cutting parameters are determined based on the category of qualified beef to be sliced. In some cases, obtaining the category of qualified beef to be sliced ​​and the beef mask pixel coordinate information using an instance segmentation deep learning algorithm based on the aligned and registered RGB images can be achieved through the following steps: setting a classification loss function and a target recognition loss function, constructing and training a deep neural network model, and using the trained deep neural network model to determine the category and beef mask pixel coordinate information.

[0092] The step of determining the cutting trajectory based on the beef mask pixel coordinate information and the registered depth image includes: drawing the minimum bounding rectangle of the beef mask area using image processing methods based on the beef mask pixel coordinate information; determining the cutting points on the two long sides of the minimum bounding rectangle according to the cutting task; the cutting task includes equal-width cutting and fixed-width cutting; determining the midpoint of the corresponding cutting points on the two long sides as the cutting point; determining the pixel coordinates and cutting angle of each cutting point in the second RGB image; the cutting angle is the angle between the direction of the short side of the minimum bounding rectangle and the transmission direction of the conveyor belt of the beef cutting production line, and the x-axis direction is the transmission direction of the beef cutting production line; and displaying the pixel coordinates and cutting angle of each cutting point in the second RGB image. The pixel coordinates in the image are converted into spatial plane coordinates. The spatial plane coordinates are the coordinates in the XOY plane of the world coordinate system, where the XOY plane is the upper surface of the conveyor belt of the beef slicing production line. The X-axis of the world coordinate system is the transmission direction of the conveyor belt, the Y-axis is the direction perpendicular to the X-axis in the XOY plane, and the Z-axis is the direction perpendicular to the XOY plane. Based on the aligned and registered depth image, the spatial height coordinates of each cutting point are determined. The spatial height coordinates are the Z-axis coordinates of the world coordinate system. Based on the spatial plane coordinates, spatial height coordinates, and cutting angle of each cutting point, a cutting trajectory is generated in ascending order of spatial plane coordinates.

[0093] In this application embodiment, two cutting tasks are set: equal-width cutting and fixed-width cutting, such as... Figure 2 As shown, Figure 2 (a) in the image is the original image. Figure 2 (b) in the diagram is a schematic diagram of the equal-width segmentation result. Figure 2(c) Schematic diagram of fixed-width segmentation result. In equal-width segmentation tasks, users can customize the number of segments; in fixed-width segmentation tasks, users can customize the segmentation width. Based on the segmentation task requirements input by the user on the human-machine interface, the industrial control computer uses OpenCV to calculate the pixel coordinates and segmentation angles of the equal-width or fixed-width segmentation points on the aforementioned minimum bounding rectangle. Simultaneously, based on the pixel coordinates of the segmentation points, it obtains the depth information of the segmentation points on the aligned and registered depth map.

[0094] like Figure 2 As shown in (b), the operation flow of the above equal-width slicing task is as follows: First, determine the long side and short side of the rectangle based on the pixel coordinates of the four vertices of the minimum bounding rectangle; then, divide the long side by the number of slices to obtain the pixel size of the sliced ​​beef width; next, calculate the pixel coordinates of each equal-width slicing point along the two long sides of the minimum bounding rectangle; finally, calculate the pixel coordinates of the midpoint and angle information of the corresponding equal-width slicing point based on the pixel coordinates of the two long sides corresponding to the equal-width slicing points.

[0095] like Figure 2 As shown in (c), the operation flow of the fixed-width slicing task is as follows: First, based on the pixel coordinates of the four vertices of the minimum bounding rectangle, determine the long side and short side of the rectangle, as well as the angle between the long side and the x-axis; then, based on the angle between the long side and the x-axis and the slicing width value, sequentially accumulate the slicing width value along the two long sides of the minimum bounding rectangle, as the pixel coordinates of the fixed-width slicing point, until it exceeds the range of the long side; finally, based on the pixel coordinates of the fixed-width slicing points corresponding to the two long sides, calculate the pixel coordinates and angle information of the midpoint of the corresponding fixed-width slicing point.

[0096] The midpoint pixel coordinates obtained in the above equal-width and fixed-width slicing tasks are the final cutting point pixel coordinates; the angle information is the rotation angle of the robotic arm end about the Z-axis.

[0097] The method described above for converting the pixel coordinates of each cutting point in the second RGB image to spatial coordinates is as follows: In practical applications, the robotic arm and 3D camera can be mounted with the eye outside the hand or with the eye on the hand. Using the checkerboard calibration method, hand-eye calibration is performed on the camera and robotic arm to obtain the transformation matrix between the pixel coordinate system and the spatial coordinate system. The transformation matrix consists of a rotation matrix and a translation vector. Based on the 3D camera's intrinsic parameter matrix, depth information, and extrinsic parameter transformation matrix, the pixel coordinates of the cutting points are converted to spatial coordinates.

[0098] The above-described method of generating cutting trajectories based on the spatial coordinates, depth, and cutting angle of each cutting point, in ascending order of spatial coordinates, is as follows: The spatial coordinates of all cutting points and their corresponding cutting angles and depths are stored in the industrial control computer's memory. The industrial control computer arranges the spatial coordinates in ascending order and plans the cutting trajectory based on the spatial coordinates, cutting angle, and depth: First, based on the spatial coordinates and depth information of the cutting points, the robot arm's end effector is planned to move above cutting point 1, exceeding the depth of cutting point 1; next, based on the cutting angle, the robot arm's end effector is rotated around the Z-axis by a certain angle; then, the robot arm's end effector drives the flat-head cutter downwards to cut; finally, the robot arm is raised upwards to a certain height above cutting point 1. The industrial control computer repeats the above cutting trajectory planning for each cutting point until the trajectory planning for all cutting points is completed.

[0099] In another exemplary embodiment of this application, step 105 may be implemented using the following steps:

[0100] The industrial control computer transmits all motion information after trajectory planning to the robotic arm. In addition, according to the type of beef, the industrial control computer adjusts the cutting parameters to the optimal cutting parameters corresponding to that type of beef and transmits them to the robotic arm, controlling the robotic arm to perform personalized cutting of equal or fixed width.

[0101] In one exemplary embodiment, such as Figure 3 As shown, a multi-adaptive intelligent beef cutting device is provided, including: a beef cutting production line, an industrial control computer 8, a freshness detection module 1-2, and a target detection and positioning module 1-3. The beef cutting production line includes: a material transfer module 1-1 and a cutting task execution module 1-4.

[0102] Material conveying mechanism 1-1 is installed in front of freshness detection module 1-2. Material conveying mechanism 1-1 includes a conveyor; conveyor belt 2 is mounted on conveyor 1. Conveyor belt 2 is made of food-grade PU material. The surface of conveyor belt 2 has a diamond-shaped texture to increase the friction between conveyor belt 2 and beef 3 to be cut. Beef 3 to be cut is placed on conveyor belt 2 and conveyed forward.

[0103] The freshness detection mechanism 1-2 includes: a first profile, a matte acrylic sheet 4, a first photoelectric sensor 5, a circular light source 6, and a USB camera 7. Its frame (i.e., the first profile) is constructed from multiple profiles into a cubic shape and is installed directly above the conveyor 1-1. The freshness detection mechanism 1-2 is surrounded by five matte acrylic sheets 4, forming a dark chamber. The bottom of the matte acrylic sheets 4 is 5cm away from the conveyor belt to allow the beef 3 to be cut on the conveyor belt 2 to move forward normally. The first photoelectric sensor 5 is installed on both sides of the conveyor 1, at the center of the dark chamber, to determine whether the beef 3 to be cut has reached the freshness detection position and to send a detection signal (i.e., a photoelectric signal) to the industrial control computer 8 in real time. Inside the freshness detection unit 1-2, there is a horizontal profile. A circular light source 6 and a USB camera 7 are fixed on the horizontal profile, with the lenses of the circular light source 6 and the USB camera 7 facing the conveyor belt 2. The circular light source 6 has a specification of 72W white light and Ra of 98. The USB camera 7 is located at the center of the circular light source 6 and is used to collect images of the beef 3 to be cut on the conveyor belt 2. The center of the circular light source 6, the lens of the USB camera 7, and the first photoelectric sensor 5 are all in a straight line to ensure that the beef 3 to be cut is within the field of view of the USB camera 7.

[0104] The target detection and positioning mechanism 1-3 includes a second profile, a camera mounting plate 9, a 3D camera 10, and a second photoelectric sensor 11. Its frame (i.e., the second profile) is constructed from two profiles and is installed on one side of the conveyor 1, behind the freshness detection mechanism 1-2. The camera mounting plate 9 is fixed to the side of the horizontal profile, and the 3D camera 10 is fixed to the camera mounting plate 9 with its lens facing the conveyor belt 2. It is used to perform category identification, spatial positioning, and depth information acquisition for the beef on the conveyor belt 2 that has been detected as fresh or acceptable. The second photoelectric sensor 11 is installed on both sides of the conveyor belt, directly below the lens of the 3D camera 10, ensuring that the beef that has been detected as fresh or acceptable is within the field of view of the 3D camera 10. The second photoelectric sensor 11 is used to determine whether the beef 3 that has been detected as fresh or acceptable has reached the target detection and positioning area (i.e., the cutting position) and sends a detection signal (photoelectric signal) to the industrial control computer 8 in real time.

[0105] like Figure 3 and Figure 4As shown, the cutting task execution mechanism 1-4 includes a six-axis robotic arm 12, a force sensor 13, a flange 14, a flat-head blade 15, a pressure device 16, a spring 18, a fixing component 17, and a universal ball 19. It is installed on the other side of the conveyor 1, in front of the target detection and positioning mechanism 1-3. The bottom of the six-axis robotic arm 12 is fixed. The force sensor 13 is installed at the end of the six-axis robotic arm 12, with a range of 500N, and is used to monitor the force value change during the cutting process in real time. The flange 14 is fixed on the force sensor 13, and the flat-head blade 15 is fixed on the flange 14. The cylindrical handles of the force sensor 13, flange 14, and flat-head blade 15 are coaxially fixedly connected, and are used to cut the beef 3 to be cut if the freshness detection is fresh or acceptable. The pressure fixing device 16 is fixed to the side of the cylindrical handle of the flat-head knife 15. Three springs 18 are fixed to the end of the pressure fixing device 16. The fixing member 17 is connected to the other side of the springs. Three universal balls 19 are fixed to the end of the fixing member 17. The bottom of the universal balls 19 is about 0.5cm away from the blade of the flat-head knife 15. They are used to fix and press the beef during the cutting process to reduce the slippage of the beef.

[0106] In this embodiment, the industrial control computer uses a multi-adaptive intelligent beef cutting method to control the beef cutting production line. The following section combines... Figure 5 A specific example is given to illustrate this process.

[0107] Step 1: Turn on conveyor 1 at material transfer mechanism 1-1, turn on the circular light source 6, USB camera 7, and first photoelectric sensor 5 at freshness detection mechanism 1-2, turn on 3D camera 10 and second photoelectric sensor 11 at target detection and positioning mechanism 1-3, turn on the six-axis robotic arm 12 and force sensor 13 at cutting task execution mechanism 1-4, and turn on industrial control computer 8. Place the beef to be cut 3 on conveyor belt 2 of conveyor 1 and transport it forward;

[0108] Step 2: Figure 6 The flowchart of the freshness detection process is as follows: Figure 6 As shown, the beef 3 to be cut on conveyor belt 2 enters the freshness detection mechanism 1-2 area. When the first photoelectric sensor 5 detects the beef 3 to be cut, the first photoelectric sensor 5 sends a photoelectric signal to the industrial control computer 8. After receiving the photoelectric signal, the industrial control computer 8 sends a photo-taking command to the USB camera 7, so that the USB camera 7 takes a picture of the beef 3 to be cut on conveyor belt 2. The photo-taking size of the USB camera 7 is set to 1280×720 pixels. After taking the picture, the first RGB image is uploaded to the industrial control computer 8. After receiving the first RGB image, the industrial control computer 8 performs image correction on the first RGB image according to the intrinsic parameters and distortion parameters of the USB camera 7.

[0109] The aforementioned USB camera 7 internal parameters include: camera lens focal length parameter f. xand f y Principal point parameter C x and C y The distortion parameters include radial distortion parameters k1, k2, and k3, and tangential distortion parameters p1 and p2. At a USB camera 7 image size of 1280×720 pixels, f... x =973.03, f y =972.66, C x =1025.09, C y =780.22, k1=0.10, k2=-0.04, k3=0, p1=0, p2=0, the intrinsic parameter matrix and distortion parameter matrix of the USB camera 7 are:

[0110]

[0111] Camera distortion =[0.10,-0.04,0,0,0] (2)

[0112] Step 3: After image correction, the industrial computer 8 uses the RT-BFE deep learning algorithm for image classification to identify the freshness of different parts of the beef 3 to be cut in the corrected first RGB image. The identification results are stored in the memory of the industrial computer 8. The freshness of the beef 3 to be cut on the conveyor 1 is then classified and counted. The freshness categories include three levels: fresh, acceptable, and spoiled. Finally, the industrial computer displays the identification and counting results on its human-machine interface to intuitively demonstrate the freshness identification results. Figure 9 The image shown is a schematic diagram of a human-computer interaction interface.

[0113] Step 4: Figure 7 A flowchart of the target detection and localization process provided by the present invention, such as Figure 7 As shown, after the freshness assessment is completed, the beef 3 to be cut enters the target detection and positioning mechanism 1-3 area on the conveyor belt 2. When the second photoelectric sensor 11 detects the beef 3 to be cut, it sends a photoelectric signal to the industrial control computer 8. After receiving the photoelectric signal, the industrial control computer 8 sends a stop signal to the conveyor 1. Based on the freshness assessment result in step 3, the industrial control computer 8 controls the 3D camera 10 to take pictures of the beef on the conveyor belt that is determined to be fresh or acceptable. The 3D camera 10 simultaneously acquires a second RGB image and a depth image, and uploads the acquired second RGB image and depth image to the industrial control computer 8. Finally, the industrial control computer 8 aligns and registers the second RGB image and the depth image, thereby achieving pixel-level matching between the RGB image and the depth image.

[0114] Step 5: After receiving the aligned and registered RGB image, the industrial control computer 8 uses an instance segmentation deep learning algorithm (exemplarily, in this example, the instance segmentation deep learning algorithm is the YOLOv8 algorithm) to obtain the category information of beef in the aligned and registered RGB image and the mask pixel coordinate information of beef in the aligned and registered RGB image. The beef categories include: tenderloin, brisket, and leg, etc. Finally, the industrial control computer 8 uses image processing methods such as threshold segmentation, morphological processing, and contour extraction to draw the minimum bounding rectangle in the beef mask portion of the aligned and registered RGB image and obtain the pixel coordinates of the four vertices of the minimum bounding rectangle.

[0115] Step 6: The human-machine interface of the industrial control computer 8 provides two cutting tasks: equal-width cutting and fixed-width cutting. In the equal-width cutting task, the user can customize the number of cuts; in the fixed-width cutting task, the user can customize the cutting width. Based on the cutting task requirements input by the user on the human-machine interface, the industrial control computer 8 uses OpenCV to calculate the pixel coordinates and cutting angle information of the equal-width or fixed-width cutting points on the minimum bounding rectangle image obtained in step 5. Simultaneously, based on the pixel coordinates of the cutting points, it obtains the depth information of the cutting points on the aligned and registered depth map. The specific determination method is the same as in the method embodiment and will not be repeated here.

[0116] Step 7: Figure 8 A flowchart for breaking down the task execution process, such as Figure 8 As shown, the six-axis robotic arm 12 and the 3D camera 10 are mounted with the eye outside the hand. Using the checkerboard calibration method, hand-eye calibration is performed on the 3D camera 10 and the six-axis robotic arm 12 to obtain the transformation matrix between the pixel coordinate system and the spatial coordinate system. The transformation matrix consists of a rotation matrix and a translation vector. Based on the intrinsic parameter matrix of the 3D camera, the depth information from step 6, and the extrinsic parameter transformation matrix of the 3D camera 10 from step 7, the final cutting point pixel coordinates obtained in step 6 are transformed to spatial coordinates.

[0117] Step 8: Store the spatial coordinates of all cutting points in Step 7 and their corresponding cutting angle and depth information from Step 6 in the memory of the industrial computer 8. The industrial computer 8 arranges the spatial coordinates in ascending order and plans the cutting trajectory based on the spatial coordinate information, cutting angle information, and depth information.

[0118] The cutting trajectory is planned as follows: First, based on the spatial coordinates and depth information of the cutting point, the flat-head cutter 15 at the end of the six-axis robotic arm 12 is moved above cutting point 1, with its height set to 5 cm more than the cutting point depth information obtained in step 6. Next, based on the cutting angle, the cutting angle value obtained by rotating the robotic arm end around the Z-axis is calculated. Then, the robotic arm end drives the flat-head cutter downwards to cut. Finally, the robotic arm is raised upwards to 5 cm above cutting point 1. The industrial control computer repeats the above cutting trajectory planning for each cutting point until the trajectory planning for all cutting points is completed.

[0119] Step 9: The industrial control computer 8 transmits all motion information after trajectory planning to the six-axis robotic arm 12. In addition, based on the beef category information obtained in Step 5, the industrial control computer 8 adjusts the cutting parameters to the optimal cutting parameters corresponding to that category of beef and transmits them to the six-axis robotic arm 12, controlling the six-axis robotic arm 12 to perform personalized slicing with equal or fixed width set by the user. During the cutting process, the force sensor 13 collects the cutting force signal in real time. When the force signal suddenly increases, it indicates that the cutting has reached the bottom, that is, the slicing of a single cutting point is completed. Then, the six-axis robotic arm 12 continues to execute the slicing of the next cutting point according to the trajectory planning in Step 8, until the slicing of all cutting points is completed.

[0120] Step 10: After the six-axis robotic arm completes the cutting task, the industrial control computer 8 controls the conveyor 1 to start, realizing the assembly line operation.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-adaptive intelligent beef cutting method, characterized in that, The cutting method is applied to a beef cutting production line, and the cutting method includes: When the beef to be cut on the beef cutting production line reaches the freshness detection position, the first RGB image of the beef to be cut is acquired; The freshness of the beef to be cut is identified based on the first RGB image, and a freshness identification result is obtained; the freshness identification result is: fresh, acceptable, or spoiled; When qualified beef to be cut on the beef cutting production line reaches the cutting position, a second RGB image and a depth image of the qualified beef to be cut are acquired; the qualified beef to be cut is beef whose freshness identification result is fresh or acceptable; the cutting position is located behind the freshness detection position. The cutting trajectory and cutting parameters of qualified beef to be cut are determined based on the second RGB image and the depth image; The beef slitting production line is controlled according to the cutting parameters and the cutting trajectory. The cutting trajectory and cutting parameters of qualified beef to be sliced ​​are determined based on the second RGB image and the depth image, specifically including: The second RGB image and the depth image are aligned and registered to obtain the aligned and registered RGB image and depth image; Based on the aligned and registered RGB images, an instance segmentation deep learning algorithm is used to obtain the category of qualified beef to be cut and the pixel coordinate information of the beef mask. The cutting trajectory is determined based on the pixel coordinate information of the beef mask and the depth image after alignment and registration. Determine the cutting parameters based on the category of qualified beef to be cut; The cutting trajectory is determined based on the pixel coordinate information of the beef mask and the depth image after alignment and registration, specifically including: Based on the pixel coordinate information of the beef mask, the minimum bounding rectangle of the beef mask region is drawn using image processing methods. Based on the cutting task, determine the cutting points on the two long sides of the minimum bounding rectangle; the cutting task includes equal-width cutting and fixed-width cutting. Determine the midpoint of the corresponding dividing points on the two long sides as the cutting point; Determine the pixel coordinates and cutting angle of each cutting point in the second RGB image; the cutting angle is the angle between the short side direction of the smallest bounding rectangle and the transmission direction of the conveyor belt of the beef slitting production line. The pixel coordinates of each cutting point in the second RGB image are converted into spatial plane coordinates; the spatial plane coordinates are the coordinates in the XOY plane of the world coordinate system, the XOY plane of the world coordinate system is the upper surface of the conveyor belt of the beef slicing production line, the X-axis of the world coordinate system is the transmission direction of the conveyor belt, the Y-axis of the world coordinate system is the direction perpendicular to the X-axis in the XOY plane, and the Z-axis of the world coordinate system is the direction perpendicular to the XOY plane. Based on the aligned and registered depth image, determine the spatial height coordinates of each cutting point; the spatial height coordinates are the Z-axis coordinates of the world coordinate system. Based on the spatial plane coordinates, spatial height coordinates, and cutting angle of each cutting point, a cutting trajectory is generated in ascending order of spatial plane coordinates.

2. The multi-adaptive intelligent beef cutting method according to claim 1, characterized in that, The freshness of the beef to be sliced ​​is identified based on the first RGB image, and a freshness identification result is obtained, specifically including: The first RGB image is corrected based on the intrinsic parameters and distortion parameters of the USB camera to obtain the corrected first RGB image; the USB camera is the camera that captures the first RGB image. Based on the corrected first RGB image, an image classification deep learning algorithm is used to identify the freshness of the beef to be cut, and the freshness identification result is obtained.

3. A multi-adaptive intelligent beef cutting device, characterized in that, The cutting device includes: a beef cutting production line, an industrial control computer, a freshness detection module, and a target detection and positioning module; The freshness detection module is mounted on the upper part of the beef cutting production line and is located at the freshness detection position of the beef cutting production line. The target detection and positioning module is set on one side of the beef cutting production line and is located at the cutting position of the beef cutting production line. Both the freshness detection module and the target detection and positioning module are connected to the industrial control computer. The freshness detection module is used to acquire the first RGB image of the beef to be cut on the beef cutting production line; the target detection and positioning module is used to acquire the second RGB image and depth image of the qualified beef to be cut on the beef cutting production line. The industrial control computer is connected to the beef slicing production line; The industrial control computer is used to control the beef cutting production line using the cutting method described in any one of claims 1-2.

4. The multi-adaptive intelligent beef cutting device according to claim 3, characterized in that, The freshness detection module includes: a first profile, a matte acrylic sheet, a first photoelectric sensor, a circular light source, and a USB camera; The first profile is installed on the beef cutting production line and is located at the freshness detection position of the beef cutting production line; The matte acrylic sheet is wrapped around the outside of the profile. The USB camera and the circular light source are both installed on the inside of the top matte acrylic sheet, with the lens of the USB camera facing the beef slicing production line and the circular light source located on the outside of the USB camera. The transmitter and receiver of the first photoelectric sensor are installed on both sides of the beef cutting production line and are located at the freshness detection position of the beef cutting production line. Both the first photoelectric sensor and the USB camera are connected to the industrial control computer; The industrial control computer is used to control the USB camera to capture a first RGB image when it receives the photoelectric signal from the first photoelectric sensor.

5. The multi-adaptive intelligent beef cutting device according to claim 3, characterized in that, The target detection and positioning module includes: a second profile, a camera mounting plate, a 3D camera, and a second photoelectric sensor; The second profile is disposed on one side of the beef slitting production line and is located at the slitting position of the beef slitting production line; The 3D camera is mounted on the second profile via the camera mounting plate; The transmitting end and receiving end of the second photoelectric sensor are respectively installed on both sides of the beef slicing production line and located at the slicing position of the beef slicing production line; Both the second photoelectric sensor and the 3D camera are connected to the industrial control computer; The industrial control computer is used to control the 3D camera to capture a second RGB image and a depth image when it receives the photoelectric signal from the second photoelectric sensor, and to control the conveyor of the beef slicing production line to stop.

6. The multi-adaptive intelligent beef cutting device according to claim 3, characterized in that, The beef slicing production line includes: a conveyor, a conveyor belt, and a cutting task execution device; The conveyor belt is mounted on the conveyor, and the conveyor is used to drive the conveyor belt. The target detection and positioning module is mounted on the other side of the conveyor, and the cutting task execution device is mounted on the other side of the conveyor, and the cutting task execution device is located at the cutting position of the beef cutting production line; The industrial control computer is connected to the control terminal of the conveyor and the control terminal of the cutting task execution device, respectively.

7. The multi-adaptive intelligent beef cutting device according to claim 6, characterized in that, The cutting task execution device includes: a six-axis robotic arm, a force sensor, a flange, a flat-end cutter, a pressure device, a spring, a fixing component, and a universal ball. The six-axis robotic arm is mounted on the other side of the conveyor, the force sensor is located at the end of the six-axis robotic arm, and the six-axis robotic arm is connected to the handle of the flat-head cutter through the flange; The pressure fixing device is located on one side of the six-axis robotic arm. One end of the spring is connected to the pressure fixing device, and the other end of the spring is connected to the top surface of the fixing member. The universal ball is located on the bottom surface of the fixing member. When the spring is in its natural state, the position of the universal ball is lower than the position of the blade of the flat-head cutter. The force sensor is connected to the industrial computer, and the industrial computer is connected to the control end of the six-axis robotic arm; The industrial control computer is used to control the six-axis robotic arm based on the cutting trajectory, cutting parameters, and feedback signals from the force sensor.

8. The multi-adaptive intelligent beef cutting device according to claim 7, characterized in that, In controlling the six-axis robotic arm based on the cutting trajectory, cutting parameters, and feedback signals from the force sensor, the industrial computer is specifically used for: Based on the spatial position of the current cutting point, the cutting angle, and the cutting parameters, the six-axis robotic arm is controlled to drive the flat-head cutter to cut the current cutting point, and the feedback signal from the force sensor is monitored in real time during the cutting process; the spatial position of the current cutting point includes the spatial plane coordinates and spatial height coordinates of the current cutting point; When the feedback signal is greater than the preset pressure threshold, the six-axis robotic arm is controlled to drive the flat-head cutter to stop cutting, and the six-axis robotic arm is controlled to drive the flat-head cutter to lift to a preset height, the preset height being not less than the depth of the current cutting point; Determine whether the cutting points described in the cutting trajectory have been successfully cut; If not completed, the next cutting point in the cutting trajectory is taken as the current cutting point, and the process returns to the step of "controlling the six-axis robotic arm to drive the flat-head cutter to cut the current cutting point according to the spatial position, cutting angle and cutting parameters of the current cutting point, and monitoring the feedback signal of the force sensor in real time during the cutting process". If completed, the conveyor is then controlled to operate.

Images