An apparatus for automatically trimming defects of fish fillets and a control method thereof

Abstract

The application discloses a kind of equipment and control method for automatically fish slice defect finishing.The equipment includes fish slice single piece loading unit, image acquisition and processing unit and finishing unit.Fish slice single piece loading unit utilizes the action of gravity to disperse and load the fish slice overlapping to image acquisition area;Image acquisition and processing unit, image acquisition and processing unit are used to collect fish slice defect image, and real-time picture information is transmitted to industrial computer, and the fish slice defect detection network based on YOLOv8 model improved in the middle part of industrial computer is used to identify defect area and plan defect finishing track;Finishing unit receives track and posture information from industrial computer, and accurately finishes fish slice, so as to complete the process of fish slice loading, detection and finishing.The application replaces traditional manual finishing mode by automatic processing, realizes real-time finishing of fish slice defect, effectively improves the operation efficiency and product quality of fish slice processing.

Description

Technical Field

[0001] This invention relates to a fish fillet inspection device in the field of aquatic engineering technology, and particularly to an automated fish fillet defect repair device and its control method. Background Technology

[0002] Reducing labor costs by increasing the automation level of production processes has become one of the major challenges facing the food processing industry. Aquatic product processing is not only a natural extension of fishing and aquaculture, but also a key way to improve industrial value-added and efficiency. Currently, the fish processing industry largely still adopts a labor-intensive production model with low levels of mechanization and automation, and labor costs account for more than half of the total processing costs. Furthermore, the harsh working environment and high labor intensity of aquatic product processing pose safety hazards to processing personnel and product quality. Compared with traditional manual processing, automated processing can achieve continuous and stable production, ensure consistent product quality, and effectively prevent product contamination through non-human contact processing, thus guaranteeing food safety. Therefore, increasing the automation level of aquatic product processing is of great significance for promoting the sustainable development of the aquatic product processing industry.

[0003] The main processing steps for fish include slicing, skinning, disinfection, trimming, grading, and freezing. Among these, the trimming step is primarily manual, involving the removal of defects affecting the appearance of the fish fillets, such as fins, tails, abdominal membranes, and bruised flesh, using hand-held knives. Trimming not only improves the product's appearance but is also a crucial step in ensuring the final product's quality and safety. Accurately identifying the diverse types and shapes of defects on fish fillets presents a challenge for automated defect trimming. Currently, research on fish fillet defect detection is relatively limited. Furthermore, in actual fish processing, the color distribution of fish from different origins varies, and some fillets may have unremoved scales or reflective water stains in the background. These factors significantly impact the defect detection results. Therefore, a highly efficient and intelligent automated fish fillet defect trimming device is urgently needed. This is the fundamental way to improve my country's aquatic product processing level, achieving the goals of liberating productivity, improving safety, standardizing quality, and rationally allocating resources. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide an automated device and method for repairing defects in fish fillets, which can automatically identify and remove defective areas, greatly improving the processing efficiency of fish fillets.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] I. An automated device for repairing defects in fish fillets

[0007] The equipment includes a single-strip fish fillet feeding and conveying unit, a horizontal conveyor belt, a second drive motor, an image acquisition and processing unit, and a trimming unit. The second drive motor is connected to the horizontal conveyor belt, and the single-strip fish fillet feeding and conveying unit is connected to the input end of the horizontal conveyor belt. The image acquisition and processing unit and the trimming unit are arranged sequentially along the transmission direction of the horizontal conveyor belt. The image acquisition and processing unit is electrically connected to the single-strip fish fillet feeding and conveying unit, the trimming unit, and the second drive motor, respectively.

[0008] The single-strip fish fillet feeding and conveying unit includes a chain conveyor belt, height limiting baffles, and a first drive motor; the input end of the chain conveyor belt serves as the feeding end, the output end of the chain conveyor belt is connected to the input end of the horizontal conveyor belt, the first drive motor is connected to the chain conveyor belt, and multiple height limiting baffles are installed on the chain conveyor belt at intervals along its transmission direction.

[0009] The chain conveyor belt has at least three sections along the transmission direction. Each section of the conveyor belt is equipped with height limiting baffles arranged sequentially and at intervals along its transmission direction. The first section is a horizontal feeding section, the second section is an inclined feeding section, and the third section is an image unit feeding section, which is connected to the input end of the horizontal conveyor belt.

[0010] The image acquisition and processing unit includes a camera bracket, a depth camera, an industrial control computer, and a first photoelectric sensor. The depth camera is fixedly mounted above the horizontal conveyor belt via the camera bracket. The first photoelectric sensor is fixedly mounted on the side of the horizontal conveyor belt below the depth camera. Both the first photoelectric sensor and the depth camera are connected to the industrial control computer. The industrial control computer is also connected to the trimming unit, the second drive motor, and the single-strip fish fillet feeding conveyor unit.

[0011] The trimming unit includes a trimming mechanism, a series robotic arm, a robotic arm base, and a second photoelectric sensor. The robotic arm base is installed on the side of the horizontal conveyor belt, the series robotic arm is fixedly installed on the robotic arm base, and the trimming mechanism is installed at the end of the series robotic arm. Both the series robotic arm and the trimming mechanism are connected to the image acquisition and processing unit.

[0012] The trimming mechanism includes a flange, a fastener, a third drive motor, a disc cutter, a universal ball, and a shock-absorbing spring. The fastener is fixedly installed at the end of the robotic arm via the flange. A shock-absorbing spring is fixedly installed on one side of the lower surface of the fastener, and a universal ball is fixedly installed at the bottom of the shock-absorbing spring. The third drive motor is fixedly installed on the other side of the lower surface of the fastener, and the output shaft of the third drive motor is coaxially and fixedly connected to the disc cutter.

[0013] The surface of the chain conveyor belt is provided with a rough texture.

[0014] The disc cutter is also equipped with a disc cutter protective cover, which is fixedly mounted on the third drive motor.

[0015] II. A control method for an automated fish fillet defect repair device.

[0016] 1) When the first photoelectric sensor detects that the fish fillet to be processed has reached the image acquisition area, it sends a photoelectric signal to the industrial control computer. After receiving the photoelectric signal, the industrial control computer sends a conveyor belt stop signal to the horizontal conveyor belt and the chain conveyor belt, and at the same time controls the depth camera to acquire the image of the fish fillet to be processed and upload it to the industrial control computer.

[0017] 2) The industrial control computer receives the fish fillet defect image captured by the depth camera, uses deep learning algorithm to identify the defect area of ​​the fish fillet defect image, and then performs path and posture planning for the defect repair process based on the defect area, obtains the repair posture planning result and sends it to the serial robotic arm, and then sends the conveyor belt working signal to the horizontal conveyor belt and the chain plate conveyor belt.

[0018] 3) When the second photoelectric sensor detects that the fish fillet to be processed has reached the trimming area, it sends a photoelectric signal to the industrial control computer. After receiving the photoelectric signal, the industrial control computer sends a conveyor belt stop signal to the horizontal conveyor belt and the chain conveyor belt. Then, the serial robotic arm moves from the initial position to above the defects of the fish fillet according to the trimming posture planning result and trims all the defects of the fish fillet to be processed. After the trimming is completed, it returns to the initial position. Finally, the horizontal conveyor belt continues to run, transporting the trimmed fish fillet to the target position.

[0019] In the industrial control computer, a deep learning algorithm is used to identify the defective regions in the fish fillet defect image, specifically:

[0020] The industrial control computer is equipped with a well-quantized fish fillet defect detection network. After using the fish fillet defect detection network to detect defects in the fish fillet defect image, the defect area of ​​the fish fillet defect image is obtained.

[0021] The fish fillet defect detection network is an improved YOLOv8 neural network, which is obtained by improving the backbone network, neck network, and loss function of the YOLOv8 neural network. In the backbone network, a coordinate attention mechanism module is added after the last SPPF module. In the neck network, the GSConv module and the VoV-GSCSP module are combined and the original Neck framework of the YOLOv8 neural network is replaced with the Slim-Neck framework. In the loss function, MPDIoU is used instead of the original CIoU loss function.

[0022] In the industrial control computer, path and posture planning for the defect repair process are performed based on the defect area to obtain the repair posture planning result, specifically:

[0023] First, threshold segmentation and Canny edge detection are performed based on the mask colors of different labels in the defect area to obtain the mask contours of different defects and the mask contours of the fish fillet itself. Then, the paths of different defects are connected through logical operations and morphological operations to obtain the final adjustment pose planning result.

[0024] This invention addresses the problems of manual operation required for fish fillet defect trimming and the limited features and poor robustness of traditional fish fillet defect segmentation methods. It proposes a fish fillet trimming device and an efficient and lightweight fish fillet defect segmentation method to achieve automation and intelligence in fish fillet trimming.

[0025] The beneficial effects of this invention are:

[0026] (1) This invention discloses an automated fish fillet defect trimming device. Structurally, it is based on a series of robotic arms with rotational degrees of freedom, enabling the trimming of defects such as fish belly membranes, fins, and tails on a plane. Through a fixed pressure module combining shock-absorbing springs and universal balls, the slippage of fish fillets during processing is effectively reduced, thereby reducing the actual error of the actual trimming path.

[0027] (2) This invention discloses a method for segmenting fish fillet defects based on the aforementioned equipment. This method enhances the robustness and universality of the defect identification model through data augmentation and the use of a large-scale dataset. By introducing an attention mechanism, replacing the original model's neck network, and optimizing the loss function, the model's segmentation accuracy and processing speed for the trimmed targets are significantly improved. Furthermore, model quantization achieves model lightweighting, facilitating effective deployment on edge devices.

[0028] (3) This invention also develops a method for path planning based on the above-mentioned defect repair and segmentation results. The repair path is extracted through image processing operations to ensure that the path is similar to the manual operation. At the same time, combined with hand-eye calibration technology, efficient automatic repair of fish fillet defects is achieved. Attached Figure Description

[0029] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0030] Figure 1 This is a schematic diagram of the device of the present invention.

[0031] Figure 2 This is a schematic diagram of the end-effector trimming module of the robotic arm of the present invention.

[0032] Figure 3 This is a schematic diagram of the process for extracting the defect repair path for fish fillets according to the present invention.

[0033] Figure 4 This is a schematic diagram of the improved YOLOv8 network structure of this invention.

[0034] Figure 5 This is a schematic diagram of the coordinate attention mechanism module used in this invention.

[0035] Figure 6 This is a schematic diagram of the VoV-GSCSP module used in this invention.

[0036] In the diagram: 1-1. Single fish fillet feeding and conveying unit; 1-2. Image acquisition and processing unit; 1-3. Trimming unit; 1. Chain conveyor belt; 2. Side baffle; 3. Height limit baffle; 4. First drive motor; 5. Camera bracket; 6. Depth camera; 7. Industrial computer; 8. Trimming mechanism; 9. Serial robotic arm; 10. Robotic arm base; 11. Second drive motor; 12. Horizontal conveyor belt; 13. First photoelectric sensor; 14. Second photoelectric sensor; 15. Discharge hopper; 16. Robotic arm end; 17. Flange; 18. Fixing component; 19. Third drive motor; 20. Disc cutter; 21. Disc cutter protective cover; 22. Universal ball; 23. Shock-absorbing spring. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, the equipment includes a single-slice feeding and conveying unit 1-1, a horizontal conveyor belt 12, a second drive motor 11, a discharge hopper 15, an image acquisition and processing unit 1-2, and a trimming unit 1-3. The second drive motor 11 is connected to the horizontal conveyor belt 12 and is used to control the movement of the horizontal conveyor belt 12. The single-slice feeding and conveying unit 1-1 is connected to the input end of the horizontal conveyor belt 12, and the output end of the horizontal conveyor belt 12 is connected to the discharge hopper 15 for collecting trimmed fish slices. The image acquisition and processing unit 1-2 and the trimming unit 1-3 are arranged on the side of the horizontal conveyor belt 12, and are arranged sequentially along the transmission direction of the horizontal conveyor belt 12. The industrial control computer 7 of the image acquisition and processing unit 1-2 is electrically connected to the first drive motor 4 of the single-slice feeding and conveying unit 1-1, the trimming unit 1-3, and the second drive motor 11, respectively. In specific implementations, the discharge hopper 15 can be the input end of the next-level processing module. The discharge hopper is equipped with a pressurized water pipe for washing and separating fish fillets and waste. The horizontal conveyor belt 12 is made of stainless steel chain plates.

[0039] The single-strip fish fillet feeding and conveying unit 1-1 includes a chain conveyor belt 1, a side baffle 2, a height limiting baffle 3, and a first drive motor 4. The input end of the chain conveyor belt 1 serves as the feeding end, and the output end of the chain conveyor belt 1 is connected to the input end of the horizontal conveyor belt 12. The first drive motor 4 is connected to the chain conveyor belt 1 and is used to control the movement of the chain conveyor belt 1. The surface of the chain conveyor belt 1 is provided with drainage holes to drain the wastewater carried by the fish fillets during the lifting process, thereby reducing water stains on the surface of the fish fillets that enter the defect detection field of view. When the inclination angle of the second lifting stage of the single-strip fish fillet feeding conveyor unit 1-1 is large, sufficient friction cannot be generated between the chain plate and the fish fillet to lift it. Therefore, height-limiting baffles 3 are installed between each section of the chain plate to accommodate the fish fillets. The height-limiting baffles 3 form slots, and the spacing between the slots is the length of the fish fillet. The height-limiting baffles 3 are determined according to the lifting inclination angle; the greater the inclination angle of the lifting conveyor belt, the higher the corresponding height-limiting baffles 3. Specifically, multiple height-limiting baffles 3 are installed on the chain plate conveyor belt 1, arranged sequentially and at intervals along its transmission direction. Side baffles 2 are fixedly installed on both sides of the chain plate conveyor belt 1 to prevent the fish fillets from falling from the sides. The surface of the chain plate conveyor belt 1 is provided with rough texture to increase the friction between the fish fillets and the conveyor belt.

[0040] The chain conveyor belt 1 has at least three sections along the transmission direction. Each section is equipped with height-limiting baffles 3 arranged at intervals along its transmission direction. The height of the height-limiting baffles 3 is slightly greater than the maximum thickness of the fish fillet cross-section, and the fixed interval is the length of the longitudinal main shaft of the fish fillet. Meanwhile, the running speed of the horizontal conveyor belt 12 is slightly higher than that of the chain conveyor belt 1. The first section is a horizontal feeding section, used to pour fish fillets into the feeding hopper. The second section is an inclined feeding section, which uses gravity to cause excess fish fillets from the upper section to fall into the height-limiting baffle grooves below, thus separating overlapping fish fillets. This ensures that at most one fish fillet can be placed between two adjacent height-limiting baffles 3 at the image unit feeding section, guaranteeing flatness of the fish fillets and preventing them from sticking together and overlapping, which would affect defect detection. The third section is the image unit feeding section, which is connected to the input end of the horizontal conveyor belt 12.

[0041] The image acquisition and processing unit 1-2 includes a camera bracket 5, a depth camera 6, an industrial control computer 7, and a first photoelectric sensor 13. The depth camera 6 is fixedly mounted on the horizontal conveyor belt 12 near its input end via a quick-release plate and the camera bracket 5, ensuring that the fish fillet is within the field of view of the depth camera 5. The depth camera uses a nine-point calibration method with the eye outside the hand; the nine-point calibration method involves moving the center point of the bottom of the end-effector of the robotic arm along a grid pattern on a plane parallel to the working plane, recording the coordinates of the moving points of the robotic arm, and simultaneously taking pictures. Halcon software is used to obtain the pixel coordinates of the nine points in the image taken by the depth camera, and the hand-eye calibration parameters are calculated. Finally, the transformation matrix between the workpiece coordinate system and the camera coordinate system of the end-effector of the robotic arm is obtained. The first photoelectric sensor 13 is fixedly mounted on the side of the horizontal conveyor belt 12 below the depth camera 6, and the positions of the depth camera 6 and the first photoelectric sensor 13 are vertically corresponding. Both the first photoelectric sensor 13 and the depth camera 6 are connected to the industrial control computer 7. The depth camera 6 communicates with the industrial control computer 7 via the USB 3.0 transmission protocol to acquire images. The industrial control computer 7 is also connected to the trimming mechanism 8 of the trimming unit 1-3, the serial robotic arm 9, the second drive motor 11, the second photoelectric sensor 14, the second drive motor 11, and the first drive motor 4 of the single fish fillet feeding and conveying unit 1-1. The industrial control computer 7 sends the processed data to the trimming unit 1-3 in hexadecimal format based on the Modbus TCP protocol. The industrial control computer 7 is used to process the fish fillet defect information and depth information captured by the depth camera 6, and also to control the end effector 16 of the robotic arm to drive the trimming mechanism 8 of the end effector 16 to trim the fish fillet defects according to the planned path and posture.

[0042] The trimming unit 1-3 includes a trimming mechanism 8, a series robotic arm 9, a robotic arm base 10, and a second photoelectric sensor 14. The robotic arm base 10 is installed on the side of the horizontal conveyor belt 12, and the series robotic arm 9 is fixedly installed on the robotic arm base 10. The trimming mechanism 8 is installed at the end 16 of the robotic arm 9. Both the series robotic arm 9 and the trimming mechanism 8 are connected to the industrial control computer 7 of the image acquisition and processing unit 1-2. The trimming mechanism 8 is located on the horizontal conveyor belt 12. In specific implementation, the main body of the series robotic arm 9 is a four-degree-of-freedom body, ensuring that the trimming area is located within the working space of the robotic arm. The end of the robotic arm has four degrees of freedom: translation along the X, Y, and Z axes and rotation around the Z axis, which are used to drive the trimming mechanism to perform trimming operations on the plane angle of the conveyor belt, so as to remove defects in fish fillets of arbitrary shapes.

[0043] like Figure 2As shown, the trimming mechanism 8 includes a flange 17, a fastener 18, a third drive motor 19, a disc cutter 20, a disc cutter protective cover 21, a ball joint 22, and a shock-absorbing spring 23. The fastener 18 is fixedly mounted on the end of the robotic arm 16 via the flange 17. A shock-absorbing spring 23 is fixedly mounted on one side of the lower surface of the fastener 18, and a ball joint 22 is fixedly mounted on the bottom of the shock-absorbing spring 23. The ball joint 22 and the shock-absorbing spring 23 form a pressure-fixing module to fix the fish fillet and prevent it from moving. The third drive motor 19 is fixedly mounted on the other side of the lower surface of the fastener 18, and the output shaft of the third drive motor 19 is coaxially fixedly connected to the disc cutter 20. A disc cutter protective cover 21 is also installed on the outside of the disc cutter 20 to prevent debris from flying. The disc cutter protective cover 21 is fixedly mounted on the third drive motor 19. The disc cutter 20 is positioned between the third drive motor 19 and the shock-absorbing spring 23.

[0044] A control method for an automated fish fillet defect trimming device includes the following steps:

[0045] 1) When the first photoelectric sensor 13 senses that the fish fillet to be processed has reached the image acquisition area, it sends a photoelectric signal to the industrial control computer 7. After receiving the photoelectric signal, the industrial control computer 7 sends a conveyor belt stop signal to the horizontal conveyor belt 12 and the chain plate conveyor belt 1, and at the same time sends a signal to open the depth camera. After receiving the signal to open, the depth camera 6 controls the depth camera 6 to acquire the image of the fish fillet to be processed and upload it to the industrial control computer 7.

[0046] 2) The industrial control computer 7 receives the fish fillet defect image collected by the depth camera 6, uses the deployed quantized deep learning algorithm to identify the defect area of ​​the fish fillet defect image in real time, and then performs path and posture planning for the defect repair process based on the defect area, obtains the repair posture planning result and sends it to the serial robotic arm 9, and then sends the conveyor belt working signal to the horizontal conveyor belt 12 and the chain plate conveyor belt 1.

[0047] 3) When the second photoelectric sensor 14 senses that the fish fillet to be processed has reached the trimming area, it sends a photoelectric signal to the industrial control computer 7. After receiving the photoelectric signal, the industrial control computer 7 sends a conveyor belt stop signal to the horizontal conveyor belt 12 and the chain plate conveyor belt 1. Then, the robotic arm 9 moves from the initial position to above the defects of the fish fillet according to the trimming posture planning result and trims all the defects of the fish fillet to be processed. After the trimming is completed, it returns to the initial position. Finally, the horizontal conveyor belt 12 and the chain plate conveyor belt 1 continue to run, and the horizontal conveyor belt 12 transports the trimmed fish fillet to the target position (i.e., the discharge hopper 15).

[0048] Among them, the industrial control computer 7 uses deep learning algorithms to identify the defect areas in the fish fillet defect image, specifically:

[0049] The industrial control computer 7 is equipped with a quantized fish fillet defect detection network. This network is used to detect defects in the fish fillet defect image, thus obtaining the defect areas. For example... Figure 4 As shown, the fish fillet defect detection network is an improved YOLOv8 neural network, obtained by modifying the backbone network, neck network, and loss function of the YOLOv8 neural network. In the backbone network, a coordinate attention mechanism module is added after the final SPPF module to improve the feature extraction capability of the YOLOv8 model. This helps the model to understand complex scenes containing small objects more deeply, thus significantly improving prediction performance without increasing computational requirements. The network structure diagram of the coordinate attention mechanism module is shown below. Figure 5 As shown, the Neck network incorporates a combination of the GSConv and VoV-GSCSP modules, replacing the original Neck framework with a Slim-Neck framework. GSConv is a hybrid convolution based on the standard convolutional module SC, channel sparse convolutional computation DSC, and a shuffle operation. Building upon GSConv, the VoV-GSCSP module, a cross-level partial network (GSCSP), is designed using Gsbottleneck and a one-time aggregation method. This not only reduces computational and network structure complexity but also maintains sufficient accuracy. Based on GSConv and VoV-GSCSP, an efficient Slim-Neck architecture is constructed and replaces the original YOLOv8 Neck network. This achieves a lightweight design for the model neck and efficient feature fusion, reducing redundant and repetitive information in the feature maps. This accelerates inference speed without affecting model complexity and size, reduces computational costs, and facilitates deployment on subsequent edge computing devices. The network structure diagram of VoV-GSCSP is shown below. Figure 6 As shown; in the loss function Loss s, MPDIoU is used instead of the original CIoU loss function to calculate the bounding box loss function, which improves the detection accuracy of the model. The formula is as follows:

[0050]

[0051]

[0052]

[0053] In the formula, A and B represent two superimposed convex hulls. and Let A represent the coordinates of the upper left and lower right points of the convex hull A, respectively. and Let these represent the coordinates of the upper left and lower right points of the convex hull B, respectively. and Let w and h represent the squares of the distances between the top left and bottom right points of the two convex hulls, respectively, and let w and h be the width and height of the input image, respectively.

[0054] In specific implementation, such as Figure 3 As shown, the depth camera 6 is placed 70cm above the horizontal conveyor belt 12 and connected to the industrial control computer 7 via a data cable. The camera acquires image data of fish fillet defects, and the image size is cropped to 640×640 pixels to obtain the original dataset. The dataset is then randomly divided into training set and test set at a ratio of 2:1.

[0055] Data augmentation operations on the training set included blurring, brightness transformation, rotation, translation, and scaling. Then, polygons were created using Labelme to annotate various fish fillet defects, and corresponding labels were assigned, resulting in a JSON-formatted annotation file. This JSON file was then converted to PNG images as ground truth, completing the construction of datasets for different fish fillet defects. Finally, the improved YOLOv8 neural network was trained on the datasets for different fish fillet defects, specifically with 300 training iterations, a learning rate of 0.01, a minimum learning rate of 0.001, a batch size of 32, a momentum of 0.937, and the SGD optimizer with a weight decay of 0.0005. After training using the PyTorch framework, the generated model's weight file (.pt format) was converted to the Open Neural Network Exchange (ONNX) format; the ONNX file was then quantized using the TensorRT inference engine with INT8 quantization.

[0056] The evaluation metric used during training is mean accuracy (mAP), which includes the segmentation accuracy of bounding boxes and masks, denoted as bbox_mAP and mask_mAP, respectively. The calculation method is as follows:

[0057]

[0058]

[0059]

[0060] Where P stands for Precision, which is the proportion of correctly detected defect pixels out of all detected defect pixels; TP represents the number of correctly predicted positive samples; and FP represents the number of incorrectly predicted negative samples. R stands for Recall, which is the proportion of correctly predicted defect pixels out of the number of defect pixels that should have been correctly detected; and FN represents the number of incorrectly predicted positive samples. mAP is the mean of the average precision across all categories, and N represents the total number of categories. P(R) is a curve formed by plotting Recall on the x-axis and Precision on the y-axis.

[0061] In this embodiment, mAP@0.5:0.95 (IoU threshold of 0.5:0.95), floating-point operations per second (FLOPs), and inference frame rate (FPS) are used as indicators to evaluate the performance of the model. A higher mAP@0.5:0.95 indicates higher detection and segmentation accuracy, and a higher FPS indicates faster inference speed.

[0062] The industrial computer 7 performs path and posture planning for the defect repair process based on the defect area, and obtains the repair posture planning result, specifically:

[0063] First, threshold segmentation and Canny edge detection are performed based on the mask colors of different labels in the defect area to obtain different defect mask contours and the mask contour of the fish fillet itself. Next, the paths of different defects are connected through logical operations and morphological operations to obtain the final adjusted pose planning result. Specifically, a logical AND operation is used to obtain the intersection of each defect contour and the fish fillet mask contour; for each intersection, the corresponding adjusted pose is planned, and then a morphological dilation operation is used to connect the broken paths to obtain the initial adjusted pose planning result; finally, the path is refined using a skeleton extraction function to obtain the final adjusted pose planning result.

[0064] The proposed fish fillet defect identification method can be implemented in the following environment: Ubuntu operating system, PyTorch architecture, NVIDIA 4090 graphics card for computation, Python 3.8 language, and CUDA 11.3 environment. The model was trained, and performance was tested on the test set. The evaluation parameter of fish fillet segmentation mAP@0.5:0.95 was 73.8%, and the detection speed exceeded 243 FPS.

[0065] Finally, it should be noted that the above embodiments and descriptions are only used to illustrate the technical solutions of the present invention and not to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the disclosure of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. An automated device for trimming defects in fish fillets, characterized in that, It includes a single fish fillet feeding and conveying unit (1-1), a horizontal conveyor belt (12), a second drive motor (11), an image acquisition and processing unit (1-2), and a trimming unit (1-3); the second drive motor (11) is connected to the horizontal conveyor belt (12), the single fish fillet feeding and conveying unit (1-1) is connected to the input end of the horizontal conveyor belt (12), the image acquisition and processing unit (1-2) and the trimming unit (1-3) are arranged on the side of the horizontal conveyor belt (12), the image acquisition and processing unit (1-2) and the trimming unit (1-3) are arranged in sequence along the transmission direction of the horizontal conveyor belt (12), and the image acquisition and processing unit (1-2) is electrically connected to the single fish fillet feeding and conveying unit (1-1), the trimming unit (1-3) and the second drive motor (11) respectively; The image acquisition and processing unit (1-2) includes a camera bracket (5), a depth camera (6), an industrial computer (7), and a first photoelectric sensor (13). The depth camera (6) is fixedly installed above the horizontal conveyor belt (12) via the camera bracket (5). The first photoelectric sensor (13) is fixedly installed on the side of the horizontal conveyor belt (12) below the depth camera (6). The first photoelectric sensor (13) and the depth camera (6) are both connected to the industrial computer (7). The industrial computer (7) is also connected to the trimming unit (1-3), the second drive motor (11), and the fish fillet single-strip feeding conveyor unit (1-1). In the industrial control computer (7), a deep learning algorithm is used to identify the defective regions of the fish fillet defect image, specifically: The industrial control computer (7) is equipped with a well-quantized fish fillet defect detection network. After the fish fillet defect detection network is used to detect defects in the fish fillet defect image, the defect area of ​​the fish fillet defect image is obtained. The fish fillet defect detection network is an improved YOLOv8 neural network, which is obtained by improving the backbone network, neck network, and loss function of the YOLOv8 neural network. In the backbone network, a coordinate attention mechanism module is added after the last SPPF module. In the neck network, the GSConv module and the VoV-GSCSP module are combined, and the original Neck framework of the YOLOv8 neural network is replaced with the Slim-Neck framework. In the loss function, MPDIoU is used instead of the original CIoU loss function. In the industrial control computer (7), the path and posture planning of the defect repair process are performed according to the defect area to obtain the repair posture planning result, specifically: First, threshold segmentation and Canny edge detection are performed based on the mask colors of different labels in the defect area to obtain the mask contours of different defects and the mask contours of the fish fillet itself. Then, the paths of different defects are connected through logical operations and morphological operations to obtain the final adjustment pose planning result.

2. The automated fish fillet defect trimming device according to claim 1, characterized in that, The single-strip feeding and conveying unit (1-1) for fish fillets includes a chain conveyor belt (1), height limiting baffles (3) and a first drive motor (4); the input end of the chain conveyor belt (1) serves as the feeding end, the output end of the chain conveyor belt (1) is connected to the input end of the horizontal conveyor belt (12), the first drive motor (4) is connected to the chain conveyor belt (1), and multiple height limiting baffles (3) are installed on the chain conveyor belt (1) and arranged at intervals along its transmission direction.

3. The automated fish fillet defect trimming device according to claim 2, characterized in that, The chain conveyor belt (1) has at least three sections along the transmission direction. Each section of the conveyor belt is equipped with a height limit baffle (3) arranged sequentially and at intervals along its transmission direction. The first section is a horizontal feeding section, the second section is an inclined feeding section, and the third section is an image unit feeding section. This section is connected to the input end of the horizontal conveyor belt (12).

4. The automated fish fillet defect trimming device according to claim 1, characterized in that, The trimming unit (1-3) includes a trimming mechanism (8), a serial robotic arm (9), a robotic arm base (10), and a second photoelectric sensor (14). The robotic arm base (10) is installed on the side of the horizontal conveyor belt (12), the serial robotic arm (9) is fixedly installed on the robotic arm base (10), and the trimming mechanism (8) is installed at the end (16) of the serial robotic arm (9). Both the serial robotic arm (9) and the trimming mechanism (8) are connected to the image acquisition and processing unit (1-2).

5. The automated fish fillet defect trimming device according to claim 4, characterized in that, The trimming mechanism (8) includes a flange (17), a fastener (18), a third drive motor (19), a disc cutter (20), a universal ball (22), and a shock-absorbing spring (23). The fastener (18) is fixedly installed at the end of the robotic arm (16) via the flange (17). A shock-absorbing spring (23) is fixedly installed on one side of the lower surface of the fastener (18). A universal ball (22) is also fixedly installed at the bottom of the shock-absorbing spring (23). A third drive motor (19) is fixedly installed on the other side of the lower surface of the fastener (18). The output shaft of the third drive motor (19) is coaxially fixedly connected to the disc cutter (20).

6. A control method for an automated fish fillet defect trimming device according to any one of claims 1 to 5, characterized in that, The control method includes the following steps: 1) When the first photoelectric sensor (13) senses that the fish fillet to be processed has reached the image acquisition area, it sends a photoelectric signal to the industrial control computer (7); after receiving the photoelectric signal, the industrial control computer (7) sends a conveyor belt stop signal to the horizontal conveyor belt (12) and the chain plate conveyor belt (1), and at the same time controls the depth camera (6) to acquire the image of the fish fillet to be processed and upload it to the industrial control computer (7). 2) The industrial control computer (7) receives the fish fillet defect image collected by the depth camera (6), uses deep learning algorithm to identify the defect area of ​​the fish fillet defect image, and then performs path and posture planning for the defect repair process based on the defect area, obtains the repair posture planning result and sends it to the serial robotic arm (9), and then sends the conveyor belt working signal to the horizontal conveyor belt (12) and the chain plate conveyor belt (1). 3) When the second photoelectric sensor (14) senses that the fish fillet to be processed has reached the trimming area, it sends a photoelectric signal to the industrial control computer (7); after receiving the photoelectric signal, the industrial control computer (7) sends a conveyor belt stop signal to the horizontal conveyor belt (12) and the chain conveyor belt (1); then the serial robotic arm (9) moves from the initial position to above the fish fillet defect according to the trimming pose planning result and trims all the defects of the fish fillet to be processed. After the trimming is completed, it returns to the initial position; finally, the horizontal conveyor belt (12) continues to run and transports the trimmed fish fillet to the target position.

Images