Freshwater fish automatic dissection precision regulation method and system based on machine vision and profiling positioning
By combining three-dimensional vision and mechanical feature matrix, precise control of the freshwater fish dissection process was achieved, solving the problem of dissection instability caused by unknown fish mechanical properties in existing technologies. The force perception closed loop was realized by using contour positioning and impedance control, which improved the dissection quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANDONG YONGZHENG FOOD CO LTD
- Filing Date
- 2026-05-25
- Publication Date
- 2026-07-31
AI Technical Summary
Existing freshwater fish dissection technology relies on two-dimensional or three-dimensional vision to obtain geometric morphological information of the fish body, but cannot perceive mechanical properties, resulting in overcutting or undercutting during the dissection process. Furthermore, it fails to achieve closed-loop control of force perception, leading to unstable dissection quality.
The three-dimensional point cloud of the fish body is acquired by a three-dimensional vision system, a mechanical feature matrix is constructed, the cutting interval is divided, and a force/position hybrid closed loop is realized by combining contour positioning and impedance control with a six-dimensional force sensor. The cutting depth and speed are adjusted in real time, and the trajectory is corrected by monitoring the cut shape with machine vision.
It significantly reduces overcutting or undercutting caused by individual differences in fish, improves the stability and safety of the dissection process, reduces the rate of rupture and visceral residue, and achieves precise control.
Smart Images
Figure CN122493493A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of aquatic product processing technology, and more specifically, to a method and system for the precise control of automated dissection of freshwater fish based on machine vision and contour positioning. Background Technology
[0002] Freshwater fish are an important aquaculture species in my country, and the gutting and evisceration process in their pre-processing is a key step affecting processing efficiency and product quality. Traditional gutting operations rely heavily on manual labor, which leads to problems such as high labor intensity, low efficiency, poor hygiene, and inconsistent product quality.
[0003] In recent years, with the development of machine vision and automation technologies, researchers have made a series of advances in the automated processing of fish. Regarding dissection devices, existing research has used compression tests to obtain the fish's ultimate pressure resistance and combined this with machine vision to plan the cutting path, achieving mechanical dissection and evisceration of freshwater fish while keeping the dissection loss rate at a low level. In terms of intelligent evisceration equipment, research has established a database of fish body geometry and abdominal cavity structure features, constructed a regression model based on neural network optimization, and used machine vision to detect the fish's shape and precise positioning in real time, autonomously determining the operating height range of the evisceration actuator, thus achieving precise control of the penetration depth.
[0004] However, existing technologies have the following drawbacks, specifically:
[0005] Existing technologies mainly rely on two-dimensional or three-dimensional vision to acquire information about the geometric shape of the fish, and then use database matching or neural network regression to predict the feed parameters of the gutting reel or cutting blade. However, the vision system can only acquire the "surface geometry" of the fish and cannot perceive its "mechanical properties"—two fish of the same size will have completely different cutting resistance and compressive deformation characteristics due to differences in factors such as muscle density, fat content, and freshness.
[0006] Existing equipment mostly uses open-loop trajectory control, executing the cutting action according to a pre-planned path. Once encountering abnormal situations such as bone spurs or sudden changes in soft tissue density, it cannot adjust the cutting depth or feed speed in real time, and can only passively bear the consequences of "overcutting and damaging internal organs" or "undercutting and leaving internal organs behind". Currently, there is no published literature reporting the application of six-dimensional force sensors and impedance control to closed-loop regulation of cutting force in freshwater fish.
[0007] Existing cutting path planning is mostly based on simple fitting of two-dimensional contours or three-dimensional midlines, without considering the impact of the fish's stress deformation on the actual cutting depth during the cutting process, and without refining the different cutting requirements of different physiological parts (head, abdominal cavity, tail), resulting in unstable cutting quality. Summary of the Invention
[0008] The purpose of this invention is to provide a precise control method for automated dissection of freshwater fish based on machine vision and contour positioning, so as to solve the problems mentioned in the background art.
[0009] To achieve the above objectives, the present invention aims to provide a method for the precise control of automated dissection of freshwater fish based on machine vision and contour positioning, comprising: Step S1: acquiring three-dimensional point cloud of the fish body through a three-dimensional vision system, extracting the three-dimensional curve of the midline of the abdomen and back, and constructing a mechanical feature matrix containing the compressive deformation coefficient of each section of the fish body.
[0010] Step S2: Based on visual recognition of key physiological structural points of the fish, the fish body is divided into multiple cutting intervals. Differentiated cutting depth control factors are set according to the physiological characteristics of each interval. Based on the midline of the abdomen and back, abdominal cavity depth and compressive deformation coefficient, a three-dimensional cutting trajectory with feedforward compensation is planned.
[0011] Step S3: Based on the 3D point cloud, identify the direction of the fish's head, tail, and back, perform posture correction through visual guidance, use a contour-following flexible clamp and monitor the clamping force in real time to position the fish in the preset cutting posture.
[0012] Step S4: During the cutting process, the contact force is collected in real time by a six-dimensional force sensor. The expected force model of each cutting section is established based on the mechanical feature matrix. Impedance control is used to realize the force / position hybrid closed loop. At the same time, the cut shape and cutting depth are monitored in real time by machine vision and fused with force feedback for trajectory correction.
[0013] Step S5: After the sectioning is completed, collect quantitative indicators of the sectioning effect. With the goal of maximizing the sectioning quality, adaptively adjust the depth control factor, feed rate and feedforward compensation parameters of each sectioning interval, and update the mechanical feature matrix.
[0014] As a further improvement to this technical solution, the compressive deformation coefficient is specifically implemented as follows:
[0015] Data on loading force and indentation displacement at each cross section were collected, and the baseline compressive deformation coefficient at each cross section position for each fish species was calculated.
[0016] The body shape features of each cross section of the fish to be processed are extracted by 3D point cloud and matched with a reference sample library of the same fish species to obtain the body shape correction coefficient of each cross section.
[0017] The body shape correction coefficient is coupled with the reference compressive deformation coefficient to obtain the compressive deformation coefficient of each section of the fish body to be processed.
[0018] As a further improvement to this technical solution, the identification of key physiological structural points of the fish abdomen includes the fish lips, the posterior edge of the gill cover, and the anus; the division of the cutting interval is based on the above-mentioned key physiological structural points to divide the fish body into the head segment, the abdominal segment, and the tail segment, and a shallower target cutting depth is set for the abdominal segment and a deeper target cutting depth is set for the tail segment.
[0019] As a further improvement to this technical solution, the cutting depth control factor is obtained in the following way: the optimal cutting depth ratio of each fish species in the head segment, abdominal segment, and tail segment is obtained, and a reference depth control factor library for each cutting interval is established; during the processing, the body shape parameters of the fish to be processed in each cutting interval are obtained in real time and compared with the standard body shape parameters of the corresponding interval in the reference sample library of the fish species, and the body shape correction coefficient of each cutting interval is calculated; the reference depth control factor is multiplied by the body shape correction coefficient of the corresponding interval to obtain the actual cutting depth control factor of the current cutting interval of the fish.
[0020] As a further improvement to this technical solution, the three-dimensional cutting trajectory with feedforward compensation is planned according to the following formula:
[0021]
[0022] in, This represents the desired position vector of the tool's center point in the world coordinate system. A three-dimensional curve function representing the midline of the fish's ventral and dorsal body. This represents the functional relationship between the arc length of the cutter moving along the length of the fish during the cutting process and time. Indicates the first Depth control factor for each section interval This indicates the vertical depth of the abdominal cavity at the current arc length position of the fish. This represents the normal unit vector at that point on the midline of the abdomen and back. This represents the feedforward compensation strength coefficient. Indicates the desired cutting force. This represents the compressive deformation coefficient of the fish body at the current arc length position.
[0023] As a further improvement to this technical solution, the specific method for posture correction is as follows: Based on three-dimensional point cloud data, the main axis direction and the curvature change characteristics of the dorsal and ventral contours of the fish body are extracted. The head and tail orientation is identified by comparing the point cloud morphological differences between the head and tail (the head is streamlined and contracted, and the tail is flat and gradually narrowed). The ventral and dorsal orientation is identified by analyzing the surface curvature differences between the dorsal and ventral sides of the fish body (the dorsal side has a larger curvature, and the ventral side is relatively flat). The identification results are transmitted to the posture pre-orientation correction mechanism, which controls the pneumatic push rod or rotating roller to apply directional thrust to the fish body, causing the fish body to rotate or translate during the conveying process until the head of the fish body faces the conveying direction and the abdomen is vertically upward. The posture adjustment is confirmed in real time through the vision system, and finally the fish body axis coincides with the conveying direction and the abdomen faces upward, achieving the preset cutting posture.
[0024] As a further improvement to this technical solution, the specific implementation of the trajectory correction is as follows: during the cutting process, the cut image is acquired in real time, and the lateral offset of the cut relative to the midline of the abdomen and back and the actual cutting depth are extracted through image processing. The lateral offset is compared with a preset threshold. If it exceeds the threshold, a lateral correction signal is generated. The actual cutting depth is compared with the target depth of the interval to calculate the depth deviation. The correction amount generated by vision and the correction amount generated by force feedback are weighted and fused to finally obtain a comprehensive correction command.
[0025] As a further improvement to this technical solution, the quantitative indicators of the dissection effect include at least one of the following: the area of exposed internal organs, the degree of abdominal cavity damage, and the smoothness of the incision.
[0026] In a second aspect, the present invention provides a system for a precise control method of automated dissection of freshwater fish based on machine vision and contour positioning, comprising: a mechanical feature matrix construction module, which acquires three-dimensional point clouds of the fish body through a three-dimensional vision system, extracts the three-dimensional curve of the midline of the abdomen and back, and constructs a mechanical feature matrix containing the compressive deformation coefficients of each section of the fish body.
[0027] The partitioned trajectory planning module divides the fish body into multiple cutting intervals based on visual recognition of key physiological structural points. It sets differentiated cutting depth control factors according to the physiological characteristics of each interval and plans a three-dimensional cutting trajectory with feedforward compensation based on the midline of the abdomen and back, abdominal cavity depth and compressive deformation coefficient.
[0028] The posture correction module identifies the fish's head, tail, and ventral orientation based on 3D point cloud data. It then performs posture correction through visual guidance, employs a contour-following flexible clamp, and monitors the clamping force in real time to position the fish in a preset cutting posture.
[0029] The trajectory correction module collects contact force in real time through a six-dimensional force sensor during the cutting process, establishes the expected force model for each cutting interval based on the mechanical feature matrix, and uses impedance control to achieve a force / position hybrid closed loop; at the same time, it monitors the cut shape and cutting depth in real time through machine vision and integrates with force feedback to perform trajectory correction.
[0030] The parameter self-optimization module collects quantitative indicators of the sectional effect after the sectioning is completed. With the goal of maximizing the sectional quality, it adaptively adjusts the depth control factor, feed rate and feedforward compensation parameters of each sectioning interval and updates the mechanical feature matrix through reinforcement learning or Bayesian optimization methods.
[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0032] The invention establishes a baseline compressive deformation coefficient library through pre-indentation mechanical tests, and combines this with fish body shape feature matching and correction extracted from 3D point clouds to construct a mechanical feature matrix including compressive deformation coefficients, tissue density, and bone spur distribution. Based on this, a 3D cutting trajectory with feedforward compensation is planned, allowing the cutter to pre-adjust its entry position according to the hardness of the fish body before cutting, significantly reducing overcutting or undercutting caused by individual differences in fish bodies.
[0033] This invention employs an impedance controller to map force errors into position correction values in real time, forming a force / position hybrid closed-loop control. When an abnormal cutting force is detected, the tool is automatically retracted and the cutting point is replanned, effectively preventing excessive cutting that could damage internal organs or cause tool jamming. This invention fills the technological gap in force-sensing closed-loop control in the field of freshwater fish cutting.
[0034] This invention, based on force feedback control, integrates machine vision for real-time monitoring of cut shape and cutting depth. It weights and fuses the lateral offset, depth deviation, and force error detected by vision, with both methods providing redundancy. Even if the force sensor signal is interfered with by vibration, vision can still provide reliable correction data, significantly improving the stability and safety of the cutting process.
[0035] Based on key physiological structures such as the fish lips, the posterior edge of the gill cover, and the anus, this invention divides the fish body into head, abdominal, and tail segments. A smaller cutting depth control factor is set for the abdominal segment to protect the internal organs, while a larger cutting depth control factor is set for the tail segment to ensure that the muscles are thoroughly cut. This achieves precise control over different parts and effectively reduces the rate of ruptured belly and residual internal organs. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 This is a schematic diagram of the implementation steps of the method of the present invention.
[0038] Figure 2 This is a schematic diagram of the system structure connection of the present invention. Detailed Implementation
[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0040] Example: Please refer to Figure 1 As shown, a method for automated and precise control of freshwater fish cutting based on machine vision and contour positioning is provided, including: Step S1: Collect the three-dimensional point cloud of the fish body through a three-dimensional vision system, extract the three-dimensional curve of the midline of the abdomen and back, and construct a mechanical feature matrix containing the compressive deformation coefficient of each section of the fish body.
[0041] In one specific embodiment, the compressive deformation coefficient is implemented as follows:
[0042] Beforehand, equidistant cross-sectional indentation mechanical tests were conducted on typical samples of different freshwater fish species along the length of the fish body. Data on the loading force and indentation displacement of each cross-section were collected, and the benchmark compressive deformation coefficient of each cross-section of each fish species was calculated.
[0043]
[0044] in, Indicates the first The reference compressive strain coefficient of a cross section (unit: N / mm) represents the external force required to produce a unit compressive displacement in the fish body; the larger the value, the "harder" the cross section. This indicates the numbering of equidistant cross sections along the length of the fish's body. , Indicates the first Standard indentation force applied to the cross section (unit: N). This represents the normal compressive displacement of the fish surface measured under this indentation force (unit: mm).
[0045] The body shape features of each cross section of the fish to be processed are extracted by 3D point cloud and matched with a reference sample library of the same fish species to obtain the body shape correction coefficient of each cross section.
[0046]
[0047] in, The dimensionless correction factor for the body shape at section i represents the difference in body shape between the fish to be processed and the reference sample at that section. The cross-sectional area of the i-th section of the fish body to be processed is represented (calculated from 3D point cloud slices, unit: mm²). This represents the average cross-sectional area (unit: mm²) of the i-th section in the baseline sample library of the same fish species. This represents the width of the i-th cross-section of the fish body to be processed (unit: mm). This represents the average width (in mm) of the i-th cross section in the baseline sample library of the same fish species. This represents the height of the i-th cross-section of the fish body to be processed (unit: mm). This represents the average height (in mm) of the i-th cross section in the baseline sample library of the same fish species.
[0048] The body shape correction coefficient is coupled with the reference compressive deformation coefficient to obtain the compressive deformation coefficient of each section of the fish body to be processed.
[0049]
[0050] in, This represents the compressive deformation coefficient of the i-th cross-section. The compressive deformation coefficients of each discrete cross-section along the length of the fish body are... Constructing continuous arc length parameters through linear interpolation or spline interpolation. function Used for trajectory planning and force control
[0051] The three-dimensional vision system is a binocular stereo vision system or a three-dimensional imaging system combining structured light and a binocular camera.
[0052] The method for obtaining the 3D curve of the midline of the abdomen and back is as follows: First, the point cloud of the fish surface acquired by the 3D vision system is downsampled and denoised. Then, principal component analysis (PCA) is used to determine the direction of the fish's principal axis. Next, the 3D model of the fish is sliced along the principal axis at equal intervals, and the geometric center point of the contour of each slice section is calculated. Finally, all center points are connected in the order of the principal axis and smoothed using B-spline curves to obtain the curve with arc length parameter. Representation of three-dimensional space curve functions ,in , The total length of the fish's body (the length of the curve from the tip of the snout to the end of the tail fin) is the three-dimensional geometric description of the midline of the fish's abdomen and back.
[0053] The construction process of the mechanical feature matrix is as follows: Based on the three-dimensional fish model obtained by three-dimensional point cloud analysis, the tissue density distribution features and bone spur distribution location parameters of each section are extracted through image processing and morphological analysis; finally, the compressive deformation coefficient, tissue density, and bone spur distribution are arranged in the order of the sections to form a two-dimensional matrix, where each row corresponds to a section, and each column stores the compressive deformation coefficient, tissue density value, and bone spur location code of that section. This matrix is the fish mechanical feature matrix used for subsequent differential control of the cutting interval, tool infeed pre-compensation, and segmented construction of the desired force model.
[0054] Step S2: Based on visual recognition of key physiological structural points of the fish, the fish body is divided into multiple cutting intervals. Differentiated cutting depth control factors are set according to the physiological characteristics of each interval. Based on the midline of the abdomen and back, abdominal cavity depth and compressive deformation coefficient, a three-dimensional cutting trajectory with feedforward compensation is planned.
[0055] In one specific embodiment, the identification of key physiological structural points in the fish abdomen includes the fish lips, the posterior edge of the gill cover, and the anus; the division of the dissection interval is based on the above-mentioned key physiological structural points to divide the fish body into the head segment, the abdominal segment, and the tail segment, and a shallower target dissection depth is set for the abdominal segment and a deeper target dissection depth is set for the tail segment.
[0056] In one specific embodiment, the cutting depth control factor is obtained in the following way: Anatomical measurements and cutting experiments are performed on typical samples of different freshwater fish species in advance, and the optimal cutting depth ratios for the head, abdominal cavity, and tail segments of each fish species are statistically obtained to establish a baseline depth control factor library for each cutting interval; during processing, the body shape parameters such as body height, body width, and abdominal cavity depth of the fish to be processed in each cutting interval are acquired in real time through a three-dimensional vision system and compared with the standard body shape parameters of the corresponding interval in the baseline sample library of the fish species to calculate the body shape correction coefficient for each cutting interval; the baseline depth control factor is multiplied by the body shape correction coefficient of the corresponding interval to obtain the actual cutting depth control factor of the current cutting interval of the fish.
[0057] In one specific embodiment, the three-dimensional cutting trajectory with feedforward compensation is planned according to the following formula:
[0058]
[0059] in, This represents the desired position vector of the tool's center point in the world coordinate system. A three-dimensional curve function representing the midline of the fish's ventral and dorsal body, using the arc length parameter. express, This represents the functional relationship between the arc length of the cutter moving along the length of the fish during the cutting process and time. Indicates the first The depth control factor (dimensionless) for each section interval takes a value between 0 and 1. This indicates the vertical depth of the abdominal cavity at the current arc length position of the fish (unit: mm). This represents the normal unit vector at that point on the midline of the abdomen and back, pointing towards the outer side of the fish's abdomen. This represents the feedforward compensation strength coefficient (dimensionless), ranging from 0 to 1, used to adjust the degree of contribution of mechanical compensation. This represents the desired cutting force, which is the target normal force (unit: N) set when the cutter comes into contact with the fish. The compressive deformation coefficient (unit: N / mm) represents the fish body's resistance to compressive deformation at the current arc length position, characterizing the tissue's ability to resist compressive deformation.
[0060] The differentiated setting method for the cutting depth control factor is as follows: Based on the physiological differences that the head segment has dense bones and thin muscles, the abdominal segment has concentrated internal organs and a soft body wall, and the tail segment has thick muscles and no internal organs, the cutting depth control factor for the abdominal segment is set to the minimum value to prevent the blade from touching the internal organs; the cutting depth control factor for the tail segment is set to the maximum value to ensure that the thick muscles are cut through; the cutting depth control factor for the head segment is set to an intermediate value between the abdominal segment and the tail segment to avoid damaging the bones while cutting through the body wall; the difference in the depth control factor values between different cutting sections is pre-calibrated based on the anatomical statistics of the fish species, and dynamically adjusted during processing based on the actual body shape parameters measured visually.
[0061] Step S3: Based on the 3D point cloud, identify the direction of the fish's head, tail, and back, perform posture correction through visual guidance, use a contour-following flexible clamp and monitor the clamping force in real time to position the fish in the preset cutting posture.
[0062] In one specific embodiment, the posture correction method is as follows: Based on three-dimensional point cloud data, the main axis direction and the curvature change characteristics of the dorsal and ventral contours of the fish body are extracted. The head and tail orientation is identified by comparing the point cloud morphological differences between the head and tail (the head is streamlined and contracted, and the tail is flat and gradually narrowed). The ventral and dorsal orientation is identified by analyzing the surface curvature differences between the dorsal and ventral sides of the fish body (the dorsal side has a larger curvature, and the ventral side is relatively flat). The identification results are transmitted to the posture pre-orientation correction mechanism, which controls the pneumatic push rod or rotating roller to apply directional thrust to the fish body, causing the fish body to rotate or translate during the conveying process until the head of the fish body faces the conveying direction and the abdomen is vertically upward. The posture adjustment is confirmed in real time through the vision system, and finally the fish body axis coincides with the conveying direction and the abdomen faces upward, achieving the preset cutting posture.
[0063] The contour-following flexible clamping mechanism specifically involves: using a flexible material for the clamping surface; automatically adjusting the clamping shape according to the local three-dimensional contour of the fish body; and monitoring the clamping force in real time through a pull wire displacement sensor or Hall sensor integrated on the clamping mechanism. When the clamping force exceeds a preset threshold, an alarm is triggered and the clamping action is stopped.
[0064] Step S4: During the cutting process, contact force is collected in real time by a six-dimensional force sensor. The desired force model for each cutting interval is established based on the mechanical feature matrix, and impedance control is used to achieve a force / position hybrid closed loop. At the same time, machine vision is used to monitor the cut shape and cutting depth in real time, and the trajectory is corrected by integrating with force feedback. When the cutting force exceeds the safety threshold, the tool is automatically retracted.
[0065] In one specific embodiment, the trajectory correction is implemented as follows: during the cutting process, the machine vision system acquires incision images in real time, and extracts the lateral offset of the incision relative to the midline of the abdomen and back through image processing. and actual cutting depth The lateral offset is compared with a preset threshold; if it exceeds the threshold, a lateral correction signal is generated. ; Compare the actual cutting depth with the target depth of that interval Compare and calculate depth deviation Combine the correction amount generated by vision with the correction amount generated by force feedback. Perform weighted fusion, such as Ultimately, a comprehensive correction command is obtained. At the same time, when the vision detects abnormal shapes such as exposed internal organs or torn incisions, an emergency lifting or retraction operation is triggered first, realizing multimodal fusion control of force and vision.
[0066] The specific method for establishing the expected force model for each section based on the mechanical feature matrix is as follows: For each section, based on the compressive deformation coefficient, tissue density, and osteophyte distribution parameters of each section within that section, a expected force curve varying along the arc length parameter s is constructed using piecewise linear interpolation or spline curve fitting. Specifically, for areas with a small compressive deformation coefficient (softer tissue) or dense internal organs, a lower expected force value is set to avoid damage. For areas with a large compressive deformation coefficient (harder tissue) or osteophytes, the expected force value is appropriately increased to ensure section continuity, but not exceeding the maximum safe force threshold that the tissue in that area can withstand. The expected force curves between each section maintain a continuous or smooth transition at the junction, ultimately forming an expected force model covering the entire length of the fish, which serves as the force command input for the impedance controller in the force / position hybrid control.
[0067] The specific method for achieving a force / position hybrid closed loop using impedance control is as follows: Real-time acquisition of the current contact force is achieved through a six-dimensional force sensor. The desired force is read from the desired force model based on the current cutting range of the tool. Calculation force error The force error is input to an impedance controller, which employs a second-order mass-spring-damped model. ,in, , , For impedance parameters, This is the position correction amount; output the position correction amount. The correction amount is superimposed on the desired trajectory generated in step S2. Above, actual control commands are formed. This allows for closed-loop adjustment of the cutting force while ensuring position tracking.
[0068] Step S5: After the sectioning is completed, collect quantitative indicators of the sectioning effect. Using reinforcement learning or Bayesian optimization methods, with the goal of maximizing the sectioning quality, adaptively adjust the depth control factor, feed rate and feedforward compensation parameters of each sectioning interval, and update the mechanical feature matrix.
[0069] In one specific embodiment, the quantitative indicators of the dissection effect include at least one of the following: the area of exposed internal organs, the degree of abdominal cavity damage, and the smoothness of the incision.
[0070] See Figure 2 As shown, a system for the automated and precise control of freshwater fish dissection based on machine vision and contour positioning is provided, including: a mechanical feature matrix construction module, which acquires three-dimensional point clouds of the fish body through a three-dimensional vision system, extracts the three-dimensional curve of the midline of the abdomen and back, and constructs a mechanical feature matrix containing the compressive deformation coefficients of each section of the fish body.
[0071] The partitioned trajectory planning module divides the fish body into multiple cutting intervals based on visual recognition of key physiological structural points. It sets differentiated cutting depth control factors according to the physiological characteristics of each interval and plans a three-dimensional cutting trajectory with feedforward compensation based on the midline of the abdomen and back, abdominal cavity depth and compressive deformation coefficient.
[0072] The posture correction module identifies the fish's head, tail, and ventral orientation based on 3D point cloud data. It then performs posture correction through visual guidance, employs a contour-following flexible clamp, and monitors the clamping force in real time to position the fish in a preset cutting posture.
[0073] The trajectory correction module collects contact force in real time through a six-dimensional force sensor during the cutting process, establishes the expected force model for each cutting interval based on the mechanical feature matrix, and uses impedance control to achieve a force / position hybrid closed loop; at the same time, it monitors the cut shape and cutting depth in real time through machine vision and integrates with force feedback to perform trajectory correction.
[0074] The parameter self-optimization module collects quantitative indicators of the sectional effect after the sectioning is completed. With the goal of maximizing the sectional quality, it adaptively adjusts the depth control factor, feed rate and feedforward compensation parameters of each sectioning interval and updates the mechanical feature matrix through reinforcement learning or Bayesian optimization methods.
[0075] In another embodiment, the test fish species was grass carp, with a body length of 300-450 mm, a body width of 50-70 mm, and a weight of 500-1200 g, totaling 100 fish.
[0076] Equipment configuration:
[0077] 3D vision system: binocular stereo camera (resolution 1280×1024, baseline distance 150mm) + structured light projector (wavelength 850nm), installed 500mm above the conveyor line.
[0078] Mechanical pre-scan module: 16 pressure-displacement sensing units are arranged along the belly of the fish, with a spacing of 15mm and the indentation force is set to 2N.
[0079] Attitude correction mechanism: pneumatic push rod (stroke 50mm) + rotating roller (speed 30r / min).
[0080] Contouring clamping module: left and right airbag clamps, closed-loop control of clamping pressure, target clamping force 5N.
[0081] Cutting actuator: a six-axis industrial robot (repeat positioning accuracy ±0.05mm), with a stainless steel cutting blade (blade length 80mm) and a six-dimensional force sensor (range ±200N, accuracy 0.1N) mounted at the end.
[0082] Real-time visual monitoring unit: High-speed camera (200fps), fixed 30mm behind the tool holder.
[0083] Step S1:
[0084] The grass carp is placed belly-up on the conveyor belt, and a binocular camera captures images from the left and right sides. The three-dimensional point cloud (point density of about 1 point / mm²) is obtained by combining structured light stripe calculation.
[0085] Point cloud preprocessing: downsampling (voxel grid size 0.5mm), outlier filtering (k=20, threshold 0.5mm).
[0086] Extracting the midline of the ventral and dorsal regions: PCA is used to determine the principal axis direction. Slices are cut at 2mm intervals along the principal axis, and the geometric center of the contour of each slice is calculated. A curve is obtained by fitting a cubic B-spline. , .
[0087] Constructing the mechanical characteristic matrix:
[0088] Call the pre-stored grass carp benchmark compressive deformation coefficient library (obtained through offline indentation tests), Values: Head segment 2.8–3.2 N / mm, Abdominal segment 1.5–1.9 N / mm, Tail segment 3.5–4.0 N / mm.
[0089] The point cloud computing of each cross section of the fish to be processed According to the formula Obtain the body shape correction factor (average in this example) (ranging from 0.95 to 1.08).
[0090] Coupling yields the actual compressive strain coefficient (Range 1.43~4.32 N / mm).
[0091] Will The tissue density (estimated from the grayscale texture of the point cloud, taking a per-unit value of 0 to 1) and the location of the bony spines (labeled according to prior knowledge of the fish species) are arranged into a 16×3 matrix.
[0092] Step S2:
[0093] Identifying key physiological structures: The fish lips (s=0), posterior edge of the gill cover (s=85mm), and anus (s=260mm) were detected from the point cloud projection image using a deep learning model (U-Net). The fish body was then divided into:
[0094] Head segment: s∈[0,85mm]
[0095] Abdominal segment: s∈(85mm,260mm)
[0096] Tail segment: s∈(260mm,350mm)
[0097] Differentiation depth control factor:
[0098] Benchmark depth factor library: , , .
[0099] Body type correction (based on average height ratio of each region): , , .
[0100] Actual factors: , , .
[0101] Planning feedforward compensation trajectory:
[0102] Take the feedforward strength coefficient Expected cutting force The abdominal segment is set at 3N, and the other segments at 5N.
[0103] Substitute into the formula:
[0104] The time-parameterized trajectory was generated using cubic spline interpolation, with a total cutting time of 3.5 seconds and a feed rate of 100 mm / s.
[0105] Step S3:
[0106] Head and tail orientation recognition: Projection of point cloud along the main axis shows a large curvature change and a sharp end in the head point cloud, while the tail is flat. The current fish head is facing the delivery direction (no correction required).
[0107] Abdominal orientation identification: The degree of arching of the dorsal side (curvature integral value > 0.05 rad / mm), and the ventral side is relatively flat (curvature integral value < 0.02 rad / mm). Currently, the abdomen is facing upwards (qualified).
[0108] Clamping: The airbag clamp is inflated to a contact pressure of 5N. The Hall sensor monitors the clamping displacement and stops feeding after stabilization.
[0109] Step S4:
[0110] The cutting process begins, and the six-dimensional force sensor collects data in real time. (Sampling rate 1kHz).
[0111] Impedance controller parameters: g, , Force error Output position correction amount This is superimposed on the desired trajectory.
[0112] Real-time visual monitoring: 200fps images, extracting cut edges, calculating lateral offset. .when Horizontal correction is generated in time. Simultaneously measure the actual depth. , and target depth Comparison of depth deviation .
[0113] Fusion Correction: In this example, a lateral offset of 2.0 mm was visually detected in the posterior part of the abdominal segment (s≈200mm). After fusion correction, the tool automatically returned to center.
[0114] Safety protection: When The tool retracts 5mm and decelerates before re-entering (this was not triggered in this embodiment).
[0115] Step S5:
[0116] After sectionalization, the image is sent to the visual evaluation unit (5-megapixel camera). Image processing calculations:
[0117] Visceral exposure area: 85% (>90% is considered excellent, this case reached 92%)
[0118] Length of abdominal cavity rupture: 3mm (threshold <10mm is acceptable)
[0119] Incision smoothness: Straightness of the incision edge 0.8mm (threshold <2mm)
[0120] Bayesian optimization update:
[0121] Use this parameter Add to dataset.
[0122] Gaussian process surrogate model predicts optimal parameters Offset.
[0123] Update the mechanical characteristic matrix: use an exponentially weighted moving average (... This will be the result of the actual test. By integrating with the historical database, the accuracy of pre-compensation for the next fish can be improved.
[0124] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.
Claims
1. An automatic freshwater fish dissection precision control method based on machine vision and profiling positioning, characterized in that, include: Step S1: Collect the three-dimensional point cloud of the fish body through a three-dimensional vision system, extract the three-dimensional curve of the midline of the abdomen and back, and construct a mechanical feature matrix containing the compressive deformation coefficient of each section of the fish body. Step S2: Based on visual recognition of key physiological structural points of the fish, the fish body is divided into multiple cutting intervals. Differentiated cutting depth control factors are set according to the physiological characteristics of each interval. Based on the midline of the abdomen and back, abdominal cavity depth and compressive deformation coefficient, a three-dimensional cutting trajectory with feedforward compensation is planned. Step S3: Based on the 3D point cloud, identify the head, tail and belly / back orientation of the fish, perform posture correction through visual guidance, use contour-following flexible clamping and monitor the clamping force in real time to position the fish in the preset cutting posture. Step S4: During the cutting process, the contact force is collected in real time by a six-dimensional force sensor. The expected force model of each cutting section is established based on the mechanical feature matrix. Impedance control is used to realize the force / position hybrid closed loop. At the same time, the cut shape and cutting depth are monitored in real time by machine vision and fused with force feedback for trajectory correction. Step S5: After the sectioning is completed, collect quantitative indicators of the sectioning effect. With the goal of maximizing the sectioning quality, adaptively adjust the depth control factor, feed rate and feedforward compensation parameters of each sectioning interval, and update the mechanical feature matrix.
2. The method according to claim 1, wherein the method is characterized in that: The compressive deformation coefficient is specifically implemented as follows: Data on loading force and indentation displacement at each cross section were collected, and the baseline compressive deformation coefficient at each cross section position for each fish species was calculated. The body shape features of each cross section of the fish to be processed are extracted by 3D point cloud and matched with the benchmark sample library of the same fish species to obtain the body shape correction coefficient of each cross section. The body shape correction coefficient is coupled with the reference compressive deformation coefficient to obtain the compressive deformation coefficient of each section of the fish body to be processed.
3. The method according to claim 2, wherein the method is characterized in that: The key physiological structural points for identifying the fish's abdomen include the fish lips, the posterior edge of the gill cover, and the anus; the division of the dissection section is based on the above key physiological structural points, dividing the fish body into the head segment, abdominal segment, and tail segment, and setting a shallower target dissection depth for the abdominal segment and a deeper target dissection depth for the tail segment.
4. The method for precise control of automated dissection of freshwater fish based on machine vision and contour positioning according to claim 3, characterized in that: The sectioning depth control factor is obtained in the following way: The optimal cutting depth ratios for the head, abdominal, and tail segments of each fish species are obtained, and a baseline depth control factor library for each cutting segment is established. During processing, the body shape parameters of the fish to be processed in each cutting segment are acquired in real time and compared with the standard body shape parameters of the corresponding segment in the baseline sample library of the fish species to calculate the body shape correction coefficient for each cutting segment. The baseline depth control factor is multiplied by the body shape correction coefficient of the corresponding segment to obtain the actual cutting depth control factor for the current cutting segment of the fish.
5. The method for precise control of automated dissection of freshwater fish based on machine vision and contour positioning according to claim 4, characterized in that: The three-dimensional cutting trajectory with feedforward compensation is planned according to the following formula: in, This represents the desired position vector of the tool's center point in the world coordinate system. A three-dimensional curve function representing the midline of the fish's ventral and dorsal body. This represents the functional relationship between the arc length of the cutter moving along the length of the fish during the cutting process and time. Indicates the first Depth control factor for each section interval This indicates the vertical depth of the abdominal cavity at the current arc length position of the fish. This represents the normal unit vector at that point on the midline of the abdomen and back. This represents the feedforward compensation strength coefficient. Indicates the desired cutting force. This represents the compressive deformation coefficient of the fish body at the current arc length position.
6. The method for precise control of automated dissection of freshwater fish based on machine vision and contour positioning according to claim 5, characterized in that: The specific method for attitude correction is as follows: Based on three-dimensional point cloud data, the main axis direction and the curvature variation characteristics of the dorsal and ventral contours of the fish body are extracted. The head and tail orientation is identified by comparing the point cloud morphology differences between the head and tail of the fish. The ventral and dorsal orientation is identified by analyzing the surface curvature differences between the dorsal and ventral sides of the fish body. The recognition results are transmitted to the attitude pre-orientation correction mechanism, which controls the pneumatic push rod or rotating roller to apply directional thrust to the fish, causing the fish to rotate or translate during the transport process until the fish's head faces the transport direction and its abdomen is vertically upward. The vision system provides real-time feedback to confirm that the attitude adjustment is in place, and finally the fish's axis coincides with the transport direction and its abdomen faces upward, achieving the preset cutting posture.
7. The method for precise control of automated dissection of freshwater fish based on machine vision and contour positioning according to claim 6, characterized in that: The specific implementation method of the trajectory correction is as follows: During the cutting process, the cut images are acquired in real time. The lateral offset of the cut relative to the midline of the abdomen and back and the actual cutting depth are extracted through image processing. The lateral offset is compared with a preset threshold. If it exceeds the threshold, a lateral correction signal is generated. The actual cutting depth is compared with the target depth of the interval to calculate the depth deviation. The correction amount generated by vision and the correction amount generated by force feedback are weighted and fused to finally obtain a comprehensive correction command.
8. The method for precise control of automated dissection of freshwater fish based on machine vision and contour positioning according to claim 7, characterized in that: The quantitative indicators of the dissection effect include at least one of the following: the area of exposed internal organs, the degree of abdominal cavity damage, and the smoothness of the incision.
9. A system for executing the automated and precise control method for dissection of freshwater fish based on machine vision and contour positioning as described in any one of claims 1-8, characterized in that: include: The mechanical feature matrix construction module acquires the three-dimensional point cloud of the fish body through a three-dimensional vision system, extracts the three-dimensional curve of the midline of the abdomen and back, and constructs a mechanical feature matrix containing the compressive deformation coefficients of each section of the fish body. The partitioned trajectory planning module divides the fish body into multiple cutting intervals based on visual recognition of key physiological structural points. It sets differentiated cutting depth control factors according to the physiological characteristics of each interval and plans a three-dimensional cutting trajectory with feedforward compensation based on the midline of the abdomen and back, abdominal cavity depth and compressive deformation coefficient. The posture correction module identifies the head, tail, and back orientation of the fish based on 3D point cloud data, performs posture correction through visual guidance, and uses a contour-following flexible clamp to monitor the clamping force in real time, positioning the fish in the preset cutting posture. The trajectory correction module collects contact force in real time through a six-dimensional force sensor during the cutting process, establishes the expected force model for each cutting interval based on the mechanical feature matrix, and uses impedance control to achieve a force / position hybrid closed loop; at the same time, it monitors the cut shape and cutting depth in real time through machine vision and integrates with force feedback to perform trajectory correction. The parameter self-optimization module collects quantitative indicators of the sectional effect after the sectioning is completed. With the goal of maximizing the sectional quality, it adaptively adjusts the depth control factor, feed rate and feedforward compensation parameters of each sectioning interval and updates the mechanical feature matrix through reinforcement learning or Bayesian optimization methods.