Image processing-based counting and sorting device and method for river crab

Abstract

The application discloses a kind of based on image processing's river crab's counting sorting device and method, belong to river crab processing technical field.A kind of based on image processing's river crab's counting sorting device, including crab entry box, the lower portion of crab entry box is connected with integrally formed blanking slide, control box inside is provided with programmable logic controller, visual camera is installed in the inside of mounting box, the lower end of dynamic sorting plate is installed with stepper motor.The application solves the problem that the conveying position of river crab is difficult to adjust during the transmission of existing river crab, and it is difficult to take pictures and subsequent identification by camera, which affects the sorting work.The application adjusts the discharge position of river crab by edge recognition of river crab image, synchronously adjusts two stepper motors, and rotates two movable sorting plates simultaneously to adjust the discharge position of river crab for sorting work.The structure is simple, convenient to operate and adjust, and suitable for continuous sorting and counting work.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of river crab processing, in particular to a river crab counting and sorting device and method based on image processing. BACKGROUND

[0002] The river crab counting and sorting device based on image processing is a device that uses image processing technology to automatically count and sort river crabs. The device captures images of river crabs through a camera, analyzes the images using advanced image processing algorithms, accurately identifies and counts the number of river crabs, and automatically sorts them into different categories based on their size, shape, and other characteristics. This device not only improves the efficiency and accuracy of river crab sorting, but also greatly reduces the labor intensity, and is widely used in river crab breeding and processing industries.

[0003] Chinese patent CN211613488U discloses a crab sorting device, which includes a sorting and feeding machine, a feeding chute inclinedly connected to the discharge port of the sorting and feeding machine, an identification and detection camera and an LED light source arranged on a monitoring station, and a data acquisition intelligent monitoring and comprehensive division cooperating with mechanical sorting to realize a full-automatic crab sorting assembly line. The device is reliable in action, high in working efficiency, saves a large amount of labor, saves crab sorting labor cost, and improves the economic benefit of crabs.

[0004] The above-mentioned crab sorting device has the problems that the conveying position of the crabs is difficult to adjust during the conveying process, and it is difficult to take pictures and subsequent recognition through the camera, affecting the sorting work. SUMMARY

[0005] The present application aims to provide a river crab counting and sorting device and method based on image processing, which solves the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solution: a river crab counting and sorting device based on image processing, comprising a crab entering box, an integrated feeding port is arranged on the upper end face of the crab entering box, an integrated discharging chute is connected below the crab entering box, a mounting box is installed on the upper end of the discharging chute, a control box is arranged on the upper end face of the mounting box, a programmable logic controller is arranged inside the control box, and the programmable logic controller comprises a processing unit, a visual camera is installed inside the mounting box, two movable sorting plates are arranged inside the discharging chute, a stepping motor is installed on the lower end of the movable sorting plate, and the stepping motor and the programmable logic controller are connected through a PLC module.

[0007] Preferably, the lower end of the feeding chute is provided with a sorting feeding port, a distribution hopper is installed below the feeding chute, the upper end of the distribution hopper is provided with a distribution feeding port, the distribution hopper communicates with the feeding chute through the distribution feeding port, and sorting partition plates are arranged on both sides of the sorting feeding port.

[0008] Preferably, a support frame is installed below the crab entering box body, rollers are installed on both sides of the lower end of the support frame, a circulating water pump is fixedly connected to one side of the support frame, a water outlet pipe and a water inlet pipe are respectively arranged on the circulating water pump, the water inlet pipe is used to connect a water source, a water outlet is arranged at the upper end of the crab entering box body, the water outlet pipe communicates with the water outlet at the upper end of the crab entering box body, and the circulating water pump and the stepping motor are fixedly connected to the support frame through a connecting frame.

[0009] Preferably, a guide plate is installed at the upper end of the inside of the crab entering box body, a pushing roller is arranged at the lower end of the guide plate, the pushing roller is rotatably connected to the crab entering box body and the feeding chute through a bearing frame, an electric motor is arranged at the lower end of the pushing roller, and the electric motor is in transmission connection with the pushing roller through a gear.

[0010] Preferably, a buffer plate is installed at the lower end of the pushing roller, the position of the buffer plate below corresponds to the position of the distribution feeding port, and the buffer plate is fixedly connected to the feeding chute.

[0011] Preferably, the mounting box is fixedly connected to the crab entering box body through buckles, a display screen is arranged on the outside of the mounting box, the display screen is electrically connected to the control box, an integrated mounting cavity is arranged at the lower end surface of the mounting box, a light supplementing lamp is installed at the lower end of the mounting box, a visual camera is fixedly connected to the mounting box through screws in the inside of the mounting cavity, and the visual camera is connected to the programmable logic controller through a PLC module.

[0012] Preferably, an adjusting shaft is installed at the lower end of the movable sorting plate, the adjusting shaft penetrates through the feeding chute and is rotatably connected to the feeding chute through a bearing, the lower end of the adjusting shaft is fixedly connected to the stepping motor through a shaft coupling, and the upper end of the movable sorting plate is abuttingly connected to both sides of the feeding chute.

[0013] Preferably, the processing unit comprises:

[0014] A first identification subunit is configured to perform edge identification on a to-be-identified river crab image collected by the visual camera, and determine a to-be-identified river crab sub-image corresponding to a region of a crab pincer, a crab foot, a crab shell or a crab navel in the to-be-identified river crab image.

[0015] The standard image library construction subunit is configured to construct multi-angle standard images of claws, legs, shells or navel of crabs based on claws, legs, shells or navel of crabs in the standardized crab images.

[0016] The image quality evaluation subunit is configured to:

[0017] The similarity of a to-be-identified sub-image of the claws, legs, shells or navel of crabs in the to-be-identified crab image and the claws, legs, shells or navel of crabs in the standardized crab images in the standard image library is calculated to determine a similarity score of the corresponding image area of the claws, legs, shells or navel of crabs; the variance of the corresponding image area of the claws, legs, shells or navel of crabs is calculated based on the Laplace operator; and the evaluation value of the corresponding image area of the claws, legs, shells or navel of crabs is determined based on the similarity score and the variance.

[0018] The evaluation value of the to-be-identified crab image is determined based on the evaluation value of the corresponding area of the claws, legs, shells or navel of crabs and a preset algorithm.

[0019] The comparison subunit is configured to compare the evaluation value of the to-be-identified crab image with a preset evaluation threshold, and determine the to-be-identified crab image as a target image when the evaluation value is greater than or equal to the preset evaluation threshold.

[0020] The second identification subunit is configured to input the target image into a pre-trained crab sorting model to identify the target image and determine the category of the crabs in the target image.

[0021] The sorting subunit is configured to determine a sorting result and target position information based on the category of the crabs in the target image, and send the sorting result and the target position information to a programmable logic controller; the sorting result is associated with the target position information.

[0022] The execution subunit is configured to perform a sorting operation on the to-be-sorted crabs based on the sorting result and the target position information.

[0023] Preferably, the training method of the crab sorting model comprises:

[0024] A crab training data set is obtained.

[0025] An initial crab sorting model is constructed.

[0026] A multi-scale crab feature is obtained by performing feature extraction on the crab training data set through a multi-scale feature extraction network in the initial crab sorting model.

[0027] A key discriminative feature of the crabs in the crab training data set is calculated; the key discriminative feature comprises a composite shape feature, a structure feature and a composite color feature of the crabs.

[0028] The initial river crab sorting model fuses multi-scale river crab features and key discriminant features through an attention-filter fusion module to obtain river crab fusion features;

[0029] A dynamic feature aggregation network and a target decoder in the initial river crab sorting model are used to obtain a predicted classification result; the dynamic feature aggregation network configures scene adaptation reference features for each label, and includes two feature interaction layers and one scene adaptation aggregation layer; the feature interaction layer calculates the similarity between the river crab fusion features and each group of reference features through cosine similarity and performs weighted summation; the scene adaptation aggregation layer adjusts the aggregation coefficient according to the actual interference scene of the sample; the target decoder normalizes the similarity between the aggregated features and the reference features into a predicted probability, and takes the maximum probability value as the predicted classification result;

[0030] A dynamic weight joint loss function is constructed to train the initial river crab sorting model, and an optimizer is used to minimize the total loss to obtain a target river crab sorting model.

[0031] An implementation method of a river crab counting and sorting device based on image processing, comprising the following steps:

[0032] Step one: pour the river crabs that need to be sorted into the feeding port, and the circulating water pump guides the clean water into the water outlet, and the water outlet discharges the clean water to flush the river crabs, and the guide plate is used for guiding;

[0033] Step two: the river crabs are guided into the two rows of pushing rollers through the guide plate, and the pushing rollers transport the river crabs one by one, and after the river crabs enter the discharge chute, the internal visual camera of the installation box is used for detection;

[0034] Step three: the visual camera sends the detected electrical signal to the plc module, and the plc module is processed, the overall integrity of the river crab is judged according to the programmable logic controller, and the stepping motor is adjusted according to the judgment data;

[0035] Step four: the river crabs detected by the visual camera are transported to the two sides of the movable sorting plate through the pushing rollers and the buffer plate, pushed by the water flow, discharged from the sorting discharge port on one side of the sorting partition plate, and counted by the visual camera above the sorting discharge port;

[0036] Step five: the river crabs detected by the visual camera are counted by the visual camera first, and then the movable sorting plate is adjusted by the stepping motor, the movable sorting plate is expanded to the two sides, and after the movable sorting plate abuts against the two sides of the discharge chute, the water flow pushes the unqualified river crabs and guides them into the shunt discharge port, and then the river crabs are guided into the distribution hopper, and the sorting is completed.

[0037] Compared with the prior art, the beneficial effects of the present application are:

[0038] 1. The river crab is poured into the feeding port, the circulating water pump introduces the clean water into the water outlet, the water outlet discharges the clean water to flow the river crab, the river crab is guided through the guide plate, the motor at the bottom of the pushing roller drives the pushing roller to rotate, the river crab between the two rows of pushing rollers is pushed to move one by one, the river crab is guided into the two rows of pushing rollers through the guide plate, and the buffer plate can be used for orderly conveying work, and the river crab is pushed by the circulating water pump, so that the river crab is prevented from being blocked in the conveying process, the structure is compact, the conveying and sorting work can be realized through the integrated crab entering box, the structure is stable, convenient and practical, convenient to operate, and good in structure stability, and conducive to long-term use, convenient for disassembly and maintenance.

[0039] 2. The edge of the river crab image is recognized, the sub-image of the key part such as crab pincers, crab feet, crab shell or crab navel can be accurately determined, the multi-dimensional evaluation mode can more accurately reflect the quality of the image, the limitation of single index is avoided, so that the target image meeting the requirements can be more reliably screened out; the target image is input into the pre-trained river crab sorting model for identification, the learning ability and generalization ability of the model are utilized, the category of the river crab can be accurately determined according to the characteristics of the river crab, and the accuracy of sorting is further improved.

[0040] 3. The movable sorting plate installed in the discharging chute is adjusted synchronously by two stepping motors, the two movable sorting plates are rotated inward at the same time, the channels on both sides of the discharging chute are opened, the river crab can slide on both sides of the sorting partition plate, and the river crab slides into the crab box and the discharging chute for sorting; the two stepping motors are adjusted synchronously, the two movable sorting plates are rotated outward at the same time, the river crab sliding out of the buffer plate is introduced into the shunt discharging port, and the river crab is sorted through the shunt discharging port and introduced into the distribution hopper, the structure is simple, convenient to operate and adjust, and suitable for continuous sorting and counting work. DETAILED DESCRIPTION

[0041] Figure 1 It is a side view of the shaft of the present application;

[0042] Figure 2 It is a top view of the shaft of the present application;

[0043] Figure 3 It is a side view of the shaft of the present application;

[0044] Figure 4 It is a bottom view of the shaft of the present application;

[0045] Figure 5 It is a front view of the shaft of the present application after adjustment;

[0046] Figure 6 It is a side view of the shaft of the present application after adjustment.

[0047] In the figure: 1, into the crab box body; 101, feed inlet; 102, discharging chute; 103, sorting discharging port; 104, water outlet; 105, guide plate; 106, push roller; 107, buffer plate; 108, shunt discharging port; 2, mounting box; 201, control box; 202, display screen; 203, mounting cavity; 204, visual camera; 205, fill light; 3, support frame; 301, roller; 4, circulating water pump; 401, water outlet pipe; 402, water inlet pipe; 5, distribution hopper; 6, movable sorting plate; 601, adjustment shaft; 602, sorting partition; 603, stepper motor. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0049] In order to solve the problem that the existing river crab sorting device is difficult to adjust the conveying position of crabs during the conveying of river crabs during the sorting process, please refer to Figure 1 Figure 4 The technical solutions of the present embodiment are as follows:

[0050] The river crab counting and sorting device based on image processing provided by the present embodiment comprises an entry crab box body 1, the upper end face of the entry crab box body 1 is provided with an integrally formed feed inlet 101, the lower part of the entry crab box body 1 is connected with an integrally formed discharging chute 102, the upper end of the discharging chute 102 is provided with a mounting box 2, the upper end face of the mounting box 2 is provided with a control box 201, a programmable logic controller is arranged inside the control box 201, the programmable logic controller comprises a processing unit, a visual camera 204 is arranged inside the mounting box 2, two movable sorting plates 6 are arranged in the discharging chute 102, a stepper motor 603 is arranged at the lower end of the movable sorting plate 6, and the stepper motor 603 and the programmable logic controller are connected through a plc module.

[0051] ​The lower part of the crab entering box body 1 is provided with a supporting frame 3, the lower ends of both sides of the supporting frame 3 are provided with rollers 301, one side of the supporting frame 3 is fixedly connected with a circulating water pump 4, the circulating water pump 4 is respectively provided with a water outlet pipe 401 and a water inlet pipe 402, the water inlet pipe 402 is used for connecting a water source, the upper end of the crab entering box body 1 is provided with a water outlet 104, the water outlet pipe 401 is communicated with the water outlet 104 of the upper end of the crab entering box body 1, the circulating water pump 4 and the stepping motor 603 are fixedly connected with the supporting frame 3 through a connecting frame, the supporting frame 3 is fixedly connected with the distributing hopper 5, the crab entering box body 1 and the discharging chute 102 through fixing screws, the river crab needing to be sorted is poured into the feeding port 101, the circulating water pump 4 guides clean water into the water outlet 104, the water outlet 104 discharges clean water to flow the river crab, the river crab is continuously pushed by the circulating water pump 4 to guide water flow, so as to prevent the river crab from being blocked in the conveying process, the structure is compact, the conveying and sorting work can be realized through the integrally formed crab entering box body 1, the device is convenient and practical, easy to operate, and has good structural stability, is beneficial to long-term use, and is convenient for disassembly and maintenance work.

[0052] The lower end of the discharging chute 102 is provided with a sorting discharging port 103, the lower part of the discharging chute 102 is provided with a distributing hopper 5, the upper end of the distributing hopper 5 is provided with a shunt discharging port 108, the distributing hopper 5 is communicated with the discharging chute 102 through the shunt discharging port 108, both sides of the sorting discharging port 103 are provided with sorting partition plates 602, the upper ends of the sorting partition plates 602 are fixedly connected with the lower ends of the movable sorting plates 6, and the upper ends of the two movable sorting plates 6 are abutted and connected with each other.

[0053] The inside of the crab entering box body 1 is provided with a guide plate 105, the lower end of the guide plate 105 is provided with a pushing roller 106, the pushing roller 106 is rotatably connected with the crab entering box body 1 and the discharging chute 102 through a bearing frame, the lower end of the pushing roller 106 is provided with an electric motor, and the electric motor is drivingly connected with the pushing roller 106 through a gear, the electric motor at the bottom of the pushing roller 106 drives the pushing roller 106 to rotate, the river crab between the two rows of pushing rollers 106 is pushed to move one by one, the river crab is guided into the two rows of pushing rollers 106 through the guide plate 105, and the river crab can be orderly conveyed through the cooperation of the buffer plate 107.

[0054] Specifically, by pouring the river crab to be sorted into the feed inlet 101, the circulating water pump 4 guides the clean water into the water outlet 104, the water outlet 104 discharges the clean water to flow the river crab, and the river crab is guided between the two rows of pushing rollers 106 through the guide plate 105, and the river crab is guided into the two rows of pushing rollers 106 through the guide plate 105. The buffer plate 107 can be used for orderly conveying work, and the river crab is pushed by the circulating water pump 4 to prevent the river crab from being blocked during conveying. The structure is compact, the crab entering box 1 can realize conveying and sorting work, is convenient and practical, is easy to operate, and has good structural stability, is conducive to long-term use, and is convenient for disassembly and maintenance work.

[0055] In order to solve the problem that the existing river crab sorting device is difficult to be photographed and recognized by the camera during transmission of the river crab during sorting, please refer to Figure 1 Figure 6 The embodiment provides the following technical scheme:

[0056] The lower end of the pushing roller 106 is provided with the buffer plate 107, the position below the buffer plate 107 corresponds to the position of the shunt discharge port 108, and the buffer plate 107 is fixedly connected with the discharge chute 102.

[0057] In fact, the mounting box 2 is fixedly connected with the crab entering box 1 through buckles. The mounting box 2 is externally provided with a display screen 202, the display screen 202 is electrically connected with a control box 201, the lower end surface of the mounting box 2 is provided with an integrally formed mounting cavity 203, the lower end of the mounting box 2 is provided with a light supplementing lamp 205, a visual camera 204 is fixedly connected with the mounting box 2 in the mounting cavity 203 through screws, the visual camera 204 is connected with a programmable logic controller through a plc module, the detected electrical signal is sent to the plc module, the plc module is processed, the integrity of the river crab is judged according to the programmable logic controller, the judgment data is adjusted, the two movable sorting plates 6 are simultaneously rotated inward, the channels on both sides of the discharge chute 102 are opened, the river crab is separated on both sides by the sorting partition plate 602, and the river crab is separated on both sides by the sorting partition plate 602. The river crab is introduced into the shunt discharge port 108 through the discharge chute 102, and is introduced into the distribution hopper 5 for sorting work. The structure is simple, convenient to operate and adjust, and is suitable for continuous sorting and counting work.

[0058] ​It needs to be specifically pointed out that the lower end of the movable sorting plate 6 is provided with an adjusting shaft 601, the adjusting shaft 601 penetrates the discharging chute 102 and is rotatably connected with the discharging chute 102 through a bearing, the lower end of the adjusting shaft 601 is fixedly connected with the stepping motor 603 through a shaft coupling, the upper end of the movable sorting plate 6 is abuttingly connected with the two sides of the discharging chute 102, the movable sorting plate 6 installed in the discharging chute 102 is adjusted synchronously through the two stepping motors 603, the two movable sorting plates 6 are simultaneously rotated inward, so that the passages on the two sides in the discharging chute 102 are opened, the river crab can be separated by the sorting partition plate 602 and slide to the two sides, and the river crab slides into the crab box body 1 and the discharging chute 102 for sorting, and the two stepping motors 603 are synchronously adjusted, and the two movable sorting plates 6 are simultaneously rotated outward.

[0059] An implementation method of a river crab counting and sorting device based on image processing, comprising the following steps:

[0060] Step one: pour the river crab to be sorted into the feeding port 101, the circulating water pump 4 guides the clean water into the water outlet 104, the water outlet 104 discharges the clean water to flow against the river crab, and the water flow is guided by the guide plate 105;

[0061] Step two: the river crab is guided by the guide plate 105 to enter between the two rows of pushing rollers 106, the pushing rollers 106 transport the river crab one by one, and after the river crab enters the discharging chute 102, the river crab is detected by the vision camera 204 in the installation box 2;

[0062] Step three: the vision camera 204 sends the detected electrical signal to the plc module, processes the electrical signal through the plc module, judges the integrity of the river crab according to the programmable logic controller, and adjusts the stepping motor 603 according to the judgment data;

[0063] Step four: the qualified river crab detected by the vision camera 204 is transported to the two sides of the movable sorting plate 6 through the pushing roller 106 and the buffer plate 107, is pushed by the water flow, enters one side of the sorting partition plate 602 from the sorting discharging port 103, and is counted by the vision camera 204 above the sorting discharging port 103;

[0064] Step five: the unqualified river crab detected by the vision camera 204 is counted by the vision camera 204 first, and then the movable sorting plate 6 is adjusted by the stepping motor 603, the movable sorting plate 6 is unfolded to the two sides, the movable sorting plate 6 abuts against the two sides of the discharging chute 102, the water flow pushes the unqualified river crab, the river crab is guided into the shunt discharging port 108, is guided into the distribution hopper 5 through the shunt discharging port 108, and the sorting is completed.

[0065] Specifically, the movable sorting plates 6 installed in the feeding chute 102 are adjusted synchronously by two stepping motors 603. When the two movable sorting plates 6 are rotated inward simultaneously, the passages on both sides of the feeding chute 102 are opened, and the river crabs can slide to both sides through the sorting partition plates 602, and then slide out of the crab box body 1 and the feeding chute 102 for sorting. When the two movable sorting plates 6 are rotated outward simultaneously, the river crabs sliding out of the buffer plate 107 are introduced into the shunt feeding port 108, and then guided into the distribution hopper 5 through the shunt feeding port 108 for sorting. The structure is simple, convenient to operate and adjust, and suitable for continuous sorting and counting work.

[0066] The processing unit comprises:

[0067] The first identification subunit is configured to perform edge identification on the to-be-identified river crab image collected by the visual camera 204, and determine a to-be-identified river crab sub-image in a corresponding region of a crab claw, a crab foot, a crab shell, or a crab navel in the to-be-identified river crab image.

[0068] The standard image library construction subunit is configured to construct multi-angle crab claw, crab foot, crab shell, or crab navel standard images based on the crab claw, crab foot, crab shell, or crab navel in the standardized river crab image.

[0069] The image quality evaluation subunit is configured to:

[0070] The image quality evaluation subunit is configured to:

[0071] The image quality evaluation subunit is configured to:

[0072] The comparison subunit is configured to compare the evaluation value of the to-be-identified river crab image with a preset evaluation threshold, and determine the to-be-identified river crab image as a target image when the evaluation value is greater than or equal to the preset evaluation threshold.

[0073] The second identification subunit is configured to input the target image into a pre-trained river crab sorting model for identification, and determine the corresponding category of the river crab in the target image.

[0074] The sorting subunit is configured to determine a sorting result and target position information based on the corresponding category of the river crab in the target image, and send the sorting result and target position information to the programmable logic controller. The sorting result is associated with the target position information.

[0075] The execution subunit is configured to perform a sorting operation on the river crab to be sorted based on the sorting result and the target position information.

[0076] In this embodiment, the evaluation value of the corresponding image region of the pincers, the pereiopods, the carapace or the visceral mass is determined based on the similarity score and the variance, and includes:

[0077] The pincers, the pereiopods, the carapace or the visceral mass perform weighted fusion on the similarity score and the variance of the corresponding region to obtain the evaluation value of the corresponding image region of the pincers, the pereiopods, the carapace or the visceral mass.

[0078] In this embodiment, the evaluation value of the image to be recognized is determined based on the evaluation value of the corresponding region of the pincers, the pereiopods, the carapace or the visceral mass and a preset algorithm, and includes:

[0079]

[0080] wherein, represents the evaluation value of the i-th image to be recognized; represents the background interference coefficient of the i-th image to be recognized; represents the overall light adaptation coefficient of the i-th image to be recognized; represents the evaluation value of the pincers region of the i-th image to be recognized; represents the evaluation value of the pereiopods region of the i-th image to be recognized; if the carapace appears in the image to be recognized, represents the evaluation value of the carapace region of the i-th image to be recognized; if the visceral mass appears in the image to be recognized, represents the evaluation value of the visceral mass region of the i-th image to be recognized; represents the sum of the reference values of the pincers, the pereiopods, the carapace or the visceral mass region evaluation value in the standardized river crab image; represents the occlusion coefficient of the key part in the i-th river crab image.

[0081] In this embodiment, obtained by calculating the contrast between the non-river crab region and the river crab region in the image, and the value range is [0, 1]; obtained by calculating the similarity between the brightness histogram of the image and the ideal distribution, and the value range is [0, 1]; obtained by calculating the proportion of pixels of the key part being occluded, and the value range is [0, 1).

[0082] The working principle and beneficial effects of the above technical solution are: through edge recognition on the river crab image, the sub-image of key parts such as the crab pincers, crab feet, crab shells or crab umbilical cords can be accurately determined. The features of these parts are crucial for judging the quality and category of river crabs, and accurate recognition helps subsequent classification more accurately; based on the standardized river crab image, a multi-angle standard image is constructed, so that in the image quality evaluation sub-unit, the to-be-recognized sub-image can be similarity calculated with the multi-angle standard image. In this way, the matching degree of the to-be-recognized river crab image with the standard can be more comprehensively evaluated, reducing misjudgment caused by factors such as shooting angle, and improving the accuracy of recognition; not only the similarity score is calculated, but also the variance of the image region is calculated based on the Laplace operator, and the evaluation value is determined comprehensively. This multi-dimensional evaluation method can more accurately reflect the quality of the image, avoid the limitations of a single indicator, and thus more reliably screen out target images that meet the requirements; the target image is input into the pre-trained river crab sorting model for recognition, and the learning ability and generalization ability of the model can accurately determine the category of the river crab according to its features, further improving the accuracy of sorting.

[0083] The training method of the river crab sorting model comprises:

[0084] A river crab training data set is obtained;

[0085] An initial river crab sorting model is constructed;

[0086] The river crab training data set is subjected to feature extraction by a multi-scale feature extraction network in the initial river crab sorting model, to obtain multi-scale river crab features;

[0087] Key discriminant features of the river crabs in the river crab training data set are calculated; the key discriminant features include composite shape features, structural features and composite color features of the river crabs:

[0088] The multi-scale river crab features and the key discriminant features are fused based on an attention-screening fusion module in the initial river crab sorting model, to obtain river crab fusion features;

[0089] A predicted classification result is obtained by a dynamic feature aggregation network and a target decoder in the initial river crab sorting model; the dynamic feature aggregation network is configured with scene adaptation reference features of each type of label, and comprises two feature interaction layers and one scene adaptation aggregation layer; the feature interaction layer calculates the similarity of the river crab fusion features and each group of reference features by cosine similarity and weighted sum; the scene adaptation aggregation layer adjusts the aggregation coefficient according to the actual interference scene of the sample; the target decoder normalizes the similarity of the aggregated features and the reference features into a predicted probability, and takes the maximum value of the probability as the predicted classification result;

[0090] The initial river crab sorting model is trained by constructing a dynamic weight joint loss function, and a target river crab sorting model is obtained by minimizing the total loss through an optimizer.

[0091] In this embodiment, the initial feature extraction unit is provided with a 7*7 convolution layer (output channel 32) and three residual blocks, each of which includes a 1*1 convolution layer (dimension reduction) + a 3*3 convolution layer (feature extraction) + a BN layer + a ReLU layer and a skip connection. The initial river crab features are obtained through the initial feature extraction unit.

[0092] In this embodiment, the multi-scale feature extraction network includes an initial feature extraction unit and a cross-scale interaction unit.

[0093] In this embodiment, the initial feature extraction unit is provided with a 7*7 convolution layer (output channel 32) and three residual blocks, each of which includes a 1*1 convolution layer (dimension reduction) + a 3*3 convolution layer (feature extraction) + a BN layer + a ReLU layer and a skip connection. The initial river crab features are obtained through the initial feature extraction unit.

[0094] The cross-scale interaction unit divides the initial river crab features into first features (shell diameter related, receptive field 16*16) and second features (carapace and cheliped details, receptive field 8*8) according to different scales. Based on the cross-scale attention mechanism, the first features are given a weight of 0.6 and the second features are given a weight of 0.4. The first features and the second features are fused to obtain multi-scale river crab features.

[0095] In this embodiment, the key discriminative features include the composite shape features, the structural features and the composite color features of the river crab: the composite shape features are calculated according to the formula ; wherein, represents the number of shell outline pixels, represents the number of carapace pixels, represents the number of cheliped pixels, and the mapping specification is when 4 two, 660≤ when 3 two, 560≤ when 2 two, 660≤ The specific mapping rules can be debugged according to the actual application scenario; the specification mapping rule: before system deployment, the threshold value is determined through calibration experiments. The specific steps are as follows: collect river crab sample images of known weight, calculate the value of each sample, establish the mapping relationship between the and the actual weight through linear regression, and determine the threshold value according to the mapping relationship; calibration explanation: the threshold value needs to be calibrated according to the camera resolution and shooting distance to ensure the consistency of the results under different devices. The structural features include edge density and cheliped extension degree; the edge density formula is Indicates the total number of pixels at the critical edge. Represents the number of pixels in the main body of the crab; formula for the extension of the chelipeds. , This represents the longest distance between the edges of the chelipeds. This represents the longest distance of the shell diameter; the freshness determination criterion is to determine the threshold using samples with known physiological states before system deployment. The specific steps are: collecting images of crab samples in different freshness states, and calculating the freshness of each sample... Value and The value is determined by cluster analysis to establish the threshold boundary. Example: <0.25 and >0.8 indicates freshness, 0.25≤ ≤0.4 and 0.5≤ ≤0.8 indicates near death. >0.4 and <0.5 indicates death; the specific mapping rules can be adjusted according to the actual application scenario; the composite color feature is calculated according to the formula. Calculate; where, , , This represents the average RGB channel value of the main area. represents the variance of the RGB channels, and c represents the mean of the grayscale image; the specific mapping rules can be adjusted according to the actual application scenario; variety discrimination threshold: before system deployment, the threshold is determined by samples of known varieties. The specific steps are: collect images of crab samples of different varieties, calculate the variance of each sample... The value is used to determine the optimal threshold through a classifier. Example: >510 (hairy crab) <320 (blue crab), 320≤ ≤510 (swimming crab).

[0096] In this embodiment, a dynamic weighted joint loss function is constructed, including: a binary cross-entropy loss function and a center loss function; wherein, the binary cross-entropy loss function includes: ,in This represents the true label of the i-th sample belonging to category c; This represents the predicted probability that the i-th sample belongs to category c; The weight of class c is dynamically calculated based on the number of samples of class c in the training set. Indicates the number of samples; Represents the total number of categories; the center loss function includes: ,in, This represents the fusion feature of the i-th sample; Indicates category The center vector; represents the norm, the square of the Euclidean distance between the feature and the class center is calculated; represents the category The dynamic adjustment coefficient is calculated according to the number of samples of the category and the difficulty; the total loss formula is wherein , According to the sample category proportion, it is dynamically adjusted.

[0097] In this embodiment, the attention-screening fusion module includes a feature concatenation layer, a double fully connected layer group, a Sigmoid weight layer, and a feature screening layer; the feature concatenation layer concatenates the multi-scale river crab features and the key discriminative features along the channel dimension; the double fully connected layer group includes a first fully connected layer group and a second fully connected layer group; the first fully connected layer group (average pooling + 2 fully connected layers (48→204 channels)) outputs global feature weights, and the second fully connected layer group (max pooling + 2 fully connected layers (48→204 channels)) outputs local feature weights; the Sigmoid weight layer maps the two types of weights to the (0, 1) interval; the feature screening layer retains the top 80% of the feature channels in terms of weight, deletes redundant channels, and obtains river crab fusion features.

[0098] In this embodiment, the scene adaptation reference features are obtained through a preset visual prompt library, and each type of label corresponds to 3 groups of reference features, which are adapted to water marks, tilts, and weak light scenes respectively; the visual prompt library contains standard images of each type of label in water marks, slight tilts (±15°), and weak light (300-400 lux) three typical scenes.

[0099] In this embodiment, the actual interference scene of the sample is exemplified: for example, the water mark scene gives the shell outer contour reference feature a weight of 1.1 times, the system automatically identifies the interference type according to the image analysis result, and dynamically adjusts the weight of the corresponding reference feature, the weight adjustment range is 0.9-1.2 times, and the specific value is determined by the interference degree evaluation.

[0100] The working principle and beneficial effects of the above technical solution are: the river crab sorting model trained effectively can accurately classify river crabs, and when applied to an actual river crab sorting system, it can greatly improve the efficiency and quality of sorting. Compared with the traditional manual sorting method, the model sorting is faster and more accurate, reduces manual errors and labor intensity, and improves the consistency and reliability of river crab sorting.

[0101] Working principle: when using, the river crab to be sorted is poured into the feeding port 101, the circulating water pump 4 guides the clean water into the water outlet 104, the water outlet 104 discharges the clean water to flow the river crab, the river crab is guided by the guide plate 105, and the motor at the bottom of the pushing roller 106 drives the pushing roller 106 to rotate, the river crab between the two rows of pushing rollers 106 is pushed to move one by one, the river crab is guided by the guide plate 105 to enter between the two rows of pushing rollers 106, and the buffer plate 107 can be used for orderly conveying work, and the river crab is pushed by the water flow guided by the circulating water pump 4 to prevent the river crab from being blocked during conveying, and the river crab is captured by the visual camera 204 during movement, the detected electrical signal is sent to the plc module, the plc module is processed, the integrity of the river crab is judged according to the programmable logic controller, the edge of the river crab image is recognized, the sub-image of the key part such as the crab claw, the crab foot, the crab shell or the crab navel can be accurately determined, the movable sorting plate 6 installed in the discharging chute 102 is adjusted according to the determined sub-image, the two movable sorting plates 6 are synchronously adjusted by the two stepping motors 603, and the two movable sorting plates 6 are synchronously adjusted by the two stepping motors 603. The channel on both sides of the discharging chute 102 is opened by rotating inward, the river crab is separated on both sides by the sorting partition plate 602 and slides out into the crab box body 1 and the discharging chute 102 for sorting, the two movable sorting plates 6 are synchronously adjusted by the two stepping motors 603, and the river crab sliding out of the buffer plate 107 is introduced into the shunt discharging port 108, and the river crab is sorted by the shunt discharging port 108 and introduced into the distributing hopper 5, the structure is simple, convenient to operate and adjust, suitable for continuous sorting and counting work, and the effectively trained river crab sorting model can accurately classify the river crab. The river crab sorting system can greatly improve the efficiency and quality of sorting.

[0102] It should be noted that, in the present document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0103] While embodiments of the present application have been shown and described with reference to the figures, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the application.

Claims

1. An image processing-based counting and sorting device for river crabs, comprising a crab entry box (1), characterized in that, The upper end face of the crab entering box body (1) is provided with an integrally formed feeding port (101), the lower end of the crab entering box body (1) is connected with an integrally formed discharging chute (102), the upper end of the discharging chute (102) is provided with a mounting box (2), the upper end face of the mounting box (2) is provided with a control box (201), the control box (201) is internally provided with a programmable logic controller, the programmable logic controller comprises a processing unit, the inside of the mounting box (2) is provided with a visual camera (204), two movable sorting plates (6) are arranged in the discharging chute (102), the lower end of the movable sorting plate (6) is provided with a stepping motor (603), the stepping motor (603) and the programmable logic controller are connected through a plc module; The lower end of the discharging chute (102) is provided with a sorting discharging port (103), the lower end of the discharging chute (102) is provided with a distribution hopper (5), the upper end of the distribution hopper (5) is provided with a distribution discharging port (108), the distribution hopper (5) is communicated with the discharging chute (102) through the distribution discharging port (108), the two sides of the sorting discharging port (103) are provided with sorting partition plates (602), the upper end of the sorting partition plate (602) is fixedly connected with the lower end of the movable sorting plate (6), the upper ends of the two movable sorting plates (6) are abutted with each other; The lower end of the discharging chute (102) is provided with a sorting discharging port (103), the lower end of the discharging chute (102) is provided with a distribution hopper (5), the upper end of the distribution hopper (5) is provided with a distribution discharging port (108), the distribution hopper (5) is communicated with the discharging chute (102) through the distribution discharging port (108), the two sides of the sorting discharging port (103) are provided with sorting partition plates (602), the upper end of the sorting partition plate (602) is fixedly connected with the lower end of the movable sorting plate (6), the upper ends of the two movable sorting plates (6) are abutted with each other; 2. The image processing-based river crab counting and sorting device according to claim 1, characterized in that, The lower end of the discharging chute (102) is provided with a sorting discharging port (103), the lower end of the discharging chute (102) is provided with a distribution hopper (5), the upper end of the distribution hopper (5) is provided with a distribution discharging port (108), the distribution hopper (5) is communicated with the discharging chute (102) through the distribution discharging port (108), the two sides of the sorting discharging port (103) are provided with sorting partition plates (602), the upper end of the sorting partition plate (602) is fixedly connected with the lower end of the movable sorting plate (6), the upper ends of the two movable sorting plates (6) are abutted with each other; 3. The image processing-based river crab counting and sorting apparatus according to claim 2, characterized in that, The lower end of the discharging chute (102) is provided with a sorting discharging port (103), the lower end of the discharging chute (102) is provided with a distribution hopper (5), the upper end of the distribution hopper (5) is provided with a distribution discharging port (108), the distribution hopper (5) is communicated with the discharging chute (102) through the distribution discharging port (108), the two sides of the sorting discharging port (103) are provided with sorting partition plates (602), the upper end of the sorting partition plate (602) is fixedly connected with the lower end of the movable sorting plate (6), the upper ends of the two movable sorting plates (6) are abutted with each other; The lower end of the discharging chute (102) is provided with a sorting discharging port (103), the lower end of the discharging chute (102) is provided with a distribution hopper (5), the upper end of the distribution hopper (5) is provided with a distribution discharging port (108), the distribution hopper (5) is communicated with the discharging chute (102) through the distribution discharging port (108), the two sides of the sorting discharging port (103) are provided with sorting partition plates (602), the upper end of the sorting partition plate (602) is fixedly connected with the lower end of the movable sorting plate (6), the upper ends of the two movable sorting plates (6) are abutted with each other; 4. The image processing-based river crab counting and sorting apparatus according to claim 3, characterized in that, The mounting box (2) is fixedly connected with the crab entering box body (1) through buckles, the mounting box (2) is externally provided with a display screen (202), the display screen (202) is electrically connected with a control box (201), the lower end surface of the mounting box (2) is provided with an integrally formed mounting cavity (203), the lower end of the mounting box (2) is provided with a light supplementing lamp (205), the visual camera (204) is fixedly connected with the mounting box (2) through screws in the inside of the mounting cavity (203), and the visual camera (204) is connected with a programmable logic controller through a plc module.

5. The image processing-based river crab counting and sorting apparatus according to claim 4, characterized in that, The lower end of the movable sorting plate (6) is provided with an adjusting shaft (601), the adjusting shaft (601) penetrates through the blanking chute (102) and is rotatably connected with the blanking chute (102) through bearings, the lower end of the adjusting shaft (601) is fixedly connected with the stepping motor (603) through a shaft coupling, and the upper end of the movable sorting plate (6) is abuttingly connected with the two sides of the blanking chute (102).

6. The image processing-based counting and sorting apparatus for river crabs according to claim 5, characterized in that, The processing unit comprises: The first identification subunit is used for edge identification on the image to be identified collected by the visual camera (204), and determines the image to be identified of the corresponding region of the crab pincers, crab feet, crab shells or crab umbilical cords in the image to be identified of the river crab; The standard image library construction subunit is used for constructing multi-angle crab pincers, crab feet, crab shells or crab umbilical cord standard images based on the crab pincers, crab feet, crab shells or crab umbilical cords in the standardized river crab images; The image quality evaluation subunit is used for: The image quality evaluation subunit is used for: The image quality evaluation subunit is used for: The comparison subunit is used for comparing the evaluation value of the image to be identified with a preset evaluation threshold value, and when it is determined that the evaluation value is greater than or equal to the preset evaluation threshold value, the image to be identified is regarded as a target image; The second identification subunit is used for inputting the target image into a pre-trained river crab sorting model for identification, and determining the corresponding category of the river crab in the target image; The sorting subunit is used for determining a sorting result and target position information based on the corresponding category of the river crab in the target image, and sending the sorting result and target position information to the programmable logic controller; the sorting result is associated with the target position information; The execution subunit is used for performing a sorting operation on the river crab to be sorted based on the sorting result and target position information.

7. The image processing-based counting and sorting apparatus for river crabs according to claim 6, characterized in that, The training method of the river crab sorting model comprises: Obtaining a river crab training data set; Constructing an initial river crab sorting model; Performing feature extraction on the river crab training data set through a multi-scale feature extraction network in the initial river crab sorting model to obtain multi-scale river crab features; The key discriminant features of the river crab in the river crab training data set are calculated; the key discriminant features include the composite shape features, structure features and composite color features of the river crab: The river crab fusion features are obtained by fusing the multi-scale river crab features and the key discriminant features based on the attention-filter fusion module in the initial river crab sorting model; The predicted classification result is obtained through the dynamic feature aggregation network and the target decoder in the initial river crab sorting model; the dynamic feature aggregation network configures the scene adaptation reference features of each label, and the dynamic feature aggregation network includes two feature interaction layers and one scene adaptation aggregation layer; the feature interaction layer calculates the similarity between the river crab fusion features and each group of reference features by cosine similarity and weighted sum; the scene adaptation aggregation layer adjusts the aggregation coefficient according to the actual interference scene of the sample; the target decoder normalizes the similarity of the aggregated features and the reference features into a predicted probability, and takes the maximum probability value as the predicted classification result; The target river crab sorting model is obtained by training the initial river crab sorting model through the dynamic weight joint loss function and minimizing the total loss through the optimizer.

8. A method of implementing the image processing-based counting and sorting apparatus for river crabs according to claim 7, characterized in that, Comprise the following steps: Step one: pour the river crab to be sorted into the feeding port (101), and the circulating water pump (4) guides the clean water into the water outlet (104), and the water outlet (104) discharges the clean water to flow the river crab, and the river crab is guided through the guide plate (105); Step two: the river crab is guided into the two rows of push rollers (106) through the guide plate (105), and the push rollers (106) transport the river crab one by one, and after the river crab enters the discharge chute (102), the vision camera (204) inside the installation box (2) is detected; Step three: the vision camera (204) sends the detected electrical signal to the plc module, which is processed by the plc module, and the overall integrity of the river crab is judged according to the programmable logic controller, and the step motor (603) is adjusted according to the judgment data; Step four: the river crab detected by the vision camera (204) is transported to the two sides of the movable sorting plate (6) through the push roller (106) and the buffer plate (107), and is pushed into the sorting baffle (602) on one side through the water flow, and is discharged from the sorting discharge port (103), and is counted by the vision camera (204) above the sorting discharge port (103); Step five: the unqualified river crab detected by the vision camera (204) is counted by the vision camera (204) first, and then the movable sorting plate (6) is adjusted by the step motor (603), the movable sorting plate (6) is expanded to the two sides, and after the movable sorting plate (6) abuts against the two sides of the discharge chute (102), the water flow pushes the unqualified river crab into the shunt discharge port (108), and the river crab is guided into the distribution hopper (5) through the shunt discharge port (108), and the sorting is completed.

Images