A neural network positioning swimming crab assisted coding method and system

Abstract

The present invention discloses a neural network positioning swimming crab auxiliary coding method, which aims to solve the problem that the current swimming crab is easy to be faked when setting a two-dimensional code mark on the packaging box and the current auxiliary coding mainly relies on mechanical structure fixation, resulting in poor positioning effect of living organisms, and includes the following steps: loading the swimming crab, and placing the swimming crab back shell upward on the pan-tilt; after obtaining the swimming crab image and processing it, the swimming crab image is input into the key point detection network to obtain key point information; according to the relationship between the key point information, the offset angle of the swimming crab is calculated, and the pan-tilt is adjusted according to the offset angle of the swimming crab; after the pan-tilt is adjusted, laser coding is performed using a laser coding module. The present invention uses a key point detection network to locate the swimming crab back shell, and then uses the pan-tilt to automatically adjust the position to face the laser marking terminal to achieve laser coding; the positioning effect is more accurate, and no manual intervention is required, reducing costs.

Description

Technical Field

[0001] The invention relates to the technical field of auxiliary coding, in particular to a neural network positioned swimming crab auxiliary coding method and system. Background Art

[0002] Currently, there is no unified traceability management for swimming crabs in the swimming crab market; although some swimming crabs on the market have QR code labels on the packaging boxes, merchants and users can scan the QR codes to view the relevant production date, breeding location and time, etc., but this type of QR code is easy to forge, and a packaging box can be used multiple times to sell fake crabs twice or more.

[0003] The existing technology mainly involves coding on the outer packaging box, which makes it easy for unscrupulous merchants to imitate the packaging box and put crabs of lower quality in it to pass off inferior products as good ones. In addition, the current auxiliary coding technology mainly relies on mechanical structure fixation, requires many links of human intervention, and has poor positioning effect on moving living organisms.

[0004] "A laser automatic coding device" disclosed in Chinese patent literature, with publication number CN218558344U, discloses a laser automatic coding device, including a product conveying unit and a laser coding unit; however, the Chinese patent with publication number CN218558344U is not applicable to the coding of living organisms. Summary of the invention

[0005] The invention solves the problem that it is easy to forge a two-dimensional code mark on a packaging box of swimming crabs and that the current auxiliary coding mainly relies on mechanical structure fixation, resulting in poor positioning effect of living organisms. A neural network positioning swimming crab auxiliary coding method and system are proposed, which uses a key point detection network to locate the back shell of the swimming crab, and then uses a pan-tilt head to automatically adjust the position to face the laser marking terminal to achieve laser coding; the positioning effect is more accurate, and no manual intervention is required, thereby reducing costs.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme: a neural network positioning swimming crab auxiliary coding method, comprising the following steps:

[0007] S1, loading the swimming crab, placing the swimming crab on the platform with its back shell facing upwards;

[0008] S2, after obtaining and processing the swimming crab image, the swimming crab image is input into the key point detection network to obtain key point information;

[0009] S3, calculating the offset angle of the swimming crab according to the relationship between the key point information, and adjusting the pan-tilt according to the offset angle of the swimming crab; S4, after the pan-tilt is adjusted, using the laser coding module to perform laser coding.

[0010] In the present invention, firstly, a loading and fixing device is used to place a swimming crab with its back shell facing upward on a pan-tilt platform, an image acquisition device is used to obtain a swimming crab image, and after scaling processing, the image enters a key point detection network. The key point detection network of the present invention is an improved Yolov3 network with a key point detection function. After obtaining the key point information, the pan-tilt platform is adjusted according to the calculated offset angle, and finally a laser coding operation is performed. The present invention adopts a neural network to automatically adjust the positioning after detection, effectively reducing the labor cost, and having higher accuracy.

[0011] Preferably, step S2 comprises the following steps:

[0012] S21, obtain the swimming crab image and scale it to 512*512*3 size, and input it into the key point detection network;

[0013] S22, after extracting features through the Darknet53 network, a feature map of size 16*16*1024 is obtained;

[0014] S23, extracts and fuses multi-scale information through the SPP module;

[0015] S24, extracts feature information through multiple convolution modules and outputs results at three feature layers of different scales;

[0016] S25, the final result is obtained through non-maximum suppression and out-of-bounds judgment.

[0017] In the present invention, after collecting the swimming crab image, the image is scaled to 512*512*3 and then input into the key point detection network. In the key point detection network, the feature is extracted by the Darknet53 network, and then the multi-scale information is extracted and fused by the SPP module. Then, the feature information is further extracted by a series of convolution modules, and the result is output. Finally, the final result is obtained after non-maximum suppression and out-of-bounds judgment.

[0018] Preferably, the key point detection network is trained using a number of labeled photos of the back of swimming crabs and labeling information, and after outputting the results, the loss function is calculated and back propagated using a stochastic gradient descent algorithm.

[0019] In the present invention, training is performed using 800 labeled photos of the back of swimming crabs and the labeled information.

[0020] Preferably, each of the feature layers has (box*4+classes+confidence+keypoints*2)*num_anchor output results, where box*4 represents the total number of anchor boxes in the current prediction layer*4 offset parameters, classes is the prediction category, confidence is the target confidence, keypoints*2 is the prediction key point information, keypoint is the number of key points, and num_anchor is the total number of anchor boxes generated by each feature layer.

[0021] In the present invention, the range of the prediction category is between 0 and 1, and the present invention includes two prediction categories, swimming crab and background. When the prediction category is greater than 0.5, it is represented as swimming crab, and in other cases it is represented as background; the range of the target confidence is between 0 and 1, indicating the confidence information that the current prediction box contains swimming crab. The higher the target confidence, the greater the probability of containing swimming crab and the more accurate the position of the box.

[0022] Preferably, the loss function is expressed as follows:

[0023] L loss =α*loss box +β*loss cls +γ*loss obj +η*loss landmark

[0024] Among them, loss box Expressed as the target prediction box regression loss, loss cls Expressed as target classification loss, loss obj Expressed as target confidence loss, loss landmark It is expressed as the key point coordinate loss, and α, β, γ, and η are the gain hyperparameters corresponding to different losses.

[0025] In the present invention, the loss function includes target prediction box regression loss, target classification loss, target confidence loss and key point coordinate loss.

[0026] Preferably, step S3 comprises the following steps:

[0027] S31, a line connecting the key points of the two sharp corners of the swimming crab and a line connecting the key points of the two eyes respectively form a first angle a and a second angle b with the horizontal line, a horizontal offset angle a+b is calculated, and the three axes of the gimbal are adjusted so that a+b is close to 0;

[0028] S32, respectively form a third angle c and a fourth angle d between the sharp corners and the eye line on both sides of the swimming crab and the vertical line, calculate the vertical offset angle c+d, and adjust the three axes of the gimbal to make c+d close to 0.

[0029] In the present invention, the offset angle of the swimming crab is calculated by connecting the corresponding key points, and the corresponding control instructions are generated according to the offset angle. The control instructions are sent to the pan-tilt head, and the pan-tilt head performs rotation adjustment.

[0030] Preferably, the target classification loss is specifically calculated by performing cross entropy loss calculation on each category value of the corresponding anchor box predicted by the network and the label value of the real match of the anchor box, and then taking the average of the sampled positive and negative sample anchor boxes to obtain the final classification loss.

[0031] Preferably, the offset parameters include the target frame center coordinates x and y and the height, width, h and w, and the key points are the two sharp corners and two eyes of the swimming crab's back shell.

[0032] In the present invention, the role of the offset parameter is to accurately locate the prediction frame.

[0033] A neural network positioning swimming crab auxiliary coding system, using the above-mentioned neural network positioning swimming crab auxiliary coding method, including

[0034] A feeding fixture, including a feeding belt, is used to feed the swimming crab into the pan-tilt platform;

[0035] The key point detection module has a key point detection network, which inputs the swimming crab image and outputs the key point information;

[0036] The PTZ control module sends control instructions to the PTZ to rotate the PTZ to a specified position;

[0037] Laser coding module, used to laser code the back shell of swimming crab.

[0038] In the present invention, the loading and fixing device places the swimming crab on the pan-tilt platform. After acquiring the image of the swimming crab, the image is processed and sent to the key point detection network of the key point detection module, and the key point information is output. After the offset angle is calculated, the pan-tilt control module controls the pan-tilt platform to rotate and adjust, and finally the laser coding module is used to perform laser coding.

[0039] The beneficial effects of the present invention are as follows: for a neural network positioning swimming crab assisted coding method and system of the present invention, a key point detection network is used to locate the back shell of the swimming crab, and then a pan-tilt head is used to automatically adjust the position to face the laser coding terminal to achieve laser coding; the positioning effect is more accurate, and no manual intervention is required, thereby reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of a key point detection network of a neural network positioning swimming crab assisted coding method and system of the present invention;

[0041] Figure 2It is a schematic diagram of a swimming crab of a neural network positioning swimming crab auxiliary coding method and system of the present invention;

[0042] Figure 3 The present invention is a schematic diagram of the calculation of the offset angle of a neural network positioning swimming crab assisted coding method and system. DETAILED DESCRIPTION

[0043] Embodiment 1:

[0044] This embodiment proposes a neural network positioning swimming crab assisted coding method, which includes the following steps.

[0045] Step S1, loading the swimming crab, placing the swimming crab with its back shell facing upward on the pan-tilt platform; specifically, a loading belt of a loading fixture delivers a single swimming crab with its back shell facing upward to the pan-tilt platform.

[0046] refer to Figure 1 , step S2, obtaining the swimming crab image and after processing, inputting the swimming crab image into the key point detection network to obtain key point information; this step mainly includes the following sub-steps.

[0047] Step S21, obtain the swimming crab image and scale it to a size of 512*512*3, and input it into the key point detection network; in this embodiment, the image is scaled before inputting into the key point detection network to meet the input requirements.

[0048] Step S22, extract features through the Darknet53 network to obtain a feature map with a size of 16*16*1024.

[0049] Step S23, extracting and fusing multi-scale information through the SPP module, so that swimming crabs of all sizes in the image can be detected.

[0050] Step S24, extract feature information through multiple convolution modules, and output results on three feature layers of different scales; each feature layer has the same number of output results; refer to Figure 1 .

[0051] More specifically, each feature layer has (box*4+classes+confidence+keypoints*2)*num_anchor output results, where box*4 represents the total number of anchor boxes in the current prediction layer*4 offset parameters, classes is the prediction category, confidence is the target confidence, keypoints*2 is the prediction key point information, keypoint is the number of key points, and num_anchor is the total number of anchor boxes generated by each feature layer. Specifically, the prediction category ranges from 0 to 1. The present invention includes two prediction categories, swimming crab and background. When the prediction category is greater than 0.5, it is represented as swimming crab, and in other cases it is represented as background; the target confidence ranges from 0 to 1, indicating the confidence information of the swimming crab contained in the current prediction box. The higher the target confidence, the greater the probability of containing swimming crab and the more accurate the position of the box.

[0052] In addition, the role of the offset parameter is to accurately locate the prediction frame; the offset parameters include the center coordinates x and y of the target frame and the height, width, and w. The key points are the two sharp corners and two eyes of the crab's back shell. The specific positions of the two sharp corners and two eyes can be found in Figure 2 .

[0053] Step S25, the final result is obtained through non-maximum suppression and cross-boundary determination.

[0054] For the key point detection network, it is trained with several labeled photos of the back of the swimming crab and the marking information. After the results are output, the loss function is calculated and back propagation is performed through the stochastic gradient descent algorithm. In this embodiment, it is specifically trained with 800 labeled photos of the back of the swimming crab and the marking information.

[0055] For the loss function, its expression is:

[0056] L loss =α*loss box +β*loss cls +γ*loss obj +η*loss landmark

[0057] In the above formula, loss box Expressed as the target prediction box regression loss, loss cls Expressed as target classification loss, loss obj Expressed as target confidence loss, loss landmark It is expressed as the key point coordinate loss, and α, β, γ, and η are gain hyperparameters corresponding to different losses. In this embodiment, α, β, γ, and η are 0.5, 0.05, 1, and 0.005, respectively.

[0058] Specifically, the target prediction box regression loss loss cls The cross-entropy loss is calculated between the category values ​​of the network prediction corresponding to the anchor box and the label value of the anchor box that actually matches the anchor box, and then the average of the sampled positive and negative sample anchor boxes is taken to obtain the final classification loss.

[0059] Target prediction box regression loss loss box The offset regression parameter predicted by the network relative to the anchor box is calculated using the CIOU method for positive and negative samples. The calculation formula is as follows:

[0060]

[0061] where v is a parameter that balances the aspect ratio,

[0062]

[0063] gt represents the target box and p represents the predicted box.

[0064] For target confidence loss obj If there is a true label in the anchor box, its value is CIOU_Loss, if there is no true label, it is 0.

[0065] Key point coordinate loss landmark Wing Loss is used for calculation; when the error is x, the L1 loss gradient magnitude is 1, and the optimization step size is x. The larger the error, the more optimization times. The L2 loss gradient magnitude is x, and the optimization step size is 1. The larger the error, the larger the gradient. Therefore, lnx loss is introduced to alleviate the disadvantage that L1 / L2 loss is too affected by large errors. The lnx gradient magnitude is The optimization step size is x 2 , when the error is small, the gradient is large and the step size is small, and when the error is large, the gradient is small and the step size is large. For large and small error scenarios, there is a good balance between gradient and step size. Considering that the error of the key point positioning task is large at the beginning, L1 loss is introduced to speed up the convergence speed. The overall formula is as follows:

[0066]

[0067] Where C = w-wln(1+|w| / ε) is a constant that ensures the smoothness of the piecewise function, and ∈ is a hyperparameter.

[0068] refer to Figure 3 , step S3, calculating the offset angle of the swimming crab according to the relationship between the key point information, and adjusting the pan / tilt according to the offset angle of the swimming crab; specifically including the following sub-steps.

[0069] Step S31: The key points of the two sharp corners of the swimming crab and the key points of the two eyes form a first angle a and a second angle b with the horizontal line, calculate the horizontal offset angle a+b, and adjust the three axes of the gimbal to make a+b close to 0. Step S32: The sharp corners and the eyes of the swimming crab on both sides form a third angle c and a fourth angle d with the vertical line, calculate the vertical offset angle c+d, and adjust the three axes of the gimbal to make c+d close to 0; the horizontal line and the vertical line are both standard lines in the positive direction, refer to Figure 3 The key point information includes the coordinate information of points 1, 2, 3, and 4 in the figure. The key points are connected to each other through the relationship between them to calculate the offset angle of the swimming crab relative to the positive direction. Specifically, the line connecting point 2 and point 3, the line connecting point 1 and point 4, the angles a and b between the two lines, and the horizontal offset angle a+b are calculated; similarly, the line connecting point 2 and point 1, the line connecting point 3 and point 4, the angles c and d between the two lines, and the vertical offset angle c+d are calculated. The horizontal offset angle and the vertical offset angle are made to approach 0 by adjusting the three axes of the gimbal.

[0070] Step S4, after the pan / tilt adjustment is completed, laser coding is performed using a laser coding module; specifically, laser coding is performed using a laser transmitter.

[0071] This embodiment also proposes a neural network positioning swimming crab auxiliary coding system, which uses the above-mentioned neural network positioning swimming crab auxiliary coding method, mainly including a feeding and fixing device, including a feeding belt, using the feeding belt to feed the swimming crab into the pan-tilt; a key point detection module, which is equipped with a key point detection network, inputs the swimming crab image, and outputs key point information; a pan-tilt control module, which sends a control instruction to the pan-tilt to rotate the pan-tilt to a specified position; a laser coding module, which performs laser coding on the back shell of the swimming crab. In this embodiment, the feeding and fixing device places the swimming crab on the pan-tilt, and after obtaining the swimming crab image, the image is processed and sent to the key point detection network of the key point detection module, and the key point information is output. After the offset angle is calculated, the pan-tilt control module controls the pan-tilt to rotate and adjust, and finally the laser coding module is used for laser coding.

[0072] The specific process of the present invention is as follows: first, a loading and fixing device is used to place the swimming crab on the pan-tilt platform with its back shell facing upwards, an image acquisition device is used to obtain the swimming crab image, and after scaling processing, it enters the key point detection network. The key point detection network of the present invention is an improved Yolov3 network with a key point detection function; after obtaining the key point information, the pan-tilt platform is adjusted according to the calculated offset angle, and finally the laser coding operation is performed; the present invention adopts a neural network to automatically adjust the positioning after detection, which effectively reduces the labor cost and has higher accuracy.

[0073] In this embodiment, reference Figure 1After collecting the image of the swimming crab, the image is scaled to 512*512*3 and then input into the key point detection network. In the key point detection network, the feature is extracted through the Darknet53 network, and then the multi-scale information is extracted and fused through the SPP module. Then, a series of convolution modules are used to further extract feature information and output the result. Finally, the final result is obtained after non-maximum suppression and out-of-bounds judgment.

[0074] In this embodiment, the loss function includes target prediction box regression loss, target classification loss, target confidence loss and key point coordinate loss.

[0075] In this embodiment, the offset angle of the swimming crab is calculated by connecting the corresponding key points, and the corresponding control instructions are generated according to the offset angle. The control instructions are sent to the gimbal, and the gimbal performs rotation adjustment.

[0076] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A neural network positioning swimming crab assisted coding method, characterized in that: The following steps are involved: S1, loading the swimming crab, placing the swimming crab on the platform with its back shell facing upwards; S2, after obtaining and processing the swimming crab image, the swimming crab image is input into the key point detection network to obtain key point information; S3, calculating the offset angle of the swimming crab according to the relationship between the key point information, and adjusting the gimbal according to the offset angle of the swimming crab; S4, after the pan / tilt adjustment is completed, use the laser marking module to perform laser marking: S3 includes: the key point connection line of the two sharp corners of the swimming crab and the key point connection line of the two eyes form a first angle a and a second angle b with the horizontal line respectively, calculate the horizontal offset angle a+b, and adjust the three axes of the gimbal to make a+b close to 0; the sharp corners and the eye connection lines on both sides of the swimming crab form a third angle c and a fourth angle d with the vertical line respectively, calculate the vertical offset angle c+d, and adjust the three axes of the gimbal to make c+d close to 0.

2. According to claim 1, a neural network positioning swimming crab assisted coding method is characterized in that: The step S2 comprises the following steps: S21, obtain the swimming crab image and scale it to 512*512*3 size, and input it into the key point detection network; S22, after extracting features through the Darknet53 network, a feature map of size 16*16*1024 is obtained; S23, extracts and fuses multi-scale information through the SPP module; S24, extracts feature information through multiple convolution modules and outputs results at three feature layers of different scales; S25, the final result is obtained through non-maximum suppression and out-of-bounds judgment.

3. A neural network positioning swimming crab assisted coding method according to claim 1 or 2, characterized in that: The key point detection network is trained by using a number of marked photos of the back of swimming crabs and marking information, and after outputting the results, the loss function is calculated and back propagated through a stochastic gradient descent algorithm.

4. The neural network positioning swimming crab assisted coding method according to claim 2 is characterized in that: There are (box*4+classes+confidence+keypoints*2)*num_anchor output results on each feature layer, where box*4 represents the total number of anchor boxes in the current prediction layer*4 offset parameters, classes is the prediction category, confidence is the target confidence, keypoints*2 is the prediction key point information, keypoint is the number of key points, and num_anchor is the total number of anchor boxes generated by each feature layer.

5. The neural network positioning swimming crab assisted coding method according to claim 3 is characterized in that: The expression of the loss function is as follows: L loss =α*loss box +β*loss cls +γ*loss obj +η*loss landmark Among them, loss box Expressed as target prediction box regression loss, loss cls Expressed as target classification loss, loss obj Expressed as target confidence loss, loss landmark It is expressed as the key point coordinate loss, and α, β, γ, and η are the gain hyperparameters corresponding to different losses.

6. The neural network positioning swimming crab assisted coding method according to claim 5 is characterized in that: The target classification loss is specifically calculated by performing cross entropy loss calculation on each category value of the corresponding anchor box predicted by the network and the label value of the real match of the anchor box, and then taking the average of the sampled positive and negative sample anchor boxes to obtain the final classification loss.

7. The neural network positioning swimming crab assisted coding method according to claim 4 is characterized in that: The offset parameters include the target frame center coordinates x and y and the height, width, h and w. The key points are the two sharp corners of the swimming crab's back shell and the two eyes.

8. A neural network positioning swimming crab assisted coding system, using a neural network positioning swimming crab assisted coding method according to any one of claims 1 to 7, characterized in that: include A feeding fixture, including a feeding belt, is used to feed the swimming crab into the pan-tilt platform; The key point detection module has a key point detection network, which inputs the swimming crab image and outputs the key point information; The PTZ control module sends control instructions to the PTZ to rotate the PTZ to a specified position; Laser coding module, used to laser code the back shell of swimming crab.

Images