Sow vaccine injection method and system based on deep learning and machine vision

Abstract

The invention discloses a sow vaccine injection method based on deep learning and machine vision. The sow vaccine injection method comprises the following steps that 1, an AGV chassis is controlled to arrive at a work station according to an instruction; 2, visually recognizing the posture of the sow to be injected, and if the recognition result is that the sow to be injected is in a non-standing posture, driving up the sow to be injected with manual assistance; if the recognition result is a standing posture, visually guiding the mechanical arm to move to a photographing position; 3, the RGBD camera shoots in real time to obtain the injection position and distance of the sow to be injected, and the mechanical arm moves to position the injection clamp to the pre-injection position; 4, injection is completed, and an injection result is recorded; the invention further discloses a sow vaccine injection system based on deep learning and machine vision. Manual operation is replaced by mechanical operation, automation of the vaccine injection process is achieved, direct contact between personnel and sows is reduced, and the risk of infectious disease transmission is reduced.

Description

Technical Field

[0001] The present invention relates to the field of machine injection technology, and specifically to a sow vaccine injection method and system based on deep learning and machine vision. Background Art

[0002] During the sow breeding process, sows will often get sick. Vaccination can improve the sow's immunity, reduce the occurrence and spread of diseases, and thus ensure the sow's health and production performance. Therefore, it is very necessary to vaccinate sows.

[0003] At present, sow immunization is mainly done manually, with people carrying vaccine liquid and immunization equipment, holding immunization syringes to immunize sows, and mainly relying on manual experience to perform injections behind the ears or pig buttocks. Due to the large number of sows, the workload and work intensity of personnel are large, and the work efficiency is low; sometimes the sows have a large stress response, causing injuries to sows or personnel; and direct contact between personnel and sows increases the risk of infection, which is not conducive to epidemic prevention. In addition, due to the remote location of pig farms and the relatively poor on-site environment, people are less willing to work in this industry and the labor cost is high. Therefore, the artificial immunization method is not only inefficient, but also has great safety hazards and infection risks.

[0004] Mechanized and automated breeding can amortize labor costs and expenses, thereby improving breeding efficiency, reducing breeding costs, and increasing the profit margin of pig farming companies. It can also effectively reduce direct contact between breeders and sows, which is beneficial to epidemic prevention work. Therefore, we need to design a sow vaccination method and system based on deep learning and machine vision to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a sow vaccine injection method and system based on deep learning and machine vision, which solves the problems of low manual operation efficiency and high infection risk during sow immunization.

[0006] According to one aspect of the present invention, a sow vaccine injection method based on deep learning and machine vision is provided, comprising the following steps:

[0007] Step 1: Control the AGV chassis to reach the work site according to the instructions;

[0008] Step 2: Visually identify the posture of the sow to be injected. If the recognition result is a non-standing posture, manual assistance is required to drive the sow to be injected up; if the recognition result is a standing posture, the vision guides the robotic arm to move to the photo-taking position;

[0009] Step 3: The RGBD camera takes real-time photos to obtain the injection position and distance of the sow to be injected, and tracks and identifies the movement state of the sow to be injected. If the sow to be injected is in motion, continue to identify the appropriate injection position; if the sow to be injected is stationary, the robotic arm moves to position the injection fixture to the pre-injection position, and continues to track and identify. If the injection site of the sow to be injected does not exist, the robotic arm needs to move back to the photo position to continue to identify the appropriate injection site; if the injection site of the sow to be injected has always existed, the injection head is close to the injection site of the sow to be injected and the laser distance is detected;

[0010] Step 4. When the distance between the injection head and the injection site of the sow to be injected is less than the preset distance, a command is issued to trigger the injection clamp, and the injection head is pushed out to contact the injection site of the sow to be injected. When the contact force reaches the set value, the button of the injection head is automatically triggered to complete the injection and record the injection result.

[0011] In some embodiments, in step 3, the RGBD camera takes real-time photos to obtain the injection position of the sow to be injected, including:

[0012] Obtain sow buttocks images through an RGBD camera, including RGB images and depth images;

[0013] Performing image processing on the collected images;

[0014] The processed image is input into the target recognition model to obtain the injection position of the sow to be injected.

[0015] In some embodiments, in step 3, the RGBD camera takes real-time photos to obtain the injection position of the sow to be injected, and also includes a pre-training process for the target recognition model, and the training process includes the following steps:

[0016] 1) Collect at least 1,500 sow buttocks images, including RGB images and depth images, establish image sample data, and group the image sample data, taking 70% of the data as the training set and the remaining 30% of the data as the test set;

[0017] 2) In the training set and the test set, the depth images are preprocessed, including filtering denoising and image enhancement, and then the preprocessed depth images are labeled with objects, while the RGB images do not need to be processed. The labeled training sets are then rotated and mirrored to obtain the training sample set and the test sample set;

[0018] 3) Use the training sample set to train the target recognition model, use the test sample set to test the model performance, and screen out the target recognition model with the best performance.

[0019] In some implementations, in step 2), object labeling is performed on the preprocessed depth image, including:

[0020] A rectangular positioning frame is used to mark the injection area in each image, and the center point, pixel normalized values ​​of width and height of each rectangular positioning frame and its corresponding category information are saved and output to a text file.

[0021] In some embodiments, the injection area includes an effective injection area and an ineffective injection area. The effective injection area is the entire buttocks, left buttocks and right buttocks of the sow, and the ineffective injection area is the vulva and tail.

[0022] In some embodiments, the step of inputting the processed image into a target recognition model to obtain an injection position of a sow to be injected comprises:

[0023] The processed image is input into the target recognition model to obtain the rectangular positioning frame coordinate information of the injection area of ​​the buttocks of the sow to be injected. When the target recognition model outputs multiple rectangular positioning frames and their corresponding confidence levels, the rectangular positioning frame corresponding to the highest confidence level is selected to generate the rectangular positioning frame coordinate information, and the center point of the rectangular positioning frame is used as the injection position of the sow to be injected.

[0024] In some embodiments, in step 4, if the button of the injection head cannot be pressed after the injection head is pushed out to the maximum distance, a feedback is given that the injection has failed, and a second attempt is made until it succeeds. After the injection is successful, the robotic arm retracts to the photo taking position.

[0025] According to another aspect of the present invention, a sow vaccine injection system based on deep learning and machine vision is provided, which is applied to the above-mentioned sow vaccine injection method based on deep learning and machine vision, including an AGV chassis, a mechanical arm and a laser radar are arranged on the AGV chassis, a controller is arranged in the AGV chassis, an injection clamp is arranged at the end of the mechanical arm, the injection clamp is used to clamp a needle-free syringe and trigger the injection button of the needle-free syringe, the needle-free syringe is provided with an injection head, and a visual recognition unit is assembled on the injection clamp, and the visual recognition unit and the mechanical arm are electrically connected to the controller; the visual recognition unit is used to collect image information and position information of the sow's buttocks, and the controller can drive the mechanical arm to move according to the image information and position information of the sow's buttocks collected by the visual recognition unit.

[0026] In some embodiments, the injection fixture is provided with an injection button trigger device, and the injection button trigger device is used to trigger a button of the injection head.

[0027] In some embodiments, the visual recognition unit includes an RGBD camera and a laser distance sensor, and both the RGBD camera and the laser distance sensor are electrically connected to the controller.

[0028] Beneficial effects of the present invention:

[0029] 1. The present invention realizes the automation of the vaccine injection process by replacing manual operation with mechanization, reduces direct contact between personnel and sows, and reduces the risk of infectious disease transmission.

[0030] 2. The present invention adopts RGBD camera vision guidance positioning, laser distance sensor adaptive distance adjustment, and injection button trigger device to ensure the accuracy of the entire process of the injection head moving from the photo position to the injection position. Visual guidance, laser distance adjustment and injection button trigger device jointly ensure the success rate and quality of vaccine injection.

[0031] 3. The present invention can autonomously navigate to the operation site through the AGV chassis, automatically identify the position of the sow through the visual recognition unit and perform automatic vaccination. After a task is issued, it can complete the immunization of all sows in the entire pig house, thereby improving the operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the operation process of the present invention;

[0033] Figure 2 It is a schematic diagram of the system structure of the present invention;

[0034] Figure 3 It is a schematic diagram of the structure of the injection fixture of the present invention;

[0035] Figure 4 It is a schematic diagram of a rectangular positioning frame of a sow's buttocks according to the present invention;

[0036] Figure 5 It is a schematic diagram of the operation logic flow of the present invention.

[0037] Among them, 1. AGV chassis; 2. LiDAR; 3. Robotic arm; 4. Needle-free syringe; 5. Injection fixture; 6. RGBD camera; 7. Injection head; 8. Injection button trigger device. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Please see attached Figure 1 -Attached Figure 5 The present invention provides a sow vaccine injection method based on deep learning and machine vision, comprising the following steps:

[0040] In large-scale piggeries, sows are kept in individual positioning pens and arranged in multiple rows in the piggery. The AGV chassis 1 is controlled to shuttle through the aisle between two rows of positioning pens. Each positioning pens is set with an operation station. After the AGV chassis 1 autonomously navigates to the station, it uses the RGBD camera 6 at the end of the mechanical arm 3 to identify the injection direction and distance, and performs the immunization operation. After the injection is completed, it enters the next station. The operation information of each station is recorded, including whether the injection has been performed and whether the injection is successful, to realize the process of automatic immunization of sows.

[0041] Step 1: Control the AGV chassis 1 to reach the work site according to the instructions;

[0042] Step 2: Visually identify the posture of the sow to be injected. If the recognition result is a non-standing posture, manual assistance is required to drive the sow to be injected up; if the recognition result is a standing posture, the vision guides the robot arm 3 to move to the photo taking position;

[0043] Step 3, the RGBD camera 6 takes real-time photos to obtain the injection position and distance of the sow to be injected, and at the same time tracks and identifies the movement state of the sow to be injected. If the sow to be injected is in motion, continue to identify the appropriate injection position; if the sow to be injected is in a stationary state, the robot arm 3 moves to position the injection fixture 5 to the pre-injection position (the injection head 7 is 50 mm away from the injection site of the sow), and continues to track and identify. If the injection site of the sow to be injected does not exist, the robot arm 3 needs to move back to the photo taking position to continue to identify the appropriate injection position; if the injection site of the sow to be injected has always existed, the injection head 7 is close to the injection site of the sow to be injected and detects the laser distance;

[0044] Step 4: When the distance between the injection head 7 and the injection site of the sow to be injected is less than the preset distance (i.e., the threshold value, which is 50 mm), a command is issued to trigger the injection clamp 5, and the injection head 7 is pushed out to contact the injection site of the sow to be injected. When the contact force reaches the set value, the button of the injection head 7 is automatically triggered to complete the injection and record the injection result.

[0045] In this embodiment, the injection site of the sow to be injected can be identified and tracked by the RGBD camera 6 and the laser distance sensor, the coordinate position of the target can be fed back, and the injection fixture 5 at the end of the robotic arm 3 can be guided to be positioned to the working position. The RGBD camera 6 has binocular stereoscopic vision that imitates human eyes. RGB images and depth images can be obtained through the RGBD camera 6. The depth image stores distance information, and the camera can convert it into a three-dimensional point cloud. Because the RGBD camera 6 is a binocular imaging similar to the human eye, there is a blind spot, and the distance cannot be obtained if it is too close. When the injection head 7 approaches the buttocks of the sow to be injected, the RGBD camera 6 only has RGB image data to judge the left and right movement, but there is no distance information from the sow's buttocks, so a laser distance sensor is also set to fill the distance data in the blind spot.

[0046] In the above step 2, the posture of the sow to be injected is visually identified, including collecting the whole image of the sow through the RGBD camera 6, including the RGB image and the depth image; performing image processing on the collected image, and inputting the processed image into the sow posture recognition model for posture recognition to identify the current posture information of the sow.

[0047] Visual recognition of the posture of the sow to be injected also includes a pre-training process for the sow posture recognition model, which includes: collecting at least 1,500 whole-body images of the sow, manually annotating them, and annotating the standing posture and non-standing posture of the whole-body images of the sow as training data, wherein the non-standing posture includes sitting, prone, lying on the belly and lying on the side; using the training data to train the sow posture recognition model to obtain a trained sow posture recognition model.

[0048] In the above step three, the RGBD camera 6 takes pictures in real time to obtain the injection position of the sow to be injected, including: obtaining the sow's buttocks image through the RGBD camera 6, including the RGB image and the depth image; performing image processing on the acquired image; inputting the processed image into the target recognition model to obtain the injection position of the sow to be injected.

[0049] The RGBD camera 6 takes real-time photos to obtain the injection position of the sow to be injected, and also includes a pre-training process for the target recognition model. The training process includes the following steps:

[0050] 1) Collect at least 1,500 sow buttocks images, including RGB images and depth images, establish image sample data, and group the image sample data, taking 70% of the data as the training set and the remaining 30% of the data as the test set;

[0051] 2) In the training set and the test set, the depth images are preprocessed, including filtering denoising and image enhancement, and then the preprocessed depth images are labeled with objects, while the RGB images do not need to be processed. The labeled training sets are then rotated and mirrored to obtain the training sample set and the test sample set;

[0052] 3) Use the training sample set to train the target recognition model, use the test sample set to test the model performance, and screen out the target recognition model with the best performance.

[0053] In the above step 2), the pre-processed depth image is marked by target (manual marking), including marking the injection area in each image with a rectangular positioning frame, and saving the center point, width and height pixel normalized values ​​and corresponding category information of each rectangular positioning frame, and outputting them into a text file. Among them, the injection area includes a valid injection area and an invalid injection area. The valid injection area is the entire buttocks, left buttocks and right buttocks of the sow, and the invalid injection area is the vulva and tail.

[0054] It should be noted that when vaccinating sows, it is easy to hit the vulva or tail. And considering that sometimes the sow is only sideways, the RGBD camera 6 may only see the left half of the buttocks or the right half of the buttocks when taking pictures in a fixed position, and there is a target of half the buttocks. Therefore, a multi-part marking method is adopted. When marking the entire buttocks of the sow, the tail, vulva, left buttocks and right buttocks are also marked. When the entire buttocks of the sow and the left buttocks and right buttocks (appearing at the same time or one) are detected, they are valid injection areas. Only the entire buttocks of the sow appear, which is a large range, but the injection point is a very small place, and the area must be reduced by combining the left and right buttocks. The vulva and tail are invalid injection areas. These 5 categories (the entire buttocks of the pig, the left buttocks, the right buttocks, the vulva and the tail) need to be marked separately for each picture.

[0055] In the above step 3), the target recognition model is trained using the training sample set, the loss value between the prediction box and the annotation box is minimized through an optimization algorithm (such as the gradient descent algorithm), the model parameters are adjusted, and finally a set of network weight parameters with high accuracy is obtained; the model performance is tested using the test sample set, and the model's hyperparameters (such as learning rate, convolution kernel size, etc.) are adjusted according to the verification results to optimize the model performance.

[0056] The processed image is input into the target recognition model to obtain the injection position of the sow to be injected, including: inputting the processed image into the target recognition model to obtain the rectangular positioning frame coordinate information of the injection area of ​​the buttocks of the sow to be injected, wherein when the target recognition model outputs multiple rectangular positioning frames and their corresponding confidence levels, the rectangular positioning frame corresponding to the highest confidence level is selected to generate the rectangular positioning frame coordinate information, and the center point of the rectangular positioning frame is used as the injection position of the sow to be injected.

[0057] In this embodiment, the target recognition model is a YOLO model, which is pre-trained to accurately identify the buttocks of a sow to be injected. The YOLO model outputs a rectangular positioning frame that identifies the injection area of ​​the buttocks of the sow to be injected and the corresponding confidence. When multiple rectangular positioning frames of different sizes and positions and the confidences corresponding to the rectangular positioning frames are output, the rectangular positioning frame with the highest confidence, which is most likely to be the injection area of ​​the pig's buttocks, is selected to generate rectangular positioning frame coordinate information, and the center point of the rectangular positioning frame is used as the injection position of the sow to be injected, and the left and right injection positions are the center points of the rectangular positioning frames of the left buttock and the right buttock (or within a certain offset threshold range).

[0058] During the whole injection process, the RGBD camera 6 is in a state of taking pictures all the time. If the rectangular positioning frame category is not found as the injection area of ​​the sow's buttocks, it will wait; after a period of time, an alarm will be issued that there is no injection target. In addition, the real-time monitoring of the movement state of the sow to be injected can be realized. If the left and right pixel movement amount of the injection point of the RGBD camera 6 is less than the preset movement threshold, it is considered to be in a static state (injectable state); when the movement amount is greater than the preset movement threshold, it is considered to be in a moving state, and the mechanical arm 3 is retracted to continue identification.

[0059] In the above step 4, if the injection head 7 is pushed out to the maximum distance and the button of the injection head 7 cannot be pressed, it is fed back that the injection failed, and a second attempt is made until it succeeds. After the injection is successful, the robot arm 3 retreats to the photo taking position.

[0060] The RGBD camera 6 and laser distance sensor of the present invention can identify the state and position of the sow in real time and feed back to the control center. Visual guidance positioning is adopted, combined with the adaptive distance adjustment of the laser distance sensor, and the distance perception function of the three-dimensional vision of the RGBD camera 6 is utilized. The conversion relationship from pixels to actual distance is used to obtain the distance between the RGBD camera 6 and the injection point at this time, and the visual guidance robot arm 3 moves to the injection position. The injection fixture 5 is triggered to perform automatic injection and feedback the injection result. If successful, the injection process is completed, the robot arm 3 is retracted, and the AGV chassis 1 is controlled to enter the site of the next positioning fence. If it fails, the RGBD camera 6 and the laser distance sensor perform secondary identification feedback and perform a second injection. The present invention uses visual guidance, laser ranging and triggering structure to ensure the success rate and quality of injection.

[0061] Guided positioning: using vision-guided positioning, combined with the adaptive distance adjustment of the laser distance sensor at the end of the robot 3, the distance between the RGBD camera 6 and the injection point is obtained through the conversion relationship from pixels to actual distance.

[0062] Distance adjustment: The laser distance sensor measures the distance between the RGBD camera 6 and the injection point in real time, and adjusts the position of the robot arm 3 through the control algorithm to ensure that the distance between the injection head 7 and the injection point is within the set range.

[0063] Injection position determination: When the distance between the injection head 7 and the injection site of the sow is less than the preset distance, an injection instruction is issued to ensure the accuracy of the injection position.

[0064] Injection start: After the injection fixture 5 is started, the injection head 7 is pushed forward to contact the target through the injection button trigger device 8. After the contact force reaches the set value, the button of the injection head 7 is automatically triggered to realize the injection function.

[0065] Injection process: After the injection head 7 contacts the sow's skin, the injection button is automatically triggered and the vaccine is injected into the sow's body. After the injection is completed, the injection head 7 is retracted for 1 second to ensure that the vaccine is fully injected, and a feedback signal is sent to the control center to retract the robotic arm 3.

[0066] Result Recording: The system records the result of each injection, including whether the injection was successful. If the injection head 7 fails to press the button when it is pushed out to the maximum distance, the system will record the injection failure and retry until the injection is successful.

[0067] Re-identification: If the sow injection point identification result is empty when approaching the injection position, the sow has moved. At this time, return to the photo taking position and continue waiting for the injection opportunity.

[0068] Continuous monitoring: The system continuously monitors the position of the sow through the visual system until it re-identifies the appropriate injection position and performs the injection operation again.

[0069] Process assurance: Through visual guidance, laser distance adjustment and contact trigger structure, the accuracy and reliability of the entire process of the injection head 7 moving from the photographing position to the injection position are guaranteed to avoid wasting vaccines or injecting the wrong part due to target movement.

[0070] Please see attached Figure 1 -Attached Figure 5The present invention also provides a sow vaccine injection system based on deep learning and machine vision, which is applied to the above-mentioned sow vaccine injection method based on deep learning and machine vision, including an AGV chassis 1, a mechanical arm 3 and a laser radar 2 are arranged on the AGV chassis 1, a controller is arranged in the AGV chassis 1, an injection clamp 5 is arranged at the end of the mechanical arm 3, the injection clamp 5 is used to clamp the needle-free syringe 4 and trigger the injection button of the needle-free syringe 4, the needle-free syringe 4 is provided with an injection head 7, and a visual recognition unit is assembled on the injection clamp 5, and the visual recognition unit and the mechanical arm 3 are both electrically connected to the controller; the visual recognition unit is used to collect image information and position information of the sow's buttocks, and the controller can drive the mechanical arm 3 to move according to the image information and position information of the sow's buttocks collected by the visual recognition unit.

[0071] The injection fixture 5 is provided with an injection button trigger device 8, which is used to trigger the button of the injection head 7. The visual recognition unit includes an RGBD camera 6 and a laser distance sensor, and the RGBD camera 6 and the laser distance sensor are both electrically connected to the controller. The RGBD camera 6 and the laser distance sensor are both installed on the injection fixture 5, and the laser distance sensor is located near the RGBD camera 6.

[0072] In this embodiment, the sow vaccination system is a sow vaccination robot. The robot uses SLAM laser navigation technology for autonomous navigation, uses a laser radar 2 to build a map, and then automatically locates its own position based on the currently observed environmental information, and then goes to the work site.

[0073] Working process of sow vaccination system:

[0074] Step 1: Arrive at the work site: After the AGV chassis 1 arrives at the work site, visual recognition is used to determine whether the pig is in a suitable state for immunization. If the pig lies down, immunization cannot be performed, and manual assistance is required to get the pig up.

[0075] After controlling the AGV chassis 1 to arrive at the designated work site, the sow is identified through the visual system to determine whether the sow is in a standing state. If the sow is in a lying state, the system will sound an alarm to prompt manual work to get the sow up.

[0076] Step 2: Confirm the status: When the visual judgment confirms that the pig's status is suitable for injection, the vision guides the robot arm 3 to move, first to the photo position, and the RGBD camera 6 takes photos in real time to obtain the target position and distance to be injected.

[0077] After the visual system confirms that the sow is in a suitable condition for injection, it guides the robotic arm 3 to move to the photo-taking position to obtain the image and distance data of the sow's injection position in real time to ensure the accuracy of the injection position.

[0078] Step 3: Detection stability: If the change of the target position is small within a certain period of time, it means that the sow is in a quiet state and suitable for injection; otherwise, it is in an active state and needs to be further identified.

[0079] The system monitors the movement of the sow within a certain period of time. If the target position changes slightly, it is judged that the sow is in a quiet state and suitable for injection. If the sow is active, the system continues to monitor until the appropriate time for injection is found.

[0080] Step 4: Move the robot arm 3: The robot arm 3 moves to position the injection fixture 5 to the pre-injection position (the injection head 7 is 50 mm away from the injection site of the sow), and continue to identify the target. If the target is still there, approach the target and check the laser distance to prevent the sow from moving during the approach process, resulting in incorrect injection position or empty injection.

[0081] The robot arm 3 moves the injection fixture 5 to the pre-injection position and measures the distance through the laser sensor to ensure that the distance between the injection head 7 and the injection position remains within 50 mm, continuously identifying the target position to prevent the sow from moving and causing injection failure.

[0082] Step 5: Repositioning: If the target does not exist, it means that the sow has moved when the robotic arm 3 approaches. In this case, the robotic arm 3 needs to move to the photo taking position to continue to identify the appropriate injection position.

[0083] If the recognition target does not exist, the robot arm 3 will move back to the photographing position and re-identify the position of the sow until a new suitable injection position is found and the injection operation is performed again.

[0084] Step 6, injection operation: when the contact distance is less than the threshold (to ensure that the injection head 7 is in close contact with the injection site), a command is issued to trigger the injection clamp 5, and the injection head 7 is pushed out to contact the injection site of the pig, and the button of the injection head is triggered at the same time to complete the injection.

[0085] When the contact distance between the injection fixture 5 and the injection site of the sow is less than the set threshold, the system issues an injection instruction, the injection head 7 is pushed out and the injection button is triggered to complete the injection operation, ensuring that the vaccine is accurately injected into the sow. It should be noted that the threshold is that the injection head 7 is 50mm away from the injection site of the sow.

[0086] Step 7, result recording: The injection result is recorded at the same time. If the button cannot be pressed when the injection head 7 is pushed out to the maximum distance, the injection is reported as failed, and a second attempt is made until it succeeds. After the injection is successful, the robot arm retreats to the photo taking position.

[0087] The system records the results of each injection, including whether the injection was successful. If the injection head 7 fails to press the button when it is pushed out to the maximum distance, the system will record the injection failure and retry until the injection is successful.

[0088] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A sow vaccination method based on deep learning and machine vision, characterized in that: The following steps are involved: Step 1: Control the AGV chassis to reach the work site according to the instructions; Step 2: Visually identify the posture of the sow to be injected. If the recognition result is a non-standing posture, manual assistance is required to drive the sow to be injected up; if the recognition result is a standing posture, the vision guides the robotic arm to move to the photo-taking position; Step 3: The RGBD camera takes real-time photos to obtain the injection position and distance of the sow to be injected, and tracks and identifies the movement state of the sow to be injected. If the sow to be injected is in motion, continue to identify the appropriate injection position; if the sow to be injected is stationary, the robotic arm moves to position the injection fixture to the pre-injection position, and continues to track and identify. If the injection site of the sow to be injected does not exist, the robotic arm needs to move back to the photo position to continue to identify the appropriate injection site; if the injection site of the sow to be injected has always existed, the injection head is close to the injection site of the sow to be injected and the laser distance is detected; Step 4. When the distance between the injection head and the injection site of the sow to be injected is less than the preset distance, a command is issued to trigger the injection clamp, and the injection head is pushed out to contact the injection site of the sow to be injected. When the contact force reaches the set value, the button of the injection head is automatically triggered to complete the injection and record the injection result.

2. A sow vaccine injection method based on deep learning and machine vision according to claim 1, characterized in that: In step 3, the RGBD camera takes real-time photos to obtain the injection position of the sow to be injected, including: Obtain sow buttocks images through an RGBD camera, including RGB images and depth images; Performing image processing on the collected images; The processed image is input into the target recognition model to obtain the injection position of the sow to be injected.

3. A sow vaccine injection method based on deep learning and machine vision according to claim 2, characterized in that: In the step 3, the RGBD camera takes pictures in real time to obtain the injection position of the sow to be injected, and also includes a pre-training process for the target recognition model, and the training process includes the following steps: 1) Collect at least 1,500 sow buttocks images, including RGB images and depth images, establish image sample data, and group the image sample data, taking 70% of the data as the training set and the remaining 30% of the data as the test set; 2) In the training set and the test set, the depth images are preprocessed, including filtering denoising and image enhancement, and then the preprocessed depth images are labeled with objects, while the RGB images do not need to be processed. The labeled training sets are then rotated and mirrored to obtain the training sample set and the test sample set; 3) Use the training sample set to train the target recognition model, use the test sample set to test the model performance, and screen out the target recognition model with the best performance.

4. A sow vaccine injection method based on deep learning and machine vision according to claim 3, characterized in that: In the step 2), target labeling is performed on the preprocessed depth image, including: A rectangular positioning frame is used to mark the injection area in each image, and the center point, pixel normalized values ​​of width and height of each rectangular positioning frame and its corresponding category information are saved and output to a text file.

5. A sow vaccine injection method based on deep learning and machine vision according to claim 4, characterized in that: The injection area includes an effective injection area and an ineffective injection area. The effective injection area is the entire buttocks, left buttocks and right buttocks of the sow, and the ineffective injection area is the vulva and tail.

6. A sow vaccine injection method based on deep learning and machine vision according to claim 4, characterized in that: The step of inputting the processed image into the target recognition model to obtain the injection position of the sow to be injected includes: The processed image is input into the target recognition model to obtain the rectangular positioning frame coordinate information of the injection area of ​​the buttocks of the sow to be injected. When the target recognition model outputs multiple rectangular positioning frames and their corresponding confidence levels, the rectangular positioning frame corresponding to the highest confidence level is selected to generate the rectangular positioning frame coordinate information, and the center point of the rectangular positioning frame is used as the injection position of the sow to be injected.

7. The sow vaccine injection method based on deep learning and machine vision according to claim 1, characterized in that: In step 4, if the injection head is pushed out to the maximum distance and the button of the injection head cannot be pressed, feedback indicates that the injection has failed, and a second attempt is made until it succeeds. After the injection is successful, the robotic arm returns to the photo taking position.

8. A sow vaccination system based on deep learning and machine vision, characterized in that: A sow vaccination method based on deep learning and machine vision applied to any one of claims 1-7, comprising an AGV chassis, a robotic arm and a laser radar are arranged on the AGV chassis, a controller is arranged in the AGV chassis, an injection clamp is arranged at the end of the robotic arm, the injection clamp is used to clamp a needle-free syringe and trigger the injection button of the needle-free syringe, the needle-free syringe is provided with an injection head, a visual recognition unit is assembled on the injection clamp, and the visual recognition unit and the robotic arm are both electrically connected to the controller; the visual recognition unit is used to collect image information and position information of the sow's buttocks, and the controller can drive the robotic arm to move according to the image information and position information of the sow's buttocks collected by the visual recognition unit.

9. A sow vaccine injection system based on deep learning and machine vision according to claim 8, characterized in that: The injection clamp is provided with an injection button triggering device, and the injection button triggering device is used to trigger the button of the injection head.

10. The sow vaccination system based on deep learning and machine vision according to claim 8, characterized in that: The visual recognition unit includes an RGBD camera and a laser distance sensor, and both the RGBD camera and the laser distance sensor are electrically connected to the controller.

Images