Morinda officinalis pose detection method and system based on machine vision and medium

Abstract

The invention relates to the field of computer vision and deep learning, in particular to a morinda officinalis pose detection method and system based on machine vision and a medium, and the method comprises the steps: collecting and marking morinda officinalis data information; a morinda officinalis pose detection model based on the yo-v8 is trained; using a morinda officinalis pose detection model based on yo-v8 to obtain a preliminary prediction result, optimizing the preliminary prediction result by using an area threshold filtering method, and calculating a grabbing coordinate and a clamping angle of morinda officinalis under camera coordinates; camera coordinates are converted into world coordinates through camera calibration to guide a mechanical arm to grab a single morinda officinalis, accurate pose detection and accurate positioning are achieved, a single medicinal material easy to grab is found, and the overall production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep learning, and particularly to a Morinda officinalis pose detection method, system and medium based on machine vision. Background Art

[0002] Morinda officinalis How is the dried root of the plant Morinda officinalis in the Rubiaceae family. It is oblate cylindrical, slightly curved, with unequal lengths, and its diameter ranges from 5 mm to 20 mm, showing grayish yellow or dark gray. In the traditional processing process of Morinda officinalis, the core of Morinda officinalis is often separated manually, and the meat part is reserved for medicinal material processing.

[0003] With the development of technology, processing Morinda officinalis through mechanical automation has gradually become a popular method, reducing the degree of manual participation and improving production efficiency and safety. However, in the mechanical automation operation, how to accurately grasp and process Morinda officinalis has become the key to processing. Therefore, how to perform accurate pose detection, locate Morinda officinalis, and find a single Morinda officinalis that is easy to grasp is the key to improving the production efficiency of Morinda officinalis. For example, Patent CN118618873A discloses an automatic feeding device and method for Morinda officinalis, which reduces the degree of manual participation and improves production efficiency and safety. However, this patent does not propose a specific and effective method for whether the Morinda officinalis in the vibrating table meets the pose recognition requirements and the pose recognition of Morinda officinalis after meeting the recognition requirements.

[0004] Therefore, providing an efficient and accurate pose detection method for automatically identifying, locating, and selecting suitable Morinda officinalis for processing is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] (I) Technical Problems to be Solved

[0006] The main purpose of the present invention is to provide a Morinda officinalis pose detection method, system and medium based on machine vision to solve the above technical problems.

[0007] (II) Technical Solutions

[0008] To achieve the above purpose, the present invention provides a Morinda officinalis pose detection method based on machine vision, including the steps of:

[0009] S1, collecting Morinda officinalis data information and performing annotation: collecting the RGB image information of Morinda officinalis on the processing platform; using an image annotation tool to annotate the collected RGB image information of Morinda officinalis to obtain an annotated Morinda officinalis image sample; wherein, the annotation of the Morinda officinalis image sample includes two types of samples, namely a single Morinda officinalis image sample and a Morinda officinalis stack image sample;

[0010] S2. Train the Morinda officinalis pose detection model based on YOLO - v8: Use the YOLO - v8 model to train based on the labeled Morinda officinalis image samples in step S1. After model training, obtain the Morinda officinalis pose detection model based on YOLO - v8;

[0011] S3. Use the Morinda officinalis pose detection model based on YOLO - v8 to predict the Morinda officinalis image data to be pose - detected on the processing platform, and obtain a preliminary prediction result;

[0012] S4. Optimize the preliminary prediction result using the area threshold filtering method, and judge whether the preliminary prediction result is a single Morinda officinalis based on the determined actual area threshold S1;

[0013] S5. For the prediction result judged as a single Morinda officinalis in step S4, calculate the grasping coordinates o(x0, y0) and the clamping angle θ of the single Morinda officinalis in the camera coordinate system;

[0014] S6. Convert the camera coordinate to the world coordinate through camera calibration to guide the robotic arm to grasp a single Morinda officinalis.

[0015] Preferably, after step S6, it further includes:

[0016] S7. When all single Morinda officinalis have been grasped, the processing platform activates the vibration mechanism to disperse the Morinda officinalis pile, and then repeat steps S3 to S6 until all Morinda officinalis have been grasped. All the Morinda officinalis are all the Morinda officinalis to be processed in the production process.

[0017] Preferably, the acquisition of the RGB image information of Morinda officinalis on the processing platform in step S1 includes: Set the camera position directly above the processing platform, and collect the RGB pictures of Morinda officinalis placed on the processing platform at an angle perpendicular to the processing platform.

[0018] Preferably, the labeled Morinda officinalis image samples based on step S1 in step S2 include: Randomly divide the labeled Morinda officinalis image samples in step S1 into a training set, a test set, and a prediction set to form a data set for model training.

[0019] Preferably, the determined actual area threshold S1 in step S4 is obtained through the following steps:

[0020] S41. Calculate the pixel equivalent A on the processing platform using the checkerboard calibration method;

[0021] S42. Calculate the actual area corresponding to each pixel unit: Based on the pixel equivalent A obtained in step S41, use the formula S = A 2Calculate the actual area corresponding to each pixel unit, where p represents the number of pixels;

[0022] S43. Image preprocessing and connected component analysis. Perform a series of image preprocessing and connected component analysis on the image classified in step S43. Based on a preset pixel threshold p1, obtain the actual area threshold S1 through the formula S1 = A 2 p1, where the image preprocessing and connected component analysis include: performing grayscale processing to obtain a grayscale image, then performing median filtering, Otsu binarization, erosion, dilation, traversing and retaining the largest connected component except the background, and calculating the total number of pixels within the largest connected component.

[0023] Preferably, the calculation method of the preset pixel threshold p1 includes: determining the preset pixel threshold p1 by statistically analyzing the differences in the number of pixels corresponding to three states of a single Morinda officinalis, two stacked Morinda officinalis, and multiple stacked Morinda officinalis, or setting the pixel threshold p1 based on experience and knowledge in the field of Morinda officinalis.

[0024] Preferably, step S4 determines whether it is a single Morinda officinalis based on the determined actual area threshold S1, including: judging the actual area of the single Morinda officinalis image obtained in the preliminary prediction result with the actual area threshold S1 as the judgment criterion. If the actual area of the predicted single Morinda officinalis image is less than or equal to the actual area threshold S1, it is judged as a single Morinda officinalis; if the actual area of the predicted single Morinda officinalis image is greater than the actual area threshold S1, it is not a single Morinda officinalis.

[0025] Preferably, S51. Extract the image information in the prediction box determined to be a single Morinda officinalis in step S4;

[0026] S52. Perform image preprocessing on the image information in step S51. The image preprocessing includes: performing grayscale processing to obtain a grayscale image, then performing median filtering, Otsu binarization, erosion, dilation, and traversing and retaining the largest connected component except the background;

[0027] S53. Perform skeleton extraction on the image data processed in step S52 and calculate the number of curve pixels. Use the cv2.ximgproc.thinning function in the Opencv library to thin the single Morinda officinalis into a single-pixel curve skeleton curve, and count the total number of pixels in the skeleton curve;

[0028] S54. Detect the two endpoints of the skeleton curve using the eight-neighborhood method;

[0029] S55. Determine the midpoint coordinate o2 using the single - end skeleton tracking method. Select one of the endpoints as the starting point and track along the skeleton curve. During the tracking process, continuously count the number of pixels that have been tracked. When the number of tracked pixels is equal to half of the total number of pixels, the tracking point at this time is the midpoint o2(x 02 ,y 02 );

[0030] S56. According to the Morinda officinalis pose detection model based on YOLO - v8, obtain the coordinates o1(x 01 ,y 01 ) of the upper - left vertex of the prediction box of the single Morinda officinalis determined in step S4. Add the coordinates o1(x 01 ,y 01 ) and the o2(x 02 ,y 02 ) obtained in step S55. The sum is the midpoint coordinate o(x 02 +x 01 ,y 02 +y 01 ) of Morinda officinalis in the camera coordinate system, that is, the grasping position coordinate o(x0,y0) of Morinda officinalis;

[0031] S57. Determine the clamping angle using the two - end skeleton tracking method. Start from the two endpoints and track along the skeleton curve towards the midpoint. During the tracking process, calculate the distance between the two tracking points until the distance between the two tracking points is equal to the gripper distance d1, and determine the coordinates a(x a ,y a ) and b(x b ,y b ) of the two tracking points in the prediction box image. Calculate the slope k based on the coordinates of point a and point b, and then determine the clamping angle θ from the slope k.

[0032] This application also provides a Morinda officinalis pose detection system based on machine vision, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the Morinda officinalis pose detection method based on machine vision as described in any one of the above.

[0033] This application also provides a computer - readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the Morinda officinalis pose detection method based on machine vision as described in any one of the above.

[0034] (III) Beneficial effects

[0035] A method, system and medium for detecting the pose of Morinda officinalis based on machine vision proposed in this application use a Morinda officinalis pose detection model based on YOLO-V8 to identify Morinda officinalis targets, and use an area threshold filtering method as an optimization method to further optimize the prediction effect of the Morinda officinalis pose detection model based on YOLO-V8. The Morinda officinalis pose detection model based on YOLO-V8 has a fast prediction speed, high accuracy, is lightweight, and is suitable for deployment on mobile devices or embedded devices. It can achieve real-time detection while accurately identifying, and can meet the real-time and low-cost requirements of the factory production line. No additional manual operation is required during the entire pose detection process, which greatly improves the detection efficiency.

[0036] A method, system and medium for detecting the pose of Morinda officinalis based on machine vision proposed in this application accurately identify the position and pose of Morinda officinalis based on traditional morphological operations, the eight-neighborhood method, skeleton tracking and other methods, which can effectively improve the success rate of gripper grasping during the feeding process and improve efficiency.

[0037] A method, system and medium for detecting the pose of Morinda officinalis based on machine vision in this application grabs single Morinda officinalis with high efficiency. And through the vibration processing platform, it can disperse the Morinda officinalis pile and grab single Morinda officinalis again, which can meet the requirement of grasping the Morinda officinalis pile scenario when grabbing single Morinda officinalis. Description of the Drawings

[0038] Figure 1 It is a flowchart showing a method for detecting the pose of Morinda officinalis based on machine vision provided by an embodiment of the present invention;

[0039] Figure 2 It is a flowchart showing the image preprocessing of step S4 of a method for detecting the pose of Morinda officinalis based on machine vision provided by an embodiment of the present invention;

[0040] Figure 3 It is a histogram showing the pixel number differences of three different stacked states of Morinda officinalis in a method for detecting the pose of Morinda officinalis based on machine vision provided by an embodiment of the present invention;

[0041] Figure 4 It is a flowchart showing the processing of step S5 of a method for detecting the pose of Morinda officinalis based on machine vision provided by an embodiment of the present invention

[0042] Figure 5 It is a schematic diagram of the hardware structure of a system for detecting the pose of Morinda officinalis based on machine vision provided by an embodiment of the present invention. Detailed Embodiments

[0043] To better explain the present invention for easy understanding, the present invention will be described in detail below with reference to the drawings through specific embodiments.

[0044] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a certain specific posture (as shown in the attached drawings). If this specific posture changes, the directional indication will also change accordingly.

[0045] In addition, in the present invention, descriptions such as "first" and "second" are only for descriptive purposes and should not be construed as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0046] In the present invention, unless otherwise clearly specified and limited, terms such as "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or integrated; "connection" can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and can be the communication inside two elements or the interaction relationship between two elements, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0047] As Figure 1 shown, in this embodiment, a method for detecting the pose of Morinda officinalis based on machine vision is provided. The method includes the steps:

[0048] S1. Collect data information of Morinda officinalis and perform annotation, and collect the RGB image (a color image format composed of the three primary colors of red, green, and blue) information of Morinda officinalis on the acquisition and processing platform; use an image annotation tool to annotate the collected RGB image information of Morinda officinalis to obtain an annotated Morinda officinalis image sample; among them, the annotation of the Morinda officinalis image sample includes two types of samples, namely a single Morinda officinalis image sample and a Morinda officinalis stack image sample; the single Morinda officinalis is a single and non-overlapping Morinda officinalis, and the Morinda officinalis stack is an overlapping Morinda officinalis; the image annotation tool can be set according to the actual situation, and can be, for example, the Labelimg tool, the Labelme tool, and the MakeSense tool. Specifically, in this embodiment, the image annotation tool uses the Labelimg tool, and its saving format is txt; in other embodiments, a single Morinda officinalis can be represented by Single_MOH, and a Morinda officinalis stack can be represented by MOH_Stack;

[0049] S2. Train a Morinda officinalis pose detection model based on YOLO - v8. Use the YOLO - v8 model, where the YOLO - v8 (You Only Look Once version 8) is one of the versions of the You Only Look Once (YOLO) object detection series of models, developed by Ultralytics. Based on the labeled Morinda officinalis image samples in step S1, perform model training. After model training, obtain a Morinda officinalis pose detection model based on YOLO - v8. The goal of model training is to enable the model to accurately identify single Morinda officinalis and Morinda officinalis piles in the image. During the training process, the performance of the model will be continuously evaluated to ensure that it has a good fitting effect. This usually involves using a validation set to calculate metrics such as the accuracy of the model.

[0050] S3. Use the Morinda officinalis pose detection model based on YOLO - v8 to predict the Morinda officinalis image data to be pose - detected on the processing platform, and obtain a preliminary prediction result. The prediction result includes multiple candidate boxes, and each candidate box contains a class label (single Morinda officinalis or Morinda officinalis pile) and a confidence score.

[0051] S4. Optimize the preliminary prediction result using the area threshold filtering method. Based on the determined actual area threshold S1, judge whether the preliminary prediction result is a single Morinda officinalis. By filtering out prediction boxes with unreasonable areas, the number of false detections can be effectively reduced, and the detection accuracy can be improved. Reducing the number of prediction boxes can reduce the computational complexity of subsequent processing, thereby improving the overall detection efficiency.

[0052] S5. For the prediction result determined to be a single Morinda officinalis in step S4, calculate the grasping coordinates o(x0, y0) and the clamping angle θ of this single Morinda officinalis in the camera coordinates.

[0053] S6. Convert the camera coordinates to world coordinates through camera calibration to guide the robotic arm to grasp a single Morinda officinalis.

[0054] Specifically, after step S6, it further includes:

[0055] S7. When all single Morinda officinalis have been grasped, the processing platform activates the vibration mechanism to disperse the Morinda officinalis pile, and then repeat steps S3 to S6 until all Morinda officinalis have been grasped. The "all Morinda officinalis" refers to all Morinda officinalis to be processed. In other embodiments, the "all Morinda officinalis" refers to all Morinda officinalis to be processed within a batch on the production line.

[0056] Optionally, the Morinda officinalis RGB image information on the acquisition and processing platform in step S1 includes: setting the camera position directly above the processing platform and acquiring the RGB image of Morinda officinalis placed on the processing platform at an angle perpendicular to the processing platform. In other embodiments, a hyperspectral imaging device is used to acquire the Morinda officinalis RGB image information on the processing platform from multiple angles, and an annular light source is used during the shooting process.

[0057] Preferably, in this embodiment, the Morinda officinalis data in step S2 based on the Morinda officinalis data in step S1 includes: randomly dividing the Morinda officinalis image information in step S1 into a training set, a test set, and a prediction set to form a data set for model training; in other embodiments, the specific training set is the core data set for Yolo-v8 learning, which is used to train the parameters of the model, including weights and biases, etc., and the quantity needs to account for 70% - 80% of the data set; the validation set is used to evaluate the performance of the model during the training process, help adjust hyperparameters, including the learning rate, etc., to prevent overfitting, and the quantity needs to account for 10% - 15% of the data set; the test set is used to finally evaluate the performance of the model, reflect the performance of the model in the real scenario, and ensure the objectivity and reliability of the evaluation results, and the quantity needs to account for 10% - 15% of the data set.

[0058] As a preferred embodiment of the present invention, the actual area threshold S1 determined in step S4 is obtained through the following steps:

[0059] S41, calculate the pixel equivalent A on the processing platform using the checkerboard calibration method; since the area threshold method is used for optimization and the area estimation is determined by the number of pixel values, this optimization method is sensitive to pixel values, and it is necessary to correct the images collected by the camera to reduce the error caused by lens distortion in the calculation.

[0060] S42, calculate the actual area corresponding to each pixel unit. Based on the pixel equivalent A obtained in step S41, use the formula S = A 2 p to calculate the actual area corresponding to each pixel unit, where p represents the number of pixels;

[0061] S43, as Figure 2 shown, perform a series of image preprocessing and connected component analysis on the classified image in step S43. Based on the preset pixel threshold p1, through the formula S1 = A 2p1 obtains the actual area threshold S1. Among them, the image preprocessing and connected component analysis include: performing grayscale processing to obtain a grayscale image, converting the color image into a grayscale image to simplify subsequent processing; then performing median filtering to remove noise in the image and make the image smoother; the processing platform in this embodiment contains obvious foreground and background, an approximate bimodal histogram image, so Otsu binarization is used to convert the grayscale image into a binary image; that is, there are only two values (0 and 255) for the pixels in the image, which is convenient for subsequent shape analysis; erosion processing and dilation processing are morphological operations used to remove small noise points or fill small holes, and to connect adjacent object parts. Traverse and retain the largest connected component except the background. In the binary image, find and retain the largest connected region, which represents a single Morinda officinalis; calculate the sum of the pixels within the largest connected component.

[0062] Preferably, the calculation method of the preset pixel threshold p1 in this embodiment includes: by statistically analyzing the differences in the corresponding pixel numbers in three states of a single Morinda officinalis, two stacked Morinda officinalis, and multiple stacked Morinda officinalis, to determine the preset pixel threshold p1. Since the situation of two stacked Morinda officinalis has a greater interference on recognition, the pixel values of two stacked Morinda officinalis also need to be statistically analyzed separately when determining the threshold. Classify the statistical analysis data according to three situations: a single Morinda officinalis image, two stacked Morinda officinalis images, and stacked Morinda officinalis, and the corresponding labels are Single_MOH, Double_MOH, and MOH_Stack respectively. In other embodiments, as Figure 3 shown, select 500 single Morinda officinalis images, 200 two-stacked Morinda officinalis images, and 200 stacked Morinda officinalis images for experiments, statistically analyze their pixel numbers, and draw a histogram. It can be clearly seen from the histogram the differences in the pixel numbers of the three different stacked states of Morinda officinalis, and determine the pixel number threshold p1 based on the drawn histogram. Or, based on experience and knowledge in the field of Morinda officinalis, set the pixel number threshold p1. For example, refer to relevant research to obtain the actual maximum size of a single Morinda officinalis, and convert the actual size into the pixel number to set the pixel threshold p1.

[0063] As a preferred embodiment of the present invention, the step S4 of judging whether it is a single Morinda officinalis based on the determined actual area threshold S1 includes: using the actual area threshold S1 as a judgment criterion to judge the actual area of the single Morinda officinalis image predicted by the Morinda officinalis pose detection model based on yolo-v8. If the actual area of the predicted single Morinda officinalis image is less than or equal to the actual area threshold S1, it is judged as a single Morinda officinalis, meeting the requirements of pose recognition and grasping; if the actual area of the predicted single Morinda officinalis image is greater than the actual area threshold S1, it is not a single Morinda officinalis, and does not meet the requirements of pose recognition and grasping.

[0064] AsFigure 4 As shown, further, in this embodiment, the step S5 includes:

[0065] S51, extracting the image information in the prediction box determined as a single Morinda officinalis in the step S4;

[0066] S52, performing image preprocessing on the image information in the step S51. The image preprocessing includes: performing grayscale processing to obtain a grayscale image, then performing median filtering to smooth the image while retaining edge information, performing Otsu binarization to convert the grayscale image into a binary image, performing erosion processing, dilation processing. Morphological operations such as erosion and dilation on the binary image can remove noise and fill holes, and traversing and retaining the largest connected component except the background;

[0067] S53, performing skeleton extraction on the image data processed in the step S52 and calculating the number of curve pixels. Using the cv2.ximgproc.thinning function in the Opencv library (Open Source Computer Vision Library) to thin a single Morinda officinalis into a skeleton curve of a single-pixel curve, and counting the total number of pixels in the skeleton curve; the curve refined in this embodiment is a white pixel and the background is a black pixel.

[0068] S54, detecting the two endpoints of the skeleton curve using the eight-neighborhood method; traversing the pixel points in the image. If the current pixel is white, detecting the eight neighborhoods of this pixel. If there is exactly one white pixel in the eight neighborhoods, then this point is determined as an endpoint.

[0069] S55, determining the midpoint coordinate o2 using the single-end skeleton tracking method. Selecting one of the endpoints as the starting point and tracking along the skeleton curve. During the tracking process, continuously counting the number of pixels that have been tracked. When the number of tracked pixels is equal to half of the total number of pixels, the current tracking point is the midpoint o2(x 02 ,y 02 ) of the skeleton curve;

[0070] S56, obtaining the coordinates o1(x 01 ,y 01 ) of the upper left vertex of the prediction box determined as a single Morinda officinalis in the step S4 according to the Morinda officinalis pose detection model based on yolo-v8. Adding the coordinates o1(x 01 ,y 01 ) and the o2(x 02 ,y 02 ) obtained in the step S55, to obtain the midpoint coordinate o(x 02 +x 01 ,y02 +y 01 ), that is, the coordinates o(x0, y0) of the Morinda officinalis grabbing position.

[0071] S57. The two-end skeleton tracking method is adopted to determine the clamping angle. Starting from the two endpoints and tracking along the skeleton curve towards the midpoint, during the tracking process, calculate the distance between the two tracking points until the distance between the two tracking points is equal to the jaw distance d1, and determine the coordinates a(x a , y a ) and b(x b , y b ) of the two tracking points in the predicted box image. Based on the coordinates of point a and point b, calculate the slope k, and then determine the clamping angle θ from the slope k to complete the pose recognition.

[0072] The present invention also provides a Morinda officinalis pose detection system based on machine vision, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the Morinda officinalis pose detection method based on machine vision as described in any one of the above are implemented.

[0073] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the Morinda officinalis pose detection method based on machine vision as described in any one of the above are implemented.

[0074] Figure 5 is a schematic hardware structure diagram for running the Morinda officinalis pose detection method based on machine vision provided by an embodiment of the present invention. As Figure 5 shown, this embodiment / computer 6 includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a program for running the Morinda officinalis pose detection method based on machine vision. When the processor 60 executes the computer program 62, the steps in each of the above embodiments of the Morinda officinalis pose detection method based on machine vision are implemented. Or, when the processor 60 executes the computer program 62, the functions of each module / unit in each of the above device embodiments are implemented.

[0075] Exemplarily, the computer program 62 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 61 and executed by the processor 60 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 62 in the computer 6.

[0076] The computer 6 may be a computing device such as a desktop computer, a notebook, a handheld computer, and a cloud server. The computer 6 device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art can understand that Figure 5 merely examples of the computer 6, which do not constitute a limitation on the computer 6, may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the computer 6 may further include input / output devices, network access devices, a bus, etc.

[0077] The so-called processor 60 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0078] The memory 61 may be an internal storage unit of the computer 6, such as the hard disk or memory of the computer 6. The memory 61 may also be an external storage device of the computer 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the terminal device. Further, the memory 61 may also include both the internal storage unit and the external storage device of the computer 6. The memory 61 is used to store the computer program and other programs and data required by the terminal device. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0079] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be assigned to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here.

[0080] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0081] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0082] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, in each embodiment of the present invention, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0085] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0086] The above are only specific application examples of the present invention and do not constitute any limitation to the protection scope of the present invention. In addition to the above embodiments, the present invention can also have other implementation manners. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.

Claims

1. A method for detecting the pose of Morinda officinalis How based on machine vision, characterized in that, Including the steps: S1. Collect Morinda officinalis data information and perform annotation: Collect the RGB image information of Morinda officinalis on the acquisition and processing platform; Use an image annotation tool to annotate the collected RGB image information of Morinda officinalis to obtain the annotated Morinda officinalis image samples; Among them, the annotation of the Morinda officinalis image samples includes two types of samples, namely single Morinda officinalis image samples and Morinda officinalis stack image samples; S2. Train a Morinda officinalis pose detection model based on yolo-v8: Use the yolo-v8 model to perform model training based on the annotated Morinda officinalis image samples in step S1, and obtain a Morinda officinalis pose detection model based on yolo-v8 after model training; S3. Use the Morinda officinalis pose detection model based on yolo-v8 to predict the Morinda officinalis image data to be pose-detected on the processing platform to obtain a preliminary prediction result; S4. Optimize the preliminary prediction result using the area threshold filtering method, and judge whether the preliminary prediction result is a single Morinda officinalis based on the determined actual area threshold S1; S5. For the prediction result determined as a single Morinda officinalis in step S4, calculate the grasping coordinates o(x0, y0) and the clamping angle θ of the single Morinda officinalis in the camera coordinate system; S6. Convert the camera coordinate to the world coordinate through camera calibration to guide the robotic arm to grasp a single Morinda officinalis.

2. The method for detecting the pose of Morinda officinalis How based on machine vision according to claim 1, wherein After step S6, it further includes: S7. When all single Morinda officinalis have been grasped, the processing platform starts the vibration mechanism to disperse the Morinda officinalis stack, and then repeat steps S3 to S6 until all Morinda officinalis have been grasped.

3. A method for detecting the pose of Morinda officinalis based on machine vision according to claim 1, characterized in that, The RGB image information of Morinda officinalis on the acquisition and processing platform in step S1 includes: Set the camera position directly above the processing platform and collect the RGB picture of Morinda officinalis placed on the processing platform at an angle perpendicular to the processing platform.

4. A method for detecting the pose of Morinda officinalis based on machine vision according to claim 1, characterized in that, The annotated Morinda officinalis image samples based on step S1 in step S2 include: Randomly divide the annotated Morinda officinalis image samples in step S1 into a training set, a test set, and a prediction set to form a data set for model training.

5. A method for detecting the pose of Morinda officinalis based on machine vision according to claim 1, characterized in that The determined actual area threshold S1 in step S4 is obtained through the following steps: S41. Calculate the pixel equivalent A on the processing platform using the checkerboard calibration method; S42. Calculate the actual area corresponding to each pixel unit: Based on the pixel equivalent A obtained in the step S41, use the formula S = A 2 p to calculate the actual area corresponding to each pixel unit, where p represents the number of pixels; S43, Image preprocessing and connected component analysis. Perform a series of image preprocessing and connected component analysis on the image classified in step S43. Based on the preset pixel threshold p1, obtain the actual area threshold S1 through the formula S1 = A 2 p1, where the image preprocessing and connected component analysis include: performing grayscale processing to obtain a grayscale image, then performing median filtering, Otsu binarization, erosion, dilation, traversing and retaining the largest connected component except the background, and calculating the total number of pixels within the largest connected component.

6. The pose detection method of Morinda officinalis How based on machine vision according to claim 5, characterized in that The calculation method of the preset pixel threshold p1 includes: Determine the preset pixel threshold p1 by statistically analyzing the differences in the corresponding pixel quantities in three states of a preset number of single Morinda officinalis, two stacked Morinda officinalis, and multiple stacked Morinda officinalis, or set the pixel threshold p1 based on experience and Morinda officinalis field knowledge.

7. A method for detecting the pose of Morinda officinalis based on machine vision according to claim 5, characterized in that, The step S4 determines whether it is a single Morinda officinalis based on the determined actual area threshold S1, which includes: judging the actual area of the single Morinda officinalis image obtained from the preliminary prediction result by using the actual area threshold S1 as the judgment criterion. If the actual area of the predicted single Morinda officinalis image is less than or equal to the actual area threshold S1, it is judged as a single Morinda officinalis; if the actual area of the predicted single Morinda officinalis image is greater than the actual area threshold S1, it is not a single Morinda officinalis.

8. A method for detecting the pose of Morinda officinalis based on machine vision according to claim 1, characterized in that, The step S5 includes: S51, extracting the image information in the prediction box determined as a single Morinda officinalis in the step S4; S52, performing image preprocessing on the image information in the step S51. The image preprocessing includes: performing grayscale processing to obtain a grayscale image, and then performing median filtering, Otsu binarization, erosion processing, dilation processing, and traversing and retaining the largest connected domain except the background; S53, performing skeleton extraction on the image data processed in the step S52 and calculating the number of curve pixels. The cv2.ximgproc.thinning function in the Opencv library is used to thin the single Morinda officinalis into a skeleton curve of a single-pixel curve, and the total number of pixels in the skeleton curve is counted; S54, detecting the two endpoints of the skeleton curve by using the eight-neighborhood method; S55. Determine the midpoint coordinate o2 using the single - end skeleton tracking method. Select one of the endpoints as the starting point and track along the skeleton curve. During the tracking process, continuously count the number of pixels that have been tracked. When the number of tracked pixels is equal to half of the total number of pixels, the tracking point at this time is the midpoint o2(x 02 , y 02 ); S56. Obtain the coordinates o1(x 01 ,y 01 ) of the upper left vertex of the prediction box of the single Morinda officinalis determined in the step S4 according to the Morinda officinalis pose detection model based on YOLO - v8. Add the coordinates o1(x 01 ,y 01 ) and o2(x 02 ,y 02 ) obtained in the step S55 to get the mid - point coordinates o(x 02 +x 01 ,y 02 +y 01 ) of the Morinda officinalis in the camera coordinates, that is, the grasping position coordinates o(x0,y0) of the Morinda officinalis; S57. The clamping angle is determined by the two - end skeleton tracking method. Starting from the two endpoints, track along the skeleton curve towards the mid - point. During the tracking process, calculate the distance between the two tracked points until the distance between the two tracked points is equal to the jaw distance d1, and determine the coordinates a(x a ,y a ) and b(x b ,y b ) of the two tracked points in the predicted box image. Calculate the slope k based on the coordinates of point a and point b, and then determine the clamping angle θ from the slope k.

9. A Morinda officinalis How pose detection system based on machine vision, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the machine vision-based Morinda officinalis pose detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the machine vision-based Morinda officinalis pose detection method according to any one of claims 1 to 8.

Images