Visual detection method, device, equipment and medium for eye feature points of pineapple

By using feature point detection and a dual-reference center column segmented interpolation algorithm for pineapple image sequences, high-precision detection and depth calculation of pineapple eye feature points are achieved, solving the problems of high cost and poor stability in existing technologies, and making it suitable for automated pineapple processing.

CN123118748APending Publication Date: 2026-11-17HEPU GUOXIANGYUAN FOOD CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611429684.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-15
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In the automated processing of pineapples, existing technologies struggle to meet the requirements of low cost, high speed, low latency, and high robustness in terms of high-precision fruit eye feature point detection and depth calculation. Furthermore, existing 2D depth estimation methods are susceptible to ambient light and exhibit poor stability.

Method used

By acquiring pineapple image sequences, the image pixel coordinates of the fruit eye feature points are extracted. The circumferential angle and height are calculated by combining the sampling reference angle. The process depth is calculated by using a dual reference center column segmented interpolation and cross-angle scale dynamic correction algorithm, thus achieving high-precision detection and depth measurement of the fruit eye feature points.

Benefits of technology

It achieves high-precision, low-latency detection of fruit eye feature points and depth calculation using only a low-cost 2D camera, solving the errors caused by mechanical eccentricity interference and the non-standard cylindrical characteristics of pineapples, and is suitable for automated pineapple processing scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118748A_ABST
    Figure CN123118748A_ABST
Patent Text Reader

Abstract

This application relates to the field of image processing technology, and provides a visual detection method for pineapple eye feature points. The method includes: acquiring an image sequence of pineapple images acquired circumferentially; detecting eye feature points in the pineapple images and extracting the image pixel coordinates of the eye feature points; calculating the circumferential angle and height of the eye feature points based on the coordinates and the sampling reference angle corresponding to the pineapple image; determining the depth measurement angle corresponding to the eye feature point and the matching target pineapple image based on the circumferential angle; further determining the intersection point of the pixel row where the eye feature point is located and the outer contour of the pineapple body; calculating the process depth of the eye feature point based on the lateral pixel distance between the intersection point and the target reference column; and generating and outputting the circumferential angle, height, and process depth. This application relies on a 2D camera to convert visual detection results into control quantities for industrial control with high precision and low latency, eliminating the need for a 3D camera and exhibiting low computational complexity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of image processing technology. Specifically, it relates to a visual detection method, apparatus, device, and medium for feature points of a pineapple fruit eye. Background Technology

[0002] In automated pineapple processing, the actuator typically needs to perform positioning, alignment, gripping, insertion, or other subsequent processing actions based on the spatial distribution information of multiple eye feature points on the pineapple surface. To ensure stable operation of the actuator, the vision system not only needs to detect the position of the eye feature points but also needs to provide angle, height, and depth information directly corresponding to process control.

[0003] To obtain high-precision depth information for fruit eyes, the industry typically prefers to use industrial 3D cameras, such as binocular vision cameras and laser structured light cameras. However, their procurement and maintenance costs are often several times, or even tens of times, higher than those of ordinary 2D monocular cameras. Existing depth extraction algorithms based on ordinary 2D cameras, such as those based on multi-frame motion reconstruction structures and monocular defocusing, often have extremely high computational complexity and are greatly affected by ambient lighting, making it difficult to meet the stringent requirements of high-speed agricultural product processing lines for high cycle times, low latency, and high robustness.

[0004] Meanwhile, the surface texture of pineapples is complex and noisy, and the grayscale and texture differences between the recessed area of ​​the fruit eye and the surrounding peel are significantly affected by the angle of illumination. Existing single-frame 2D depth estimation methods mostly rely on local grayscale gradients or template matching to infer depth, which are easily affected by boundary discontinuities, changes in fruit eye shape, and image phase differences. The output depth data fluctuates wildly and has poor stability, making it difficult to use directly for process control.

[0005] Therefore, there is an urgent need for a low-cost vision processing method for automated pineapple processing lines that can achieve a complete technical solution that simultaneously realizes high-precision detection of fruit eye feature points, circumferential angle mapping, height positioning, and stable depth calculation using only a 2D monocular camera. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a visual detection method, apparatus, device, and medium for pineapple eye feature points.

[0007] In a first aspect, this application provides a visual detection method for eye feature points of a pineapple, the method comprising: A sequence of images of a pineapple is obtained by circumferentially acquiring the images, the image sequence including multiple frames of pineapple images acquired at multiple sampling reference angles; For any frame of pineapple image in the image sequence, perform fruit eye feature point detection on the pineapple image and extract the image pixel coordinates of each fruit eye feature point in the image coordinate system; Based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image, calculate the circumferential angle and height of each fruit eye feature point in the pineapple image; For any of the aforementioned fruit eye feature points, the depth measurement angle corresponding to the fruit eye feature point is determined based on the circumferential angle of the fruit eye feature point, and the target pineapple image in the image sequence that matches the sampling reference angle and the depth measurement angle is determined. In the target pineapple image, the intersection point of the pixel row containing the eye feature point and the outer contour of the pineapple body is determined. Based on the horizontal pixel distance between the intersection point and the target reference column, the processing depth of the eye feature point is calculated. The target reference column is the reference reference column corresponding to the target pineapple image. Generate and output a set of fruit eye parameters, which includes the circumferential angle, height and process depth of each fruit eye feature point.

[0008] In some optional implementations, the image pixel coordinates of the fruit eye feature point include horizontal pixel coordinates and vertical pixel coordinates; The step of calculating the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image includes: For any of the fruit eye feature points in the pineapple image, based on the horizontal pixel coordinates of the fruit eye feature point and the horizontal pixel offset between the reference column in the pineapple image, the local angle increment of the fruit eye feature point is determined by converting it according to a preset first ratio coefficient from pixel to angle. The local angle increment, the sampling reference angle, and the preset global angle offset are added together, and the circumferential angle of the fruit eye feature point is obtained after angle normalization. Based on the vertical pixel offset between the vertical pixel coordinates of the fruit eye feature point and the height zero reference row, the height of the fruit eye feature point is determined by conversion according to a preset second ratio coefficient from pixel to height.

[0009] In some alternative implementations, the reference column is determined in the following manner: For any current pineapple image in the image sequence, determine the current sampling reference angle corresponding to the current pineapple image; If the current sampling reference angle is within the first angle range from the first sampling reference angle to the second sampling reference angle, then with the current sampling reference angle as the independent variable, linear interpolation is performed between the first reference reference column corresponding to the first sampling reference angle and the second reference reference column corresponding to the second sampling reference angle to calculate the reference reference column corresponding to the current pineapple image. If the current sampling reference angle is within the range of the second angle corresponding to the second sampling reference angle to the first sampling reference angle, then with the current sampling reference angle as the independent variable, linear interpolation is performed between the second reference column corresponding to the second sampling reference angle and the first reference column corresponding to the first sampling reference angle to calculate the reference reference column corresponding to the current pineapple image.

[0010] In some alternative implementations, the first pixel-to-angle scaling factor is determined in the following manner: In the image sequence, the same target fruit eye feature point is matched in two adjacent pineapple images, and the horizontal pixel displacement of the target fruit eye feature point in the two adjacent pineapple images is determined. The first proportional coefficient from pixel to angle is determined based on the ratio of the difference in the sampling reference angle between two adjacent pineapple images to the horizontal pixel displacement.

[0011] In some optional implementations, the depth measurement angle corresponding to the fruit eye feature point differs from the circumferential angle of the fruit eye feature point by 90°; The step of calculating the process depth of the fruit eye feature point based on the lateral pixel distance between the intersection point and the target reference column includes: The difference between the horizontal pixel coordinates of the target reference column and the intersection point is calculated to obtain the horizontal pixel distance; The horizontal pixel distance is converted according to a preset third proportional coefficient from pixel to process depth to obtain the original depth; The original depth is subjected to constraint processing to obtain the process depth.

[0012] In some optional implementations, the method further includes: The target pineapple image is segmented to obtain a mask image corresponding to the target pineapple image; connected component identification is performed on the mask image, and the boundary of the largest connected component is used as the outer contour of the pineapple body; And / or, Based on the pineapple image corresponding to the depth measurement angle of each of the fruit eye feature points, the fruit eye feature points are grouped; the depth measurement angle of each fruit eye feature point in each group corresponds to the same pineapple image; for the target pineapple image, the processing depth of each fruit eye feature point corresponding to the target pineapple image is calculated uniformly.

[0013] In some alternative implementations, prior to the output eye parameter set, the following is also included: The detected fruit eye feature points are deduplicated: if the height difference between two fruit eye feature points is less than a preset height threshold and the angle difference is less than a preset angle threshold, then the two fruit eye feature points are determined to be the same fruit eye, and the fruit eye feature point with the highest confidence is retained.

[0014] Secondly, this application provides a visual detection device for eye feature points of a pineapple fruit, the device comprising: The image acquisition module is used to acquire an image sequence obtained by circumferentially acquiring images of a pineapple. The image sequence includes multiple frames of pineapple images acquired at multiple sampling reference angles. The feature point detection module is used to detect the fruit eye feature points of any pineapple image in the image sequence and extract the image pixel coordinates of each fruit eye feature point in the image coordinate system. The circumferential angle and height calculation module is used to calculate the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image; The depth image matching module is used to determine the depth measurement angle corresponding to any of the fruit eye feature points based on the circumferential angle of the fruit eye feature points, and to determine the target pineapple image in the image sequence that matches the sampling reference angle with the depth measurement angle. The process depth calculation module is used to determine the intersection point of the pixel row containing the fruit eye feature point and the outer contour of the pineapple body in the target pineapple image, and calculate the process depth of the fruit eye feature point based on the horizontal pixel distance between the intersection point and the target reference column; the target reference column is the reference reference column corresponding to the target pineapple image; The output module is used to generate and output a set of fruit eye parameters, which includes the circumferential angle, height and process depth of each of the fruit eye feature points.

[0015] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the visual detection method for pineapple eye feature points described in the first aspect or any corresponding embodiment.

[0016] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the visual detection method for pineapple eye feature points according to the first aspect or any corresponding embodiment.

[0017] In the solution provided by the first aspect of this invention, the lateral pixel distance from the eye of the pineapple to its body contour is extracted and then converted according to a preset pixel-to-process depth coefficient to obtain the process depth to which the actuator can cut. Simultaneously, by introducing a dual-reference center column segmented interpolation and cross-angle scale dynamic correction algorithm, the mechanical eccentricity interference and errors caused by the non-standard cylindrical characteristics of the pineapple during rotary processing are resolved. This solution enables the visual inspection results to be uniformly converted into control quantities that can be directly used for industrial control with high precision and low latency, relying solely on a low-cost 2D camera. This overcomes the problems of high cost of 3D cameras and high computational complexity and poor stability of 2D depth estimation algorithms in existing technologies. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A schematic diagram of an architecture for a pineapple eye processing system is shown. Figure 2 A flowchart of a visual detection method for pineapple eye feature points according to an embodiment of the present invention is shown; Figure 3 This is a schematic diagram of a pineapple image; Figure 4 A schematic diagram showing a circumferential top-down view of a pineapple is provided; Figure 5 This diagram illustrates a principle for calculating process depth. Figure 6 This diagram illustrates two adjacent frames of pineapple images. Figure 7 This diagram illustrates one method for determining the depth of the fruit eye process. Figure 8 An architecture diagram of the visual detection method for pineapple eye feature points according to an embodiment of the present invention is shown; Figure 9 A block diagram of a visual detection device for pineapple eye feature points according to an embodiment of the present invention is shown; Figure 10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0022] Before providing a detailed description of the embodiments of this application, some of the terms and concepts involved in the embodiments of this application will be explained. These explanations are intended to make the embodiments of this application easier to understand and should not be considered as limiting the scope of protection claimed in this application.

[0023] (1) Fruit eye feature points refer to local target points on the surface of pineapple that can be identified by visual algorithms and can be used as the basis for subsequent process actions; these target points are batch detection objects, rather than being limited to individual fixed points.

[0024] (2) Sampling reference angle refers to the rotation angle of the pineapple at the detection station relative to the circumferential reference zero position of the equipment, driven by the rotating mechanism under the condition of fixed camera arrangement; this angle can be obtained from the encoder, servo drive feedback value of the rotating motor or the PLC recorded value.

[0025] (3) Image pixel coordinates refer to the coordinates of the center of the fruit eye feature point in the image coordinate system, including the horizontal pixel coordinate x and the vertical pixel coordinate y.

[0026] (4) Circumferential angle refers to the circumferential position of the fruit eye feature point relative to the circumferential reference system of the equipment.

[0027] (5) Height refers to the vertical position of the fruit eye feature point relative to the zero height point (e.g., a preset horizontal reference plane).

[0028] (6) Process depth refers to the process depth control value obtained by converting the monocular contour geometric agent quantity, which is used to control the insertion depth of the actuator.

[0029] (7) Reference column refers to the reference column in the image used for angle mapping and depth conversion, which can be determined by piecewise linear interpolation.

[0030] (8) Depth measurement angle refers to the target angle used for depth measurement, which is 90° different from the circumferential angle of the fruit eye feature point.

[0031] (9) The angle normalization function is defined as follows: The normalized result is located in [0, 360). Using the angle normalization function, the axial angle can be normalized to [0, 360).

[0032] (10) Angle difference function, defined as It is used to measure the minimum difference between two angles on a circle.

[0033] Figure 1 A schematic diagram of an architecture for a pineapple fruit eye processing system is shown, illustrating its main viewpoint. Figure 1 As shown, the system includes a camera 101, a support platform 102, and an actuator 103. The support platform 102 is located at the inspection station, and a rotating shaft 112 is provided in the support platform 102. The pineapple 100 to be processed is placed on the support platform 102 and passes through the rotating shaft 112. The rotating shaft 112 can drive the pineapple 100 to rotate, or the support platform 102 can drive the pineapple 100 to rotate along the rotating shaft 112.

[0034] Camera 101 is positioned on one side of support platform 102 and faces support platform 102 to acquire relevant images of pineapple 100; Figure 1 The pineapple 100 is represented by a dashed line. The position of the camera 101 is generally fixed. The actuator 103 is used to locate the eye of the pineapple 100 and then perform corresponding processing on the eye (such as clamping, piercing, etc.).

[0035] For any eye P on pineapple 100, its position in three-dimensional space needs to be determined. In this embodiment, the circumferential angle θ of eye P is used as the reference point. Figure 1 (not shown in the figure), height h, and process depth L represent the three-dimensional spatial position of the eye P.

[0036] Among them, the circumferential angle θ is the relative rotation angle of the fruit eye P around the rotation axis 112, and the value ranges from 0 to 360°.

[0037] The height h is the distance between the eye P and the preset horizontal reference plane. Figure 1 In this embodiment, the horizontal plane corresponding to the position of the execution end of the actuator 103 at the initial stage is used as the horizontal reference plane; other horizontal reference planes may also be used, and this embodiment does not limit them.

[0038] The process depth L is the distance between the fruit eye P and the rotating shaft 112.

[0039] The actuator 103 can be a high-degree-of-freedom robotic arm. Alternatively, the actuator 103 can only move within a certain plane. For example, the actuator 103 and the camera 101 are in the same vertical plane, and the actuator 103 can only move up and down, with its end effector ( Figure 1 The protruding part on the lower right side of the actuator 103 is the actuator end (which can extend and retract forward and backward). Figure 1 (It can extend and retract left and right).

[0040] After determining the circumferential angle θ, height h, and processing depth L of the fruit eye P, the support platform 102 can be rotated to position the fruit eye P in the vertical plane where the actuator 103 is located. Furthermore, the actuator 103 is controlled to move up and down according to the height h, and its end effector is controlled to extend and retract forward and backward according to the processing depth L, so that the end effector aligns with the position of the fruit eye P, thereby enabling the clamping, insertion, or other subsequent processing actions to be performed on the fruit eye P.

[0041] To process each eye in a pineapple, it is necessary to identify the circumferential angle θ, height h, and processing depth L corresponding to each eye feature point using camera 101. Traditional solutions require a 3D camera or complex image processing algorithms, which are costly in terms of hardware and computational complexity.

[0042] This application provides a visual detection method for pineapple eye feature points, which does not rely on expensive 3D imaging hardware and can achieve high-precision depth measurement of pineapple eyes based on 2D monocular vision. Relying only on a low-cost 2D camera, batch detection of pineapple eye feature points on the surface can be performed in industrial production lines, and the visual detection results can be uniformly converted into control quantities that can be directly used for industrial control with high precision and low latency.

[0043] This invention provides a visual detection method for eye feature points in pineapples, see [link to relevant documentation]. Figure 2 As shown, the method includes: Step S201: Obtain the image sequence obtained by circumferentially acquiring images of the pineapple. The image sequence includes multiple frames of pineapple images acquired at multiple sampling reference angles.

[0044] Images of the same pineapple were acquired at several circumferential angles to form an image sequence. Each frame of the image is associated with a sampling reference angle. Where k is the image number, This represents the sampling reference angle corresponding to the k-th frame image.

[0045] Specifically, the image acquisition method involves a fixed camera and a pineapple rotating at fixed angles, triggering a photo at regular intervals. Acquisition modes include single-frame mode and multi-angle sequence mode: in single-frame mode, a single image is input along with its sampling reference angle. In multi-angle sequence mode, the sequence is formed by circumferential sequential acquisition. Each pineapple image is associated with a sampling reference angle. .

[0046] Pineapple images in the acquired sequence can be preprocessed as follows: exposure, gain, and white balance are fixed; cropped to the region of interest containing the pineapple body; and size normalization is performed if necessary.

[0047] In one implementation, detection and depth measurement can be performed based on the same batch of original image files; wherein, the detection window and the stitching window are composed of... , , , Parameters are limited, among which, Indicates the baseline reference column, The initial pixel-to-angle scaling factor (i.e., the first scaling factor) is used for single-frame angle mapping and depth conversion. minY and maxY are the vertical boundaries of the region of interest.

[0048] By using a fixed-angle step-rotation acquisition method, the uniformity of the angular distribution of the image sequence is ensured, providing a reliable benchmark for subsequent angle mapping and phase matching. By preprocessing to fix parameters such as exposure, gain, and white balance, the interference of illumination changes on the detection results is reduced, and the detection stability is improved.

[0049] Step S202: For any frame of pineapple image in the image sequence, perform fruit eye feature point detection on the pineapple image and extract the image pixel coordinates of each fruit eye feature point in the image coordinate system.

[0050] For a single image The image width is The height is The image coordinate system is used, with the horizontal pixel coordinates as follows: Rightward is positive, and the vertical pixel coordinates are: Downwards is positive; for the i-th fruit eye feature point, by detecting it, the center of the fruit eye feature point can be output. That is, the image pixel coordinates of the i-th fruit eye feature point in the image coordinate system are .

[0051] Specifically, the detection of fruit eye feature points can employ traditional thresholding methods, target detection methods, or other methods capable of outputting the pixel coordinates of the target image. Regardless of the detector used, the image pixel coordinates are uniformly used in the subsequent circumferential angle and height mapping process.

[0052] Figure 3 This is a schematic diagram of a pineapple image. (For example...) Figure 3As shown, the pineapple image corresponds to an image coordinate system, with the horizontal axis X pointing to the right as positive and the vertical axis Y pointing downwards as positive; the sampling reference angle of the pineapple image is 15°. , , The physical meaning of the parameters can be found in [reference needed]. Figure 3 As shown.

[0053] By employing mature detection algorithms, the eye feature points on the pineapple surface can be accurately and quickly located, providing reliable input data for subsequent angle, height, and depth calculations. By using the center coordinates for subsequent processing, the data processing flow is simplified, and the consistency of the system is improved.

[0054] Step S203: Calculate the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image.

[0055] For the k-th pineapple image The i-th eye feature point in the image is determined based on the image pixel coordinates. And pineapple image Corresponding sampling reference angle The two-dimensional center coordinates of the fruit eye feature points can be obtained. Converted into circumferential angles and heights, this enables the mapping from the pixel coordinate system to process parameters (including circumferential angles and heights).

[0056] Step S204: For any of the fruit eye feature points, determine the depth measurement angle corresponding to the fruit eye feature point based on the circumferential angle of the fruit eye feature point, and determine the target pineapple image in the image sequence that matches the sampling reference angle with the depth measurement angle.

[0057] Depth measurement is not performed directly on the frame where the visual feature point is identified, but rather at a certain angle around its circumferential angle, for example: For the i-th fruit eye feature point, let its corresponding depth measurement angle be:

[0058] in, This represents the depth measurement angle of the i-th fruit eye feature point. This represents the circumferential angle of the i-th fruit eye feature point. That is, the depth measurement angle corresponding to the fruit eye feature point differs from the circumferential angle of that fruit eye feature point by 90°.

[0059] Let the set of sampling reference angles for the image sequence be... The matching image index is:

[0060] in, It is the sampling reference angle corresponding to the j-th image. Indicates the angle closest to the target. The image number of that pineapple. Its function is to be included in all sampling reference angle sets. Find the angle closest to the target. That pineapple image.

[0061] Furthermore, the angle difference between the sampling reference angle and the target angle of this image is less than half of the sampling step angle, as follows:

[0062] in, The step size of the sampling reference angle represents the number of rotations of the pineapple. Capture a single frame (e.g., with a stride of 10 ... , indicating each rotation Take a picture.

[0063] Based on this, it can be determined that the target pineapple image is the first image in the image sequence. A pineapple image. Subsequently, the processing depth of the i-th eye feature point is calculated on this target pineapple image first.

[0064] Figure 4 A schematic diagram showing a circumferential top-down view of a pineapple is provided, as follows: Figure 4 As shown, for the second pineapple image (whose sampling reference angle is...) The i-th eye feature point in the diagram has a corresponding depth angle of . that angle With the Sampling reference angle of a pineapple image The closest, therefore the first One pineapple image is used as the target pineapple image.

[0065] Based on this, the present invention does not directly obtain the true spatial depth in the sense of three-dimensional reconstruction, but rather obtains the process depth by changing the perspective and observing the distance between objects.

[0066] Step S205: In the target pineapple image, determine the intersection point of the pixel row containing the eye feature point and the outer contour of the pineapple body. Calculate the processing depth of the eye feature point based on the lateral pixel distance between the intersection point and the target reference column. The target reference column is the reference reference column corresponding to the target pineapple image.

[0067] Figure 5 This diagram illustrates a principle for calculating process depth. Figure 5 This is also a diagram taken from a top-down perspective. For example... Figure 5As shown, for the eye feature point P on the surface of pineapple 100, its processing depth is L; in the pineapple image acquired by camera 101, the circumferential angle of the eye feature point P can be determined, and by adding 90° to the circumferential angle, the circumferential angle corresponding to the reference point Q can be determined, which is the depth measurement angle mentioned above.

[0068] After rotating the pineapple 90° clockwise, the image of the pineapple captured by camera 101 is the target pineapple image corresponding to the depth measurement angle. At this point, reference point Q is located at the center of the target pineapple image, while the eye feature point P is located at the edge of the target pineapple image. Therefore, the distance between reference point Q and eye feature point P in the target pineapple image is the processing depth L of eye feature point P.

[0069] That is, in the target pineapple image, the intersection point of the pixel row where the eye feature point P is located (which can be determined based on the height of the eye feature point) and the outer contour of the pineapple body can be determined. This intersection point is the location of the eye feature point P.

[0070] Furthermore, each pineapple image has a reference column. The benchmark reference column Align with the axis of rotation of the pineapple image (e.g., axis 112); Figure 3 As shown, the benchmark reference column It can be the center column of the pineapple image. For ease of description, the reference column corresponding to the target pineapple image is called the "target reference column".

[0071] After determining the intersection point, the horizontal pixel distance between the intersection point and the target reference column can be determined in the target pineapple image. Then, based on the horizontal pixel distance, the process depth L of the fruit eye feature point P can be obtained.

[0072] In this embodiment, the process depth estimation of the fruit eye feature points does not process the outline of the fruit eye itself, but rather the outer outline of the entire pineapple body. In other words, the process depth is not directly calculated from the thickness of the fruit eye target frame, but is indirectly calculated through the geometric relationship between the row containing the fruit eye and the boundary of the pineapple body.

[0073] Step S206: Generate and output the fruit eye parameter set, which includes the circumferential angle, height and process depth of each fruit eye feature point.

[0074] The circumferential angle, height, and processing depth of each fruit eye feature point are finally generated to form a fruit eye parameter set for subsequent control.

[0075] For example, the visual parameter set can be sent to the PLC, written into the PLC register as an array, and the PLC can be notified to read it by execution pulse, so as to realize the linkage between visual detection results and actuator.

[0076] The visual detection method for pineapple eye feature points provided in this invention extracts the lateral pixel distance from the eye to the pineapple's body contour and then converts it according to a preset pixel-to-process depth coefficient to obtain the process depth to which the actuator can cut. Simultaneously, by introducing a dual-reference center column segmented interpolation and cross-angle scale dynamic correction algorithm, it solves the errors caused by mechanical eccentricity interference in rotational processing and the non-standard cylindrical characteristics of the pineapple. This solution extracts the lateral pixel distance from the eye to the pineapple's body contour as a monocular depth geometry, directly mapping the pixel coordinates to the cutting process depth. It can rely solely on a low-cost 2D camera to convert visual detection results into control quantities that can be directly used for industrial control with high precision and low latency. This overcomes the problems of high cost of 3D cameras and high computational complexity and poor stability of 2D depth estimation algorithms in existing technologies, making it suitable for automated pineapple processing, positioning, and actuator linkage scenarios.

[0077] In some optional implementations, the image pixel coordinates of the fruit eye feature points include horizontal pixel coordinates x and vertical pixel coordinates y; the above step "calculate the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image" may include steps a1 to a3.

[0078] Step a1: For any of the fruit eye feature points in the pineapple image, based on the horizontal pixel coordinates of the fruit eye feature point and the horizontal pixel offset between the reference column in the pineapple image, the local angle increment of the fruit eye feature point is determined by converting it according to a preset first ratio coefficient from pixel to angle.

[0079] Step a2: Add the local angle increment, the sampling reference angle, and the preset global angle offset, and obtain the circumferential angle of the fruit eye feature point after angle normalization processing.

[0080] Step a3: Based on the vertical pixel offset between the vertical pixel coordinates of the fruit eye feature point and the zero-point height reference row, the height of the fruit eye feature point is determined by converting it according to a preset second ratio coefficient from pixel to height.

[0081] Specifically, for the i-th eye feature point in the k-th frame of the pineapple image, its circumferential angle can be expressed as:

[0082] The corresponding height formula is:

[0083] in, This represents the horizontal pixel coordinate of the i-th fruit eye feature point; This column represents the baseline reference column, used for angle mapping and depth conversion. The first scaling factor representing the pixel-to-angle ratio; This represents the sampling reference angle corresponding to the k-th frame of the pineapple image; This represents the global angular offset, used to compensate for installation errors or zero-position deviations; it can be set to 0. This represents the vertical pixel coordinate of the i-th fruit eye feature point; The reference row indicating the location of the zero altitude point; This represents the second scaling factor from pixel to height. For the physical meaning of some parameters, please refer to [link / reference needed]. Figure 3 As shown.

[0084] This represents the lateral pixel offset of the feature point of the fruit eye relative to the reference column; under the premise of fixed imaging geometry and fixed field of view, this lateral pixel offset can be achieved through... Converted into the local angle increment of the fruit eye feature point; then superimposed with the sampling reference angle of the current image. and system angle zero compensation The circumferential angle of the fruit eye feature point is obtained after angle normalization.

[0085] by As the reference row where the height zero point is located, the vertical pixel offset is... Converted to the height of the fruit eye feature point.

[0086] For example, such as Figure 3 As shown, the sampling reference angle It is 15°. Horizontal pixel value =1280, =1096.

[0087] This can be obtained through actual calibration. =0.015. Global angle offset. =0, then according to It can be calculated In other words, the circumferential angle of the fruit eye feature point is 17.76°.

[0088] In some alternative implementations, the first scaling factor from pixel to angle Determined in the following ways: In the image sequence, the same target fruit eye feature point is matched in two adjacent pineapple images, and the horizontal pixel displacement of the target fruit eye feature point in the two adjacent pineapple images is determined. The first proportional coefficient from pixel to angle is determined based on the ratio of the difference in the sampling reference angle between two adjacent pineapple images to the horizontal pixel displacement.

[0089] Specifically, in cross-angular scale correction modes, it is not necessary to consistently use a single [scale]. Furthermore, the horizontal pixel displacement of the same eye feature point in adjacent reference angle images can be used to make a secondary correction to the first proportional coefficient corresponding to the angle.

[0090] Figure 6 This diagram illustrates two adjacent frames of pineapple images. (For pineapple images...) and These two images are two adjacent pineapple images in an image sequence, and their sampling reference angles are respectively... and If the same fruit eye feature point p is matched in two frames, the horizontal coordinates of the fruit eye feature point in the two frames are respectively , Then its horizontal pixel displacement is denoted as:

[0091] Based on this lateral pixel displacement, the first proportional coefficient corresponding to feature point p at this height position can be calculated:

[0092] in, This represents the sampling angle step size of the image sequence, which is the difference in the sampling reference angle between two pineapple images.

[0093] The difference in the sampling reference angle between two adjacent pineapple images is The same visual feature point p will have a lateral pixel displacement between these two frames. Therefore, the angular increment corresponding to a unit pixel can be directly obtained by dividing the known angular difference by the pixel displacement. Since the lateral displacement at different heights may vary slightly, the above... Establish a correspondence based on row coordinates, and then select the first proportional coefficient that is closest to the height of each fruit eye feature point to be tested.

[0094] In some alternative implementations, the reference column is determined in the following way: For any current pineapple image in the image sequence, determine the current sampling reference angle corresponding to the current pineapple image; If the current sampling reference angle is within the first angle range from the first sampling reference angle (e.g., 0°) to the second sampling reference angle (e.g., 180°), then with the current sampling reference angle as the independent variable, linear interpolation is performed between the first reference reference column corresponding to the first sampling reference angle and the second reference reference column corresponding to the second sampling reference angle to calculate the reference reference column corresponding to the current pineapple image. If the current sampling reference angle is located within the second angle range from the second sampling reference angle (e.g., 180°) to the first sampling reference angle (e.g., 0°, which is also 360°), then with the current sampling reference angle as the independent variable, linear interpolation is performed between the second reference reference column corresponding to the second sampling reference angle and the first reference reference column corresponding to the first sampling reference angle to calculate the reference reference column corresponding to the current pineapple image.

[0095] In this embodiment, industrial equipment (such as motors and rotating spindles) typically experiences mechanical eccentricity or radial runout of the spindle when rotating 360°. This causes a slight lateral drift in the projected position of the rotation axis center on the image plane as the rotation angle changes, resulting in different reference columns for the front and rear halves of the equipment. In this case, the two reference columns for the front and rear halves can be obtained through on-site calibration of the equipment. Specifically, a calibration component is placed on the rotating axis of the equipment, and when the equipment rotates to 0° and 180° respectively, the pixel column coordinates of the actual physical rotation center on the image are extracted using camera vision.

[0096] Specifically, when the reference columns for the front and rear halves of the equipment are different, i.e. In this case, piecewise linear interpolation can be used for the reference column. Among these, This is the first reference column corresponding to the first sampling reference angle (0°). This is the second reference column corresponding to the second sampling reference angle (180°).

[0097] when hour:

[0098] when hour:

[0099] in, This is the baseline reference column obtained after piecewise linear interpolation. The first reference column corresponds to the first sampling reference angle. This is the second reference column corresponding to the second sampling reference angle. This represents the sampling reference angle corresponding to the k-th frame image.

[0100] Therefore, the formula for the circumferential angle after the second correction can be:

[0101] in, The first scaling factor from pixel to angle is related to the feature point of the fruit eye. The pixel rows in the pineapple image are relevant.

[0102] It is understandable that the purpose of this correction step is not to change the definition of the circumferential angle, but to use the observation difference of the same eye feature point in adjacent angle images to compensate for the local angle ratio at different heights, so that the circumferential angle mapping is more in line with the actual image deformation.

[0103] Without considering mechanical eccentricity, the calculated eye feature's lateral pixel offset would include mechanical system errors. This would ultimately lead to the PLC's eye-cutting tool deviating from its cutting position (inaccurate eye-cutting). This embodiment introduces this interpolation design, which uses visual algorithms to compensate for insufficient underlying mechanical precision at a low cost.

[0104] In this embodiment, pixel coordinates are converted into circumferential angles and heights through linear calibration, achieving accurate mapping from the visual coordinate system to the process coordinate system. Cross-angle scale correction and piecewise linear interpolation compensate for angle distortions caused by mechanical eccentricity and non-standard cylinders, improving the accuracy of angle mapping. By introducing global angle offsets and a zero-point height reference row, installation errors and system zero-position deviations can be flexibly compensated, improving the system's adaptability.

[0105] In some optional embodiments, the method further includes: performing image segmentation processing on the target pineapple image to obtain a mask image corresponding to the target pineapple image; performing connected component identification on the mask image, and using the boundary of the largest connected component as the outer contour of the pineapple body.

[0106] A segmentation model can be used to segment the pineapple body, and the results can be saved as a reusable mask file. The mask file is named the original image filename with the suffix "_mask.jpg". The mask can be pre-generated after the image enters the processing flow, or it can be generated as needed during the depth measurement stage. If a corresponding mask file already exists for the image to be measured, that mask file is read directly; otherwise, a new mask is generated.

[0107] After performing thresholding on the masked binary graph, a set of connected components is obtained. Select the connected component with the largest area as the pineapple body region: The boundary of the largest connected component is taken as the outer contour of the pineapple body: .

[0108] Optionally, the method further includes: grouping the various fruit eye feature points according to the pineapple images corresponding to the depth measurement angles of each fruit eye feature point; the depth measurement angles of each fruit eye feature point in each group correspond to the same pineapple image; and uniformly calculating the processing depth of each fruit eye feature point corresponding to the target pineapple image.

[0109] In this embodiment, by grouping and batch processing, it is not necessary to perform segmentation and contour extraction for each fruit eye separately. Instead, the target pineapple image is first grouped, and then the process depth is uniformly calculated for multiple rows in the same group, thereby reducing redundant calculations.

[0110] Specifically, Figure 7 A schematic diagram illustrating the determination of the depth of the fruit eye process is shown. For example... Figure 7 As shown, the angle is calculated based on the depth. The target pineapple image was matched. Next, the target pineapple image Perform contour extraction; that is, first extract the contour from the target pineapple image. Perform instance segmentation, and take the largest connected component as the pineapple ontology region, i.e., ontology segmentation result M. k The boundary of the largest connected component is the outline of the pineapple itself.

[0111] Next, the target pineapple image Perform row scanning processing. For a specific target pineapple image... and any of the scan lines therein Find the set of coordinates of the intersection points of the horizontal line and the contour on contour C:

[0112] Where W is the image width, x [10,W-10] is used to remove intersections within a 10-pixel range of the left and right edges of the image, thus avoiding noise interference at the image boundaries.

[0113] In this embodiment, the first intersection column coordinate in the set is taken as the depth pixel column value, denoted as:

[0114] in, This represents the x-coordinate of the intersection point. This formula is used to extract the first element in an ordered set of intersection points, which is sorted in ascending order of x-coordinate. In this implementation, it is not necessary to take the left and right intersection points simultaneously and then calculate the median, mean, or radius; the coordinates of the first intersection point can be directly used as the basis for depth measurement.

[0115] The eye feature points that meet the matching criteria will be assigned to the same depth image group for unified processing. Let... Indicates being assigned to the same matching image If the set of feature point indices for the fruit eye is given, then the corresponding set of scan rows is denoted as:

[0116] Then, the ontology contour is extracted only once for the image, and the intersection of all rows in the set is calculated at once, and the depth is backfilled in batches.

[0117] In this embodiment, by shifting the depth measurement angle by 90°, the change of the overall outer contour of the pineapple to the horizontal section of the row is more obvious under this phase, making it more suitable as a geometric measurement basis from the center to the contour boundary; by phase matching and group batch processing, the repetitive segmentation and contour extraction calculations are reduced, significantly improving processing efficiency; by setting matching tolerance conditions, the accuracy and reliability of depth measurement are ensured.

[0118] In some optional implementations, the above step S205, "calculating the process depth of the fruit eye feature point based on the lateral pixel distance between the intersection point and the target reference column," may specifically include steps b1 to b3.

[0119] Step b1: Calculate the difference between the horizontal pixel coordinates of the target reference column and the intersection point to obtain the horizontal pixel distance.

[0120] Step b2: Calculate the lateral pixel distance according to the preset third proportional coefficient from pixel to process depth to obtain the original depth.

[0121] Step b3: Perform constraint processing on the original depth to obtain the process depth.

[0122] Specifically, assuming the target benchmark reference column is... Then the original depth for:

[0123] in, Let be the horizontal pixel coordinates of the intersection point corresponding to the i-th fruit eye feature point. This represents the third scaling factor from pixel to process depth.

[0124] The raw depth here is not the absolute spatial depth in the sense of binocular vision, structured light, or laser ranging. Its essence is the lateral pixel distance from the reference column to the first intersection point of the pineapple body boundary, multiplied by the calibration ratio. The process depth value can then be obtained. This method uses the lateral pixel distance as a monocular surrogate quantity related to the depth of the process action, and establishes a linear mapping relationship from pixels to process depth through on-site calibration. The final output is the depth in the sense of equipment control, rather than the true spatial depth in the sense of 3D reconstruction.

[0125] Furthermore, in this embodiment, the original depth can be determined first. Execution upper limit clamp:

[0126] Then, a global depth offset is added:

[0127] Finally, the lower limit clamping is performed to obtain the process depth. :

[0128] in, This represents the global depth offset, used to compensate for the zero position of the mechanism or the process allowance. The upper and lower limits of 55 and 25 in the formula are the preset upper and lower depth limits; the actual values ​​can be obtained through on-site debugging.

[0129] It is used to compensate for the zero position, mechanical allowance, or calibration margin of the actuator. In other words, even if the original process depth corresponding to the same lateral pixel distance is the same, the control depth can still be shifted to the actual process depth range under different equipment installation conditions through this offset.

[0130] In addition, when hour, It may be negative, in which case the original depth is... It could also be negative. In this case, subsequent upper and lower limit constraints can be used to ensure that the final output falls within a controllable range.

[0131] In this embodiment, the horizontal pixel distance from the fruit eye to the outline of the pineapple body is extracted as a monocular depth geometry proxy, which breaks through the high cost and high computing power bottleneck of traditional 2D vision in acquiring three-dimensional control quantities. Through upper limit clamping, depth offset and lower limit clamping constraint processing, the stability and controllability of the output depth are ensured, and the impact of outliers on process control is avoided. Through the mask caching mechanism, the computational overhead caused by repeated segmentation is reduced and the processing efficiency is improved.

[0132] In some optional implementations, before outputting the set of fruit eye parameters, the method further includes: performing deduplication processing on the detected fruit eye feature points: if the height difference between two fruit eye feature points is less than a preset height threshold and the angle difference is less than a preset angle threshold, then the two fruit eye feature points are determined to be the same fruit eye, and the fruit eye feature point with the highest confidence is retained.

[0133] For the set of fruit eye feature points obtained after multi-angle detection, they can be sorted by angle and deduplicated using the criterion of "similar height + similar angle".

[0134] For example, if the i-th fruit eye feature point and the j-th fruit eye feature point satisfy: mm, and, If both are considered to belong to the same visual feature point, only the more reliable one, i.e., the one with the highest confidence level, will be retained. Here, 6mm is a preset height threshold, which can be determined based on the actual situation.

[0135] This invention also provides an architecture diagram of a visual detection method for eye feature points in pineapple fruits, see [link / reference]. Figure 8 As shown, after image acquisition is completed, a dual-link collaborative processing architecture is adopted: The first processing link is used for fruit eye feature point detection and angle and height mapping, including: for any frame of pineapple image in the image sequence, fruit eye feature point detection is performed, and the image pixel coordinates of each fruit eye feature point in the image coordinate system are extracted; based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image, the circumferential angle and height of each fruit eye feature point in the pineapple image are calculated.

[0136] The second processing link is used for pineapple body segmentation, contour extraction, row scanning, and process depth calculation, including: for any of the eye feature points, determining the depth calculation angle corresponding to the eye feature point based on the circumferential angle of the eye feature point, and determining the target pineapple image in the image sequence that matches the sampling reference angle and the depth calculation angle; in the target pineapple image, determining the intersection point of the pixel row where the eye feature point is located and the outer contour of the pineapple body, and calculating the process depth of the eye feature point based on the horizontal pixel distance between the intersection point and the target reference column.

[0137] After the two links work together, they perform deduplication on the fruit eye feature points, generate and output the circumferential angle, height and process depth of the fruit eye feature points and send them to the PLC.

[0138] Based on the same inventive concept, this invention also provides a visual detection device for pineapple fruit eye feature points, see [link to relevant documentation]. Figure 9 As shown, the device includes: Image acquisition module 901 is used to acquire an image sequence obtained by circumferential acquisition of a pineapple, the image sequence including multiple frames of pineapple images acquired at multiple sampling reference angles respectively; The feature point detection module 902 is used to detect the fruit eye feature points of any pineapple image in the image sequence and extract the image pixel coordinates of each fruit eye feature point in the image coordinate system. Processing module 903 is used to calculate the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image; The depth image matching module 904 is used to determine the depth measurement angle corresponding to any of the fruit eye feature points based on the circumferential angle of the fruit eye feature points, and to determine the target pineapple image in the image sequence that matches the sampling reference angle with the depth measurement angle. The process depth calculation module 905 is used to determine the intersection point of the pixel row where the fruit eye feature point is located and the outer contour of the pineapple body in the target pineapple image, and calculate the process depth of the fruit eye feature point based on the horizontal pixel distance between the intersection point and the target reference column; the target reference column is the reference reference column corresponding to the target pineapple image; The output module 906 is used to generate and output a set of fruit eye parameters, which includes the circumferential angle, height and process depth of each fruit eye feature point.

[0139] In some optional implementations, the image pixel coordinates of the fruit eye feature point include horizontal pixel coordinates and vertical pixel coordinates; The step of calculating the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image includes: For any of the fruit eye feature points in the pineapple image, based on the horizontal pixel coordinates of the fruit eye feature point and the horizontal pixel offset between the reference column in the pineapple image, the local angle increment of the fruit eye feature point is determined by converting it according to a preset first ratio coefficient from pixel to angle. The local angle increment, the sampling reference angle, and the preset global angle offset are added together, and the circumferential angle of the fruit eye feature point is obtained after angle normalization. Based on the vertical pixel offset between the vertical pixel coordinates of the fruit eye feature point and the height zero reference row, the height of the fruit eye feature point is determined by conversion according to a preset second ratio coefficient from pixel to height.

[0140] In some alternative implementations, the reference column is determined in the following manner: For any current pineapple image in the image sequence, determine the current sampling reference angle corresponding to the current pineapple image; If the current sampling reference angle is within the first angle range from the first sampling reference angle to the second sampling reference angle, then with the current sampling reference angle as the independent variable, linear interpolation is performed between the first reference reference column corresponding to the first sampling reference angle and the second reference reference column corresponding to the second sampling reference angle to calculate the reference reference column corresponding to the current pineapple image. If the current sampling reference angle is within the range of the second angle corresponding to the second sampling reference angle to the first sampling reference angle, then with the current sampling reference angle as the independent variable, linear interpolation is performed between the second reference column corresponding to the second sampling reference angle and the first reference column corresponding to the first sampling reference angle to calculate the reference reference column corresponding to the current pineapple image.

[0141] In some alternative implementations, the first pixel-to-angle scaling factor is determined in the following manner: In the image sequence, the same target fruit eye feature point is matched in two adjacent pineapple images, and the horizontal pixel displacement of the target fruit eye feature point in the two adjacent pineapple images is determined. The first proportional coefficient from pixel to angle is determined based on the ratio of the difference in the sampling reference angle between two adjacent pineapple images to the horizontal pixel displacement.

[0142] In some optional implementations, the depth measurement angle corresponding to the fruit eye feature point differs from the circumferential angle of the fruit eye feature point by 90°; The step of calculating the process depth of the fruit eye feature point based on the lateral pixel distance between the intersection point and the target reference column includes: The difference between the horizontal pixel coordinates of the target reference column and the intersection point is calculated to obtain the horizontal pixel distance; The horizontal pixel distance is converted according to a preset third proportional coefficient from pixel to process depth to obtain the original depth; The original depth is subjected to constraint processing to obtain the process depth.

[0143] In some optional embodiments, the processing module 903 is further configured to: The target pineapple image is segmented to obtain a mask image corresponding to the target pineapple image; connected component identification is performed on the mask image, and the boundary of the largest connected component is used as the outer contour of the pineapple body; And / or, Based on the pineapple image corresponding to the depth measurement angle of each of the fruit eye feature points, the fruit eye feature points are grouped; the depth measurement angle of each fruit eye feature point in each group corresponds to the same pineapple image; for the target pineapple image, the processing depth of each fruit eye feature point corresponding to the target pineapple image is calculated uniformly.

[0144] In some alternative implementations, before the output fruit eye parameter set, the processing module 903 is further configured to: The detected fruit eye feature points are deduplicated: if the height difference between two fruit eye feature points is less than a preset height threshold and the angle difference is less than a preset angle threshold, then the two fruit eye feature points are determined to be the same fruit eye, and the fruit eye feature point with the highest confidence is retained.

[0145] Figure 10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown.

[0146] The following is a detailed reference. Figure 10The diagram illustrates a structural schematic suitable for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from memory 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device. The processor 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0147] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0148] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1009, or installed from a memory 1008, or installed from a ROM 1002. When the computer program is executed by the processor 1001, it performs the functions defined in the visual detection method for pineapple eye feature points according to embodiments of this application.

[0149] Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0150] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the visual detection method for pineapple eye feature points shown in the above embodiments is implemented.

[0151] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0152] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A visual detection method for eye feature points of a pineapple fruit, characterized in that, The method includes: A sequence of images of a pineapple is obtained by circumferentially acquiring the images, the image sequence including multiple frames of pineapple images acquired at multiple sampling reference angles; For any frame of pineapple image in the image sequence, perform fruit eye feature point detection on the pineapple image and extract the image pixel coordinates of each fruit eye feature point in the image coordinate system; Based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image, calculate the circumferential angle and height of each fruit eye feature point in the pineapple image; For any of the aforementioned fruit eye feature points, the depth measurement angle corresponding to the fruit eye feature point is determined based on the circumferential angle of the fruit eye feature point, and the target pineapple image in the image sequence that matches the sampling reference angle and the depth measurement angle is determined. In the target pineapple image, the intersection point of the pixel row containing the eye feature point and the outer contour of the pineapple body is determined. Based on the horizontal pixel distance between the intersection point and the target reference column, the processing depth of the eye feature point is calculated. The target reference column is the reference reference column corresponding to the target pineapple image. Generate and output a set of fruit eye parameters, which includes the circumferential angle, height and process depth of each fruit eye feature point.

2. The visual detection method for pineapple fruit eye feature points according to claim 1, characterized in that, The image pixel coordinates of the fruit eye feature points include horizontal pixel coordinates and vertical pixel coordinates; The step of calculating the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image includes: For any of the fruit eye feature points in the pineapple image, based on the horizontal pixel coordinates of the fruit eye feature point and the horizontal pixel offset between the reference column in the pineapple image, the local angle increment of the fruit eye feature point is determined by converting it according to a preset first ratio coefficient from pixel to angle. The local angle increment, the sampling reference angle, and the preset global angle offset are added together, and the circumferential angle of the fruit eye feature point is obtained after angle normalization. Based on the vertical pixel offset between the vertical pixel coordinates of the fruit eye feature point and the height zero reference row, the height of the fruit eye feature point is determined by conversion according to a preset second ratio coefficient from pixel to height.

3. The visual detection method for pineapple fruit eye feature points according to claim 1 or 2, characterized in that, The reference column is determined in the following way: For any current pineapple image in the image sequence, determine the current sampling reference angle corresponding to the current pineapple image; If the current sampling reference angle is within the first angle range from the first sampling reference angle to the second sampling reference angle, then with the current sampling reference angle as the independent variable, linear interpolation is performed between the first reference reference column corresponding to the first sampling reference angle and the second reference reference column corresponding to the second sampling reference angle to calculate the reference reference column corresponding to the current pineapple image. If the current sampling reference angle is within the range of the second angle corresponding to the second sampling reference angle to the first sampling reference angle, then with the current sampling reference angle as the independent variable, linear interpolation is performed between the second reference column corresponding to the second sampling reference angle and the first reference column corresponding to the first sampling reference angle to calculate the reference reference column corresponding to the current pineapple image.

4. The visual detection method for pineapple fruit eye feature points according to claim 2, characterized in that, The first scaling factor from pixel to angle is determined in the following way: In the image sequence, the same target fruit eye feature point is matched in two adjacent pineapple images, and the horizontal pixel displacement of the target fruit eye feature point in the two adjacent pineapple images is determined. The first proportional coefficient from pixel to angle is determined based on the ratio of the difference in the sampling reference angle between two adjacent pineapple images to the horizontal pixel displacement.

5. The visual detection method for pineapple eye feature points according to claim 1, characterized in that, The depth measurement angle corresponding to the fruit eye feature point differs from the circumferential angle of the fruit eye feature point by 90°; The step of calculating the process depth of the fruit eye feature point based on the lateral pixel distance between the intersection point and the target reference column includes: The difference between the horizontal pixel coordinates of the target reference column and the intersection point is calculated to obtain the horizontal pixel distance; The horizontal pixel distance is converted according to a preset third proportional coefficient from pixel to process depth to obtain the original depth; The original depth is subjected to constraint processing to obtain the process depth.

6. The visual detection method for pineapple eye feature points according to claim 1, characterized in that, The method further includes: The target pineapple image is segmented to obtain a mask image corresponding to the target pineapple image; connected component identification is performed on the mask image, and the boundary of the largest connected component is used as the outer contour of the pineapple body; And / or, Based on the pineapple image corresponding to the depth measurement angle of each of the fruit eye feature points, the fruit eye feature points are grouped; the depth measurement angle of each fruit eye feature point in each group corresponds to the same pineapple image; for the target pineapple image, the processing depth of each fruit eye feature point corresponding to the target pineapple image is calculated uniformly.

7. The visual detection method for pineapple eye feature points according to claim 1, characterized in that, Before the output fruit eye parameter set, it also includes: The detected fruit eye feature points are deduplicated: if the height difference between two fruit eye feature points is less than a preset height threshold and the angle difference is less than a preset angle threshold, then the two fruit eye feature points are determined to be the same fruit eye, and the fruit eye feature point with the highest confidence is retained.

8. A visual detection device for eye feature points of a pineapple fruit, characterized in that, The device includes: The image acquisition module is used to acquire an image sequence obtained by circumferentially acquiring images of a pineapple. The image sequence includes multiple frames of pineapple images acquired at multiple sampling reference angles. The feature point detection module is used to detect the fruit eye feature points of any pineapple image in the image sequence and extract the image pixel coordinates of each fruit eye feature point in the image coordinate system. The circumferential angle and height calculation module is used to calculate the circumferential angle and height of each fruit eye feature point in the pineapple image based on the image pixel coordinates and the sampling reference angle corresponding to the pineapple image; The depth image matching module is used to determine the depth measurement angle corresponding to any of the fruit eye feature points based on the circumferential angle of the fruit eye feature points, and to determine the target pineapple image in the image sequence that matches the sampling reference angle with the depth measurement angle. The process depth calculation module is used to determine the intersection point of the pixel row containing the fruit eye feature point and the outer contour of the pineapple body in the target pineapple image, and calculate the process depth of the fruit eye feature point based on the horizontal pixel distance between the intersection point and the target reference column; the target reference column is the reference reference column corresponding to the target pineapple image; The output module is used to generate and output a set of fruit eye parameters, which includes the circumferential angle, height and process depth of each of the fruit eye feature points.

9. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the visual detection method for pineapple eye feature points according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the visual detection method for pineapple eye feature points according to any one of claims 1 to 7.