Fruit core area positioning, fruit core outer end and fruit radius ratio calculation method and system

Abstract

The present invention relates to a method and system for locating the core area and calculating the ratio of the outer end of the core room to the fruit radius, which comprises: making a data set and training network parameters to construct a network model; inputting an image in a real-time video acquisition into the network model to locate the core and flesh position area in the acquired video image; in the core and flesh position area, determining the core area, and calculating the ratio of the outer end of the core room to the fruit radius according to the core area. The present invention can accurately distinguish apple flesh and core, accurately calculate the ratio of the outer end of the core room to the fruit radius, simplify the apple quality detection process, and improve the accuracy of grade classification. At the same time, it can accurately locate the core area and provide data support for mechanized fine core removal; it can be widely used in the field of fruit quality detection technology.

Description

Technical Field

[0001] The invention relates to the technical field of fruit quality detection, and in particular to a method and system for locating a fruit core area and calculating the ratio of the outer end of the fruit core chamber to the fruit radius. Background Art

[0002] As people's living standards improve, apple consumption and production are increasing year by year. The core is the inedible part of the apple, and the size of the core as a percentage of the apple is an important factor in determining the quality of the apple. Among the existing core size evaluation indicators, the evaluation specification requires: randomly select 10 fruits, cut them at one time along the maximum horizontal diameter of the fruit, and the cross-section is as follows: Figure 1 As shown, the relative position of the outer end of the fruit core to the fruit radius is observed (calculated as the average value of 10 fruits), and the size of the apple core is divided into three grades: small, medium and large. The classification standards are shown in Table 1.

[0003] Table 1 Evaluation criteria for apple core size

[0004]

[0005] The traditional method of measuring the outer end of the fruit core and the radius of the fruit relies on vernier calipers and visual inspection. Due to factors such as the eccentricity and irregular shape of the fruit core, the measurement error is large. After measurement, the data needs to be manually entered to calculate the mean, which is labor-intensive and time-consuming.

[0006] In the deep processing of apples, removing the core is a very important step. With the improvement of mechanical automation, many automated core removal devices have been invented. These devices rely on mechanical force to remove the internal cylindrical area of ​​the apple from top to bottom. The shape and size of the apple area removed by these methods are fixed, and there is a large amount of edible flesh, which is seriously wasted. Providing these mechanical devices with accurate location data of the apple area to be removed is the best solution to improve the utilization rate of apples. Summary of the invention

[0007] In view of the above problems, the purpose of the present invention is to provide a method and system for locating the core area and calculating the radius ratio of the core outer end to the fruit, which can accurately distinguish the apple flesh and core, accurately calculate the radius ratio of the core outer end to the fruit, simplify the apple quality detection process, and improve the accuracy of grade classification. At the same time, the core area can be accurately located to provide data support for mechanized fine core removal.

[0008] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a method for locating the fruit core area and calculating the ratio of the outer end of the fruit core room to the radius of the fruit, which comprises: preparing a data set and training network parameters to construct a network model; inputting images in real-time video acquisition into the network model to locate the fruit core and fruit pulp position area in the acquired video image; in the located fruit core and fruit pulp position area, determining the fruit core area, and calculating the ratio of the outer end of the fruit core room to the radius of the fruit based on the fruit core area.

[0009] Further, the making of the data set includes:

[0010] Use the image annotation tool Labelme to select the core and flesh area edges of the fruit and annotate the pixels belonging to different areas;

[0011] The annotated images are randomly divided into training set, validation set and test set according to a preset ratio.

[0012] Furthermore, the network model is an improved Unet convolutional neural network model; the network model includes:

[0013] The network model uses a backbone feature extraction network of the VGG16 structure to obtain a total of 5 preliminary effective feature layers, and passes the obtained fifth preliminary effective feature layer as input to the ViT structure;

[0014] The feature layer first passes through the Patch Embedding layer, where the kernel_size and stride of the convolution layer Conv2d are 1, which is only used to adjust the channel value; Position Embedding is performed before being input to the Transformer Encoder layer to supplement the spatial information;

[0015] In the Transformer Encoder layer, the Encoder Block is stacked 12 times. The Encoder Block includes LayerNorm, Multi-Head Attention, Drop Path, and MLP Block.

[0016] The ViT output result is reshaped and then upsampled, stacked with the valid feature layer of the previous layer, and then two standard convolutions are performed to enhance feature extraction to obtain the final valid feature layer;

[0017] After the SegmentationHead layer, the number of channels of the obtained effective feature layer is adjusted to 3 categories.

[0018] Furthermore, the loss function of the network model includes: a cross entropy loss function and a Dice Loss loss function.

[0019] Further, the positioning and collecting of the fruit core and fruit pulp position areas in the video image includes:

[0020] The network model is used to segment the real-time collected image, the flesh, the core and the background area are separated, and the area of ​​the area is calculated according to the number of pixels in the area;

[0021] Evaluate the segmentation results to achieve accurate segmentation;

[0022] The segmented single-channel pulp and core images are expanded using a smoothing operator to form a connected domain, fill the holes in the segmented images, and achieve a hole-free area in the pulp and core regions.

[0023] Furthermore, the evaluation of the segmentation result includes: evaluating using recall rate, precision rate, segmentation accuracy rate, average pixel accuracy rate, average intersection-over-union ratio and F-value score.

[0024] Further, the determining of the fruit core area and calculating the ratio of the outer end of the fruit core chamber to the fruit radius according to the fruit core area include:

[0025] After image processing of the single-channel flesh and core images after precise segmentation, the minimum circumscribed circle method is used to simulate the approximate outline of the fruit contour to obtain the circumscribed circle radius;

[0026] The minimum circumscribed circle method is used for the fruit center to obtain the average value of the radius of the outer end of the ventricle of the fruit center;

[0027] Divide the circumscribed circle radius of the fruit center by the circumscribed circle radius of the fruit to get the ratio of the two.

[0028] A fruit core area positioning and fruit core outer end and fruit radius ratio calculation system comprises: a model construction module, which makes a data set and trains network parameters to construct a network model; a positioning module, which inputs an image in a real-time video acquisition into the network model to locate the fruit core and fruit pulp position area in the captured video image; and a radius ratio acquisition module, which determines the fruit core area in the located fruit core and fruit pulp position area, and calculates the ratio of the fruit core outer end and the fruit radius based on the fruit core area.

[0029] A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any of the above methods.

[0030] A computing device comprises: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above methods.

[0031] The present invention adopts the above technical solution, which has the following advantages:

[0032] 1. The present invention implements a method for measuring the core and radius of the fruit based on machine vision, and can accurately locate the core area and calculate the core and flesh area ratio of the apple cross section. The present invention changes the evaluation standard of the apple core size from manual visual inspection to automatic recognition and statistical calculation, which improves the detection efficiency and the detection accuracy of the evaluation standard. It also provides data basis for accurately removing the core and improving the utilization rate of apples during the deep processing of apples.

[0033] 2. The improved Unet image segmentation model based on Xception convolution in the present invention reduces the calculation amount of the model, speeds up the recognition speed, and achieves the purpose of optimizing the model.

[0034] 3. In the data acquisition method of the present invention, the distance between the camera and the cross section of the apple is fixed through the image acquisition box. The actual area corresponding to each pixel can be calculated using the camera parameters and the distance. At the same time, the position of the camera and the apple can be used to accurately calculate the center position area of ​​the fruit.

[0035] 4. The present invention calculates the area size of the region by using the number of pixels through the region segmentation results and the distance of the acquisition device, finds the minimum concentric circles of different regions, and calculates the ratio of the radius of the pulp and the core. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a model diagram of the size of the apple core in the prior art;

[0037] Figure 2 It is a flow chart of a method for locating the fruit core area and calculating the ratio of the outer end of the fruit core area to the fruit radius in an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of an improved Unet convolutional neural network model in an embodiment of the present invention;

[0039] Figure 4 is a graph showing the change of the loss function during the training process in an embodiment of the present invention;

[0040] Figure 5a In the embodiment of the present invention, an image is captured;

[0041] Figure 5b This is a diagram showing the effect after segmentation in the embodiment of the present invention;

[0042] Figure 6 Schematic diagram of the circumscribed circle of the pulp and core area in the embodiment of the present invention;

[0043] Figure 7 It is a schematic diagram of the structure of a computing device in one embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0046] The present invention provides a method and system for locating the core area and calculating the ratio of the outer end of the core to the fruit radius based on machine vision, including: making a data set and training network parameters to build a network model; inputting images in real-time video acquisition into the network model to locate the core and flesh position areas in the acquired video images; in the located core and flesh position areas, determining the core area, and calculating the ratio of the outer end of the core to the fruit radius according to the core area. The present invention achieves the purpose of changing the evaluation standard of the apple core size from manual visual inspection to automatic recognition and statistical calculation, and improves the detection accuracy of the evaluation standard.

[0047] In one embodiment of the present invention, a method for locating the core area and calculating the radius ratio of the outer end of the core area to the fruit is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, based on an improved Unet convolutional neural network, the core and flesh areas in the image are marked using Labelme software, and the network parameters are trained using the marked images to construct a network model for segmenting the apple core and flesh. This achieves real-time and accurate segmentation of the core, flesh, and background areas in video images, and then uses the minimum circumscribed circle to calculate the radius ratio of the core and the fruit. Figure 2 As shown, the method comprises the following steps:

[0048] 1) Create a data set and train network parameters to build a network model;

[0049] 2) Input the image in the real-time video collection into the network model to locate the core and pulp area in the video collection image;

[0050] 3) In the area where the fruit core and the fruit flesh are located, the fruit core area is determined, and the ratio of the outer end of the fruit core chamber to the radius of the fruit is calculated based on the fruit core area.

[0051] In the above step 1), making a data set includes the following steps:

[0052] 1.1) Use the image annotation tool Labelme to select the fruit core and the edge of the flesh area, and label the pixels belonging to different areas;

[0053] In this embodiment, more than 300 apples of different qualities are randomly purchased, and according to the requirements of the existing agricultural industry standard apple quality index evaluation specification, they are cut horizontally at the maximum radius. The distance between the camera and the apple is fixed, and the camera is used to vertically shoot the cross-section of the left and right halves of the apple, to obtain nearly a thousand apple cross-section image data.

[0054] 1.2) The annotated images are randomly divided into a training set, a validation set, and a test set according to a preset ratio. In this embodiment, the preset ratio is preferably 7:2:1.

[0055] In the above step 1), the network model is an improved Unet convolutional neural network model, such as Figure 3 As shown in the figure; the structural features of the VGG16 network are adopted in the backbone feature extraction network of the model, and the structure of the first 13 layers of the VGG16 network is used, that is, the last three fully connected layers of the VGG16 network are removed in the feature extraction network part, and only the first 13 convolutional layers are retained; the model also adds the Vision Transformer (ViT) structure. Before upsampling, the feature layer obtained by the backbone feature extraction network is input into the ViT structure, and the output result of ViT is further upsampled after passing through the Patch Embedding layer, Transformer Encoder layer and SegmentationHead layer respectively.

[0056] The network model is as follows: the network model uses the backbone feature extraction network of the VGG16 structure to obtain a total of 5 preliminary effective feature layers, and passes the obtained fifth preliminary effective feature layer as input to the ViT structure;

[0057] The feature layer first passes through the Patch Embedding layer, where the kernel_size and stride of the convolution layer Conv2d are 1, which is only used to adjust the channel value; Position Embedding is performed before being input to the Transformer Encoder layer to supplement the spatial information;

[0058] In the Transformer Encoder layer, the Encoder Block is stacked 12 times. The Encoder Block includes LayerNorm, Multi-Head Attention, Drop Path, and MLP Block.

[0059] The ViT output result is reshaped and then upsampled, stacked with the valid feature layer of the previous layer, and then two standard convolutions are performed to enhance feature extraction to obtain the final valid feature layer;

[0060] Finally, the number of channels of the obtained effective feature layer is adjusted to 3 categories through the SegmentationHead layer.

[0061] The improved Unet convolutional neural network model used in this embodiment is still composed of two parts: the backbone feature extraction network and the enhanced feature extraction network. This model can effectively reduce the calculation time of the network model and improve the performance of the segmentation network without losing counting accuracy. Improve the reliability of the detection method. The backbone feature extraction network structure parameter configuration is shown in Table 2.

[0062] Table 2 Backbone feature extraction network structure parameter configuration

[0063]

[0064] In the above embodiment, the loss function of the network model includes: a cross entropy loss function and a Dice Loss loss function.

[0065] Among them, cross entropy is mainly used to express the difference between the input and output results of the neural network. The smaller the cross entropy, the closer the input and output results are. The cross entropy loss function is shown in formula (1), where L is the cross entropy, M represents the number of categories, and y true is a one-hot vector. If the category is the same as the sample category, it takes 1, otherwise it takes 0. pred Represents the probability that the predicted sample belongs to c.

[0066]

[0067] Dice Loss is an evaluation index for semantic segmentation, which is used to solve the scenario of extreme imbalance between positive and negative samples in semantic segmentation. Dice Loss comes from Dice coefficient, which is usually used to evaluate the similarity between two samples. Its value range is between 0 and 1. The larger the value of Dice coefficient, the more similar it is. The definition formula is shown in formula (2), where |x∩y| represents the intersection of x and y, |x| and |y| represent the number of elements in x and y respectively. When x and y are highly overlapped, in order to ensure that the value of Dice coefficient is between [0,1], multiply the numerator by 2.

[0068]

[0069] For the semantic segmentation task, |x| and |y| represent the ground truth and perdict mask respectively. Based on the Dice coefficient, the calculation formula of Dice Loss is shown in formula (3). The smaller the value of Dice Loss, the better the effect of the segmentation network.

[0070]

[0071] In this embodiment, the step of training the model parameters is also included. The initial learning rate of the model training is 0.0001, and the stochastic gradient descent method is used for training. The decay rate is set to 0.9, the batch size is set to 2, and the training is performed for 50 epoch iterations. In order to improve the accuracy, the present invention will improve the Unet network to perform pre-training on the VOC2007 data set, obtain the pre-training weights, and then perform transfer learning. After freezing the weights for 10 times, continue training for 40 times to obtain the final model parameters. The loss function changes during the training process as shown in the figure. Figure 4 shown.

[0072] In the above step 2), locating the fruit core and fruit pulp position area in the captured video image includes the following steps:

[0073] 2.1) Use the network model to segment the real-time collected image, separate the flesh, core and background areas, and calculate the area of ​​the area based on the number of pixels in the area;

[0074] In this embodiment, the height of the photography studio for producing the captured video is preferably 40cm*40cm, LED light sources are used on the top and both sides, the distance between the fixed camera and the cross-section of the apple is 12cm, and the captured image is transmitted to the model in real time through the computer U port, the actual area value per unit pixel is calculated using the camera parameters and distance, the trained model is used to segment the flesh, core and background areas, and the regional area is calculated based on the number of pixels in the area.

[0075] 2.2) Evaluate the segmentation results to achieve accurate segmentation;

[0076] 2.3) Using a smoothing operator, the segmented single-channel pulp and core images are expanded to form a connected domain, filling the holes in the segmented image to achieve a hole-free pulp and core region.

[0077] In the above step 2.2), the evaluation of the segmentation results includes: using recall rate (Recall), precision (Precision), segmentation accuracy (Accuracy, Acc), mean pixel accuracy (Mean Pixel Accuracy, MPA), mean intersection over Union (Mean Intersection over Union, MIoU) and F-value score (F1-Score) for evaluation.

[0078] The calculation formulas for recall and precision are shown in formula (4) and formula (5), where TP represents that both the prediction and the actual are positive samples; FP represents that the prediction result is a positive sample and the actual is a negative sample; FN represents that the prediction is a negative sample and the actual is a positive sample; TN represents that the prediction is a negative sample and the actual is also a negative sample. The calculation formulas for segmentation accuracy and average intersection-over-union ratio are shown in formula (6) and formula (7). The evaluation indicators MPA and MIoU in this embodiment need to divide formula (6) and formula (7) by the total number of samples to obtain the average.

[0079]

[0080]

[0081]

[0082]

[0083] The calculation formula of F-Score is shown in formula (8), where β controls the importance of Precision and Recall. When β = 1, as shown in formula (9), the corresponding F1-score indicates that Precision and Recall are equally important. F1-score is an important indicator for evaluating the quality of segmentation results. The larger the value of F1-score, the better the segmentation effect.

[0084]

[0085]

[0086] like Figure 5a , Figure 5b Shown is the image taken. The central five-pointed star shape is the core of the fruit, the outer circular area is the flesh, and the black is the background.

[0087] In the above step 3), the fruit core area is determined, and the ratio of the outer end of the fruit core chamber to the fruit radius is calculated based on the fruit core area, which includes the following steps:

[0088] 3.1) After image processing of the single-channel flesh and core images after precise segmentation, considering that the fruit is round, the minimum circumscribed circle method is used to simulate the approximate outline of the fruit contour to obtain the circumscribed circle radius; among them, the circle can better display the geometric information of the image compared with rectangles, ellipses, etc.

[0089] In this embodiment, image processing such as Otsu thresholding method is performed on the image.

[0090] 3.2) Since the fruit center is in the shape of a five-pointed star, the minimum circumscribed circle method is used to obtain the average radius of the outer end of the fruit center. The circumscribed circle result is as follows Figure 6 shown.

[0091] 3.3) Divide the circumcircle radius of the fruit center by the circumcircle radius of the fruit to get the ratio of the two.

[0092] Among them, the ratio of the fruit center to the radius of the fruit circumscribed circle for:

[0093]

[0094] Among them, R C R is the radius of the circumscribed circle of the fruit center, F is the radius of the circumscribed circle of the fruit. The evaluation criteria for the fruit core size are shown in Table 1.

[0095] The present invention uses an improved Unet network model to process the camera-collected images to obtain data such as the number of pixels of the flesh and the core, the ratio of the radius of the flesh and the core, etc. These data are stored in a database to obtain a segmentation map of the apple fruit and the core. The core size is evaluated using formula (10) on the segmentation map, and the calculation results are displayed on an interactive interface.

[0096] In summary, the present invention realizes the measurement of the core, flesh area and radius based on machine vision, and can accurately locate the core area and calculate the core and flesh area positions of the apple cross section. The present invention changes the evaluation standard of the apple core size from manual visual inspection to automatic recognition and statistical calculation, which improves the detection efficiency and accuracy.

[0097] In one embodiment of the present invention, a system for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius is provided, which comprises:

[0098] Model building module, making data sets and training network parameters to build network models;

[0099] The positioning module inputs the image in the real-time video into the network model to locate the core and pulp area in the video image;

[0100] The radius ratio acquisition module determines the fruit core area in the fruit core and flesh position area, and calculates the radius ratio of the outer end of the fruit core chamber to the fruit based on the fruit core area.

[0101] The system provided in this embodiment is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for specific processes and detailed contents, which will not be repeated here.

[0102] like Figure 7 As shown, it is a schematic diagram of the structure of a computing device provided in an embodiment of the present invention. The computing device can be a terminal, which can include: a processor, a communication interface, a memory, a display screen and an input device. Among them, the processor, the communication interface and the memory complete mutual communication through a communication bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. When the computer program is executed by the processor, a computing method is implemented; the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be realized through WIFI, a management network, NFC (near field communication) or other technologies. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computing device, or an external keyboard, touchpad or mouse. The processor can call the logic instructions in the memory to execute the following method: create a data set and train network parameters to build a network model; input the image in the real-time video acquisition into the network model to locate the fruit core and flesh position area in the acquired video image; in the located fruit core and flesh position area, determine the fruit core area, and calculate the ratio of the outer end of the fruit core chamber to the fruit radius based on the fruit core area.

[0103] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0104] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computing device to which the scheme of the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] In one embodiment of the present invention, a computer program product is provided, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, comprising: making a data set and training network parameters, and constructing a network model; inputting images in real-time video capture into the network model, and locating the core and pulp position areas in the captured video images; in locating the core and pulp position areas, determining the core area, and calculating the ratio of the outer end of the core chamber to the radius of the fruit based on the core area.

[0106] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, which stores server instructions. The computer instructions enable a computer to execute the methods provided in the above embodiments, for example, including: making a data set and training network parameters to build a network model; inputting images in real-time video capture into the network model to locate the core and pulp position areas in the captured video image; in locating the core and pulp position areas, determining the core area, and calculating the radius ratio of the outer end of the core chamber to the fruit based on the core area.

[0107] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius, characterized in that: include: Create data sets and train network parameters to build network models; Input the image in the real-time video collection into the network model to locate the core and pulp position areas in the video collection image; In the fruit core and flesh location area, a fruit core area is determined, and a ratio of the outer end of the fruit core to the fruit radius is calculated based on the fruit core area; The network model is an improved Unet convolutional neural network model; the network model includes: The network model uses a backbone feature extraction network of the VGG16 structure to obtain a total of 5 preliminary effective feature layers, and passes the obtained fifth preliminary effective feature layer as input to the ViT structure; The feature layer first passes through the Patch Embedding layer, where the kernel_size and stride of the convolutional layer Conv2d are 1, which is only used to adjust the channel value; Position Embedding is performed before being input to the Transformer Encoder layer to supplement the spatial information; In the Transformer Encoder layer, the Encoder Block is stacked 12 times. The Encoder Block includes LayerNorm, Multi-Head Attention, Drop Path, and MLP Block. The ViT output result is reshaped and then upsampled, stacked with the valid feature layer of the previous layer, and then two standard convolutions are performed to enhance feature extraction to obtain the final valid feature layer; After the SegmentationHead layer, the number of channels of the obtained effective feature layer is adjusted to 3 categories.

2. The method for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius as claimed in claim 1, characterized in that: The production data set includes: Use the image annotation tool Labelme to select the core and flesh area edges of the fruit and annotate the pixels belonging to different areas; The annotated images are randomly divided into training set, validation set and test set according to a preset ratio.

3. The method for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius as claimed in claim 1, characterized in that: The loss functions of the network model include: a cross entropy loss function and a Dice Loss loss function.

4. The method for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius as claimed in claim 1, characterized in that: The positioning and acquisition of the fruit core and fruit pulp position area in the video image includes: The network model is used to segment the real-time collected image, the flesh, the core and the background area are separated, and the area of ​​the area is calculated according to the number of pixels in the area; Evaluate the segmentation results to achieve accurate segmentation; The segmented single-channel pulp and core images are expanded using a smoothing operator to form a connected domain, fill the holes in the segmented images, and achieve a hole-free area in the pulp and core regions.

5. The method for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius as claimed in claim 4, characterized in that: The evaluation of the segmentation result includes: evaluating by recall rate, precision rate, segmentation accuracy rate, average pixel accuracy rate, average intersection-over-union ratio and F-value score.

6. The method for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius as claimed in claim 1, characterized in that: Determining the fruit core area and calculating the ratio of the outer end of the fruit core chamber to the fruit radius according to the fruit core area include: After image processing of the single-channel flesh and core images after precise segmentation, the minimum circumscribed circle method is used to simulate the approximate outline of the fruit contour to obtain the circumscribed circle radius; The minimum circumscribed circle method is used for the fruit center to obtain the average value of the radius of the outer end of the ventricle of the fruit center; Divide the circumscribed circle radius of the fruit center by the circumscribed circle radius of the fruit to get the ratio of the two.

7. A system for locating the fruit core area and calculating the ratio of the outer end of the fruit core to the fruit radius, characterized in that: include: Model building module, making data sets and training network parameters to build network models; A positioning module, which inputs the image in the real-time video collection into the network model to locate the core and pulp position area in the video collection image; A radius ratio acquisition module, in the fruit core and flesh location area, determines the fruit core area, and calculates the radius ratio of the outer end of the fruit core room to the fruit according to the fruit core area; The network model is an improved Unet convolutional neural network model; the network model includes: The network model uses a backbone feature extraction network of the VGG16 structure to obtain a total of 5 preliminary effective feature layers, and passes the obtained fifth preliminary effective feature layer as input to the ViT structure; The feature layer first passes through the Patch Embedding layer, where the kernel_size and stride of the convolutional layer Conv2d are 1, which is only used to adjust the channel value; Position Embedding is performed before being input to the Transformer Encoder layer to supplement the spatial information; In the Transformer Encoder layer, the Encoder Block is stacked 12 times. The Encoder Block includes LayerNorm, Multi-Head Attention, Drop Path, and MLP Block. The ViT output result is reshaped and then upsampled, stacked with the valid feature layer of the previous layer, and then two standard convolutions are performed to enhance feature extraction to obtain the final valid feature layer; After the SegmentationHead layer, the number of channels of the obtained effective feature layer is adjusted to 3 categories.

8. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 6.

9. A computing device, characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in claims 1 to 6.

Images