Automated cabbage core removal system

Abstract

The present invention relates to an automated system for removing the core of a cabbage. The automated system for removing the core of a cabbage according to the present invention extracts the cross-sectional angle of the cabbage, the location of the core, and the depth of the core based on an image of the cabbage captured by a capturing means.

Description

Technology Field

[0001] The present invention relates to an automated system for removing the core of a cabbage. Background Technology

[0003] Kimchi is a major food source supplying vitamins and minerals to Koreans, serving as a guardian that prevents obesity and protects health in an era of calorie excess. Furthermore, as the final form of consumption for key vegetables such as napa cabbage, chili peppers, and garlic, the supply and demand of these vegetables are significantly influenced by kimchi consumption.

[0004] The reason why domestic kimchi is about twice as expensive as Chinese kimchi is that, in addition to the unit cost of materials, labor costs are high. Labor costs in kimchi factories account for 15-30% of manufacturing costs, which is very high compared to the average labor cost ratio of 7% in the food manufacturing industry, because the manufacturing process is carried out manually. Therefore, although technology to automate the kimchi manufacturing process is continuously being developed, there are still many limitations to achieving perfect automation.

[0005] In particular, while technologies attempting image analysis through camera capture are being developed to remove the cabbage core, conventional technology photographs the entire cabbage and estimates the location of the core based on its center relative to its outline. Consequently, there is a disadvantage in that the location of the core cannot be accurately identified if it is not actually situated at the center of the cabbage. Furthermore, conventional technology merely estimates the location of the core and fails to determine its depth at all. Therefore, there is a drawback in that it is difficult to accurately remove the core, which is a factor that directly affects the quality of kimchi. Prior art literature

[0007] (Patent Document 0001) KR 10-2022-0036376 A The problem to be solved

[0008] The present invention aims to solve the problems of the aforementioned prior art. One aspect of the present invention relates to an automated system for removing cabbage cores that can accurately remove the cabbage core by extracting the cross-sectional angle of the cabbage, the location of the cabbage core, and the depth of the cabbage core based on an image of the cabbage captured by a capturing means. means of solving the problem

[0010] An automated system for removing the cabbage core according to an embodiment of the present invention extracts the cross-sectional angle of the cabbage, the location of the cabbage core, and the depth of the cabbage core based on an image of the cabbage captured by a capturing means.

[0011] In addition, in the automated system for removing cabbage cores according to an embodiment of the present invention, the focal length calculated through parameter estimation is used as the distance value between the cabbage core removal means and the cabbage core.

[0012] In addition, in the automated system for removing the cabbage core according to an embodiment of the present invention, the rotation matrix calculated through parameter estimation is used as an angle value between the cabbage core removal means and the cabbage core.

[0013] In addition, in an automated system for removing a cabbage core according to an embodiment of the present invention, the imaging means photographs the cabbage in a direction parallel to the cabbage core, photographs a cross-section of the cabbage cut in a direction parallel to the cabbage core, and learns a transfer learning-based model based on the image captured by the imaging means.

[0014] In addition, in the automated system for removing cabbage core according to an embodiment of the present invention, at least one of the position, depth, and angle of the cabbage core is labeled based on an image captured by the capturing means.

[0015] In addition, in an automated system for removing the cabbage core according to an embodiment of the present invention, at least one of the area of ​​the cabbage core and the maximum depth of the cabbage core is labeled based on an image of a cross-section of the cabbage captured by the capturing means.

[0016] In addition, in the automated system for removing cabbage cores according to an embodiment of the present invention, when labeling the location of the cabbage core, the area of ​​the cabbage core is extracted as a rectangle based on an image of the cabbage taken parallel to the cabbage core by the capturing means, and the center point of the cabbage core is extracted as X,Y coordinate values.

[0017] In addition, in the automated system for removing the cabbage core according to an embodiment of the present invention, the transfer learning-based model includes a source model utilizing an image captured in a parallel direction of the cabbage core and a target model utilizing an image captured in a cross-section of the cabbage.

[0018] In addition, in the automated system for removing the core of a cabbage according to an embodiment of the present invention, when transitioning from the source model to the target model, the weight values ​​and bias values ​​of a specific convolution layer are transferred or frozen from the source model to the target model.

[0019] In addition, in the automated system for removing cabbage cores according to an embodiment of the present invention, the depth of the cabbage core is extracted based on the data learned from the transfer learning-based model when removing the cabbage core, using only images taken in the direction parallel to the cabbage core.

[0021] The features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings.

[0022] Prior to this, terms and words used in this specification and claims shall not be interpreted in their ordinary and dictionary meanings, but must be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor may appropriately define the concept of the terms to best describe his invention. Effects of the invention

[0024] According to the present invention, the cross-sectional angle of the cabbage, the location of the cabbage core, and the depth of the cabbage core are extracted based on an image of the cabbage captured by a capturing means, thereby providing the advantage of accurately removing the cabbage core. Brief explanation of the drawing

[0026] FIG. 1 is a flowchart of an automated system for removing cabbage cores according to an embodiment of the present invention. FIG. 2 is a conceptual diagram of an automated cabbage core removal system according to an embodiment of the present invention. As shown in FIG. 3, the automated cabbage core removal system is a conceptual diagram illustrating a robot camera (shooting means), a calibration board, markers, etc. Figure 4 is a photograph of a fixing frame (arm box) for fixing cabbages. Fig. 5 is a photograph of a cabbage inside a fixed frame, Figure 6 is a photograph illustrating the process of labeling the location of the cabbage core. Figure 7 is a photograph illustrating the process of labeling the area of ​​the cabbage core and the maximum depth of the cabbage core. Figure 8 is a flowchart of a model for extracting the location and depth of a cabbage core, Figure 9 is a flowchart of a transfer learning-based model, and Figure 10 is a flowchart illustrating the transfer process in a transfer learning-based model. Specific details for implementing the invention

[0027] The object, specific advantages, and novel features of the present invention will become more apparent from the following detailed description and preferred embodiments in conjunction with the accompanying drawings. It should be noted that in assigning reference numbers to the components of each drawing in this specification, identical components are assigned the same number whenever possible, even if they are shown in different drawings. Furthermore, terms such as "first," "second," etc., are used to distinguish one component from another, and the components are not limited by these terms. In the following description of the present invention, detailed descriptions of related prior art that could unnecessarily obscure the essence of the invention are omitted.

[0028] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0030] FIG. 1 is a flowchart of an automated system for removing cabbage cores according to an embodiment of the present invention. As shown in FIG. 1, the automated system for removing cabbage cores according to the present embodiment extracts the cross-sectional angle of the cabbage, the location of the cabbage core, and the depth of the cabbage core software-wise in a processing unit based on an image of the cabbage captured by a capturing means.

[0032] Through the automated cabbage core removal system according to the present embodiment, the cross-sectional angle of the cabbage, the location of the cabbage core, and the depth of the cabbage core extracted by software in the computation unit are transmitted to the cabbage core removal means (a robot including a blade) shown in FIG. 2, so that the cabbage core removal means can accurately remove the cabbage core.

[0034] As illustrated in FIG. 3, the automated system for removing the cabbage core extracts intrinsic parameter values ​​of the camera using a robot camera (shooting means), a calibration board, markers, etc. For example, intrinsic parameter values ​​such as focal length (x), focal length (y), scale factor, and asymmetry factor can be extracted. Additionally, extrinsic parameter result values ​​are extracted. For example, a rotation matrix and a translation matrix can be extracted as extrinsic parameter result values. Here, the focal length can be used as the distance value between the cabbage core removal means (robot blade) and the cabbage core position. Additionally, the rotation matrix can be used as the angle value between the cabbage core removal means (robot blade) and the cabbage core. The angle value is input as the value of the robot's last axis (the insertion angle of the robot blade relative to the cabbage core) and can be used as a basis for removing the actual cabbage core from the center point of the cabbage core position extracted by the deep learning model.

[0036] Meanwhile, the imaging means (robot camera) can photograph the cabbage in a direction parallel to the cabbage core (photographed in the XY plane) and photograph the cross-section (cut surface) of the cabbage cut in a direction parallel to the cabbage core (photographed in the XZ plane). For example, as shown in FIGS. 2 and FIGS. 4 to 5, the cabbage can be fixed inside a fixed frame (arm box) and the cabbage can be photographed in two directions using two imaging means (robot cameras).

[0037] As described above, based on an image of the cabbage captured by a photographing means, at least one of the position, depth, and angle of the cabbage core is labeled. Preferably, the position, depth, and angle of the cabbage core can all be labeled. Specifically, as shown in FIG. 6, when labeling the position of the cabbage core, the area of ​​the cabbage core is extracted as a rectangle based on an image of the cabbage captured parallel to the cabbage core by a photographing means, and the center point of the cabbage core is extracted as an XY coordinate value. The extracted XY coordinate value can be used as a value to set the position of the axis of the cabbage core removal means (the blade of the robot). In addition, as shown in FIG. 7, based on an image of the cross-section (cut surface) of the cabbage captured by a photographing means, at least one of the area of ​​the cabbage core and the maximum depth of the cabbage core can be labeled. Preferably, both the area of ​​the cabbage core and the maximum depth of the cabbage core can be labeled. In Figure 7, the blue dots are the result of labeling the area of ​​the cabbage core, and the brown lines are the result of labeling the maximum depth of the cabbage core.

[0038] Overall, as illustrated in FIG. 8, the model for extracting the location and depth of the cabbage core consists of a single model, and based on an image of the cabbage, the single model is divided into two sub-models, and may include a step of converting the size of the original image of the cabbage before being divided into two sub-models.

[0039] Specifically, the size of the image of the cabbage is 3000*2000, and a step of removing the fixed frame and background from this image, followed by a step of resizing the image to 256*256 based on the center of the labeled cabbage core, can be performed. The resized image can be input into a model that extracts the location and depth of the cabbage core. Cubic interpolation techniques can be applied to minimize image information loss during the resizing step.

[0040] Here, the model for extracting the location and depth of the cabbage core may be a transfer learning-based model (a cabbage core region recognition model based on the U-Net source model).

[0041] As illustrated in FIG. 9, the transfer learning-based model may include a source model utilizing XY-plane images and a target model utilizing XZ-plane images. In this case, the basic structure of the two models is identical. As illustrated in FIG. 10, regarding the process of transferring from the source model to the target model, the weight and bias values ​​of a specific convolution layer can be transferred or frozen from the source model to the target model. For example, if the optimization result appears as "O", both the weight and bias values ​​are transferred; if it appears as "△", only the weight values ​​are transferred; and if it appears as "X", all values ​​are frozen.

[0042] Specifically, the convolution layer is a core layer of deep learning models such as the U-Net source model or a Convolutional Neural Network (CNN), designed to extract features from input data. It undergoes computational processes such as convolution, pooling, and non-linear activation functions. It performs convolution operations between input data (XY-plane image data and XZ-plane image data) and filters (kernels) using filters. The convolution layer can extract spatial correlations of data by performing convolution operations while sliding across the entire input data using filters of a specific size in a 2D form. Input data undergoes computations by passing through convolution, pooling, and non-linear activation functions (e.g., ReLU), respectively. Convolution layers include not only the simple input volume but also variables such as weights, biases, depth, stride, and padding. A key characteristic of convolution layers is the sharing of weights for the extracted feature maps. The weight values ​​of a convolution layer are proportional to [squared filter size × number of kernels × number of channels], while the bias value is proportional to the number of kernels, as each kernel has a single value. Training a Convolutional Neural Network (CNN) means constructing the CNN's free parameters (weights) based on the backpropagation algorithm, which is an optimization process for deep learning frameworks. In other words, it is equivalent to adjusting the values ​​of the extracted weight matrix.It involves taking the dot product of the extracted feature map matrix and weight matrix values ​​and adding the bias matrix to extract a new learned feature map matrix.

[0043] Based on the data learned from the aforementioned transfer learning-based model, the operation unit extracts not only the location of the cabbage core but also the depth of the cabbage core using only the XY plane image data used as the source model when actually removing the cabbage core.

[0045] Meanwhile, optimization is applied during the transition from the source model to the target model, and the fitness function of the optimization can be expressed by the following equation.

[0046] f(x) = (length of original cabbage core depth - length of predicted cabbage core depth) + distance between two dimensions (feature of XY plane image and feature of XZ plane image)

[0047] Here, the length of the original cabbage core depth refers to the actual depth of the cabbage core, and the length of the predicted cabbage core depth refers to the predicted depth. Additionally, during the transfer process, the difference between the feature map values ​​of the source model and the target model is calculated as the distance between the two feature maps using the cosine distance formula. Consequently, the goodness-of-fit function of the optimization aims to minimize the final loss value by adding the distance between the two feature maps to (actual core depth minus predicted core depth).

[0048] The constraint for this optimization is to train (non-transfer) at least one convolution layer. In the process of selecting the convolution layer to be transferred to minimize the value of the above fit function, the optimization decision variable is the number of convolution layers. For example, assuming there are 10 convolution layers in a transfer learning-based model, if the decision variable value of the convolution layer to be transferred is 0, the weight and bias values ​​are not transferred; if the decision variable value is 1, only the weight values ​​are transferred; and if the decision variable value is 2, both the weight and bias values ​​are transferred.

[0050] Although the present invention has been described in detail through specific embodiments, this is for the purpose of specifically explaining the invention, and the invention is not limited thereto. It is evident that modifications or improvements can be made by those skilled in the art within the technical scope of the present invention.

[0052] All simple variations or modifications of the present invention fall within the scope of the present invention, and the specific scope of protection of the present invention will be clarified by the appended claims.

Claims

Claim 1 An automated system for removing cabbage cores, comprising: extracting the cross-sectional angle of the cabbage, the location of the cabbage core, and the depth of the cabbage core based on an image of the cabbage captured by a capturing means; wherein the capturing means captures the cabbage in a direction parallel to the cabbage core and captures the cross-section of the cabbage cut in a direction parallel to the cabbage core; and, based on the image captured by the capturing means, training a transfer learning-based model; wherein the transfer learning-based model includes a source model utilizing an image captured in a direction parallel to the cabbage core and a target model utilizing an image captured in a cross-section of the cabbage. Claim 2 A cabbage core removal automation system according to claim 1, wherein the focal length calculated through parameter estimation is used as the distance value between the cabbage core removal means and the cabbage core. Claim 3 In claim 1, the rotation matrix calculated through parameter estimation is used as an angle value between the cabbage core removal means and the cabbage core in an automated cabbage core removal system. Claim 4 delete Claim 5 An automated system for removing a cabbage core according to claim 1, which labels at least one of the position, depth, and angle of the cabbage core based on an image captured by the capturing means. Claim 6 An automated system for removing a cabbage core according to claim 1, which labels at least one of the area of ​​the cabbage core and the maximum depth of the cabbage core based on an image of a cross-section of the cabbage taken by the imaging means. Claim 7 An automated system for removing a cabbage core according to claim 5, wherein when labeling the location of the cabbage core, the area of ​​the cabbage core is extracted as a rectangle based on an image of the cabbage taken parallel to the cabbage core by the shooting means, and the center point of the cabbage core is extracted as X,Y coordinate values. Claim 8 delete Claim 9 An automated system for removing cabbage core according to claim 1, wherein when transferring from the source model to the target model, the weight values ​​and bias values ​​of a specific convolution layer are transferred or frozen from the source model to the target model. Claim 10 An automated system for removing a cabbage core according to claim 1, which extracts the depth of the cabbage core using only images taken in a direction parallel to the cabbage core when removing the cabbage core, based on data learned from the transfer learning-based model.

Images