A method, system, and device for generating fruit pattern codes based on fruit spot texture recognition.

Abstract

This invention discloses a method, system, and device for generating fruit pattern codes based on fruit spot texture recognition. The method includes: acquiring a top image of a fruit; segmenting the fruit image from the image using a convolutional neural network model and identifying all spots on it; generating an outer rectangle for the fruit image and using its center as the origin; calculating the center coordinates of the outer rectangle for each spot to form a coordinate array; calculating the distance between the center of the outer rectangle for each spot and the origin to form a distance array, and arranging the distance array and its corresponding coordinate array in ascending order; using the straight line between the first coordinate in the coordinate array and the origin as a guideline, calculating the angle between the line connecting the current coordinate and the origin and the guideline to form an angle array; desensitizing the distance array and the angle array; concatenating the desensitized distance array and the angle array and using a hash function to obtain their hash code. This invention realizes the electronic encoding of fruits with spots and patterns, providing a foundation for subsequently attaching commercial information about the fruit.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, system, and device for generating fruit pattern codes based on fruit spot texture recognition. Background Technology

[0002] As of 2022, my country's annual total fruit production exceeded 3 trillion yuan, and the total transaction volume exceeded one trillion yuan. However, under the current fruit trading model, offline transactions in fruit wholesale markets account for more than 90% of the total transaction volume. The traditional offline trading model centered around fruit wholesale markets is usually based on acquaintances and personal trust. In this model, fruit farmers are unaware of the actual process of the fruit they export to the trading market. This leads to varying degrees of passivity for fruit farmers when it comes to final transaction settlement and production planning, and their economic rights are easily damaged.

[0003] In the fruit retail sector, consumers often rely solely on their own experience to judge whether the fruit they are about to buy can meet their needs. However, due to factors such as information asymmetry and knowledge asymmetry, this experience-based judgment is often prone to errors and risks, and sometimes it may even lead consumers to buy fruit that is completely contrary to their consumption intentions.

[0004] Therefore, throughout the entire fruit distribution process, we need to ensure that accurate and reliable information about the fruit flows seamlessly with the actual fruit, thereby optimizing the entire process from production to end-consumer. However, this requires first identifying the fruit's identity, but currently, there is no technology that can identify fruit at a very low cost while simultaneously preventing information tampering. Therefore, if we could utilize the fruit's natural texture to create a unique code for its identity, we could then attach digital information such as the fruit's production time, origin, and specifications to this unique code. This would facilitate the coordinated flow of digital information about the fruit with the actual fruit, significantly advancing the development of the fruit production and sales industry. Summary of the Invention

[0005] This invention provides a method, system, and device for generating fruit pattern codes based on fruit spot texture recognition, enabling the identification of fruits while preventing information tampering. This provides a prerequisite for subsequently attaching more proprietary data and information about fruit objects to this fruit pattern code.

[0006] This invention is achieved through the following technical solution: a method for generating fruit pattern codes based on fruit spot texture recognition, comprising the following steps:

[0007] S1. Obtain an image of the top of the fruit using a camera;

[0008] S2. Use a depthwise separable convolutional neural network model to segment fruit images from an image;

[0009] S3. Using a depthwise separable convolutional neural network model, identify and extract all the blob patterns from the segmented fruit images;

[0010] S4. Generate an outer rectangle for the fruit object, and use the center of the outer rectangle as the origin O of the coordinate system.

[0011] S5. Calculate the coordinates of the center point of the bounding rectangle of each spot, forming a coordinate array P[] = {p1, p2, p3, ..., p i ,…,p n};

[0012] S6. Calculate the distance between the center point of the bounding rectangle of each spot and the origin, forming a distance array L[] = {l1, l2, l3, ..., l i ,…,l n Arrange the distance array and its corresponding coordinate array in ascending order of distance value to obtain P′[] and L′[];

[0013] S7. Using the straight line p′1O between the first coordinate point p′1 of the coordinate array P′[] and the origin O as the directrix, calculate the current coordinate point p′ one by one. i The angle θ between the line connecting the origin O and the directrix i , that is, θ i =p′ i Op′1, forming the included angle array θ[];

[0014] S8. Desensitize L′[] and θ[] to obtain L″[] and θ′[];

[0015] S9. Combine L″[] and θ′[] into a new array X[], and use the Hash function to find the hash code of X[], which is the fruit pattern code.

[0016] Step S8 describes the desensitization process for L′[] and θ[], which is performed as follows:

[0017]

[0018] Where f() is the desensitization function, int() is the floor function, dpi is the image resolution, x∈N is the scale parameter, the value of which is determined according to the identified fruit category, and S1∈N + S2∈N + The larger the desensitization step size, the lower the data resolution of the calculation results.

[0019] Furthermore, step S2, which involves segmenting the fruit object from the image, comprises the following steps:

[0020] The input image is processed into a multi-pixel matrix;

[0021] By using MobileNet to build a deep separable convolutional neural network model, the pixel matrix is ​​convolved to segment fruit objects from the image.

[0022] Furthermore, step S3, which utilizes a depthwise separable convolutional neural network model to identify and extract all blotches from the segmented fruit image, includes the following processing steps:

[0023] The fruit image is processed into a multi-pixel matrix;

[0024] By using MobileNet to build a deep separable convolutional neural network model, pixel matrix convolution is performed to segment blob objects from the image.

[0025] Furthermore, step S9, which involves combining L″[] and θ′[] into a new array X[], is processed as follows:

[0026] Convert the array elements in arrays L″[] and θ′[] to string elements;

[0027] Each element in the array X[] is assigned the value l″. i θ′ i , that is, X[]={l″1θ′1,l″2θ′2,…,l″ i θ′ i ,…l″ n θ′ n}

[0028] Furthermore, the hash function described in step S9 uses the SM3 cryptographic hash algorithm promulgated by the State Cryptography Administration in December 2010.

[0029] The present invention has the following advantages over the prior art:

[0030] 1. Compared with the traditional method of affixing paper labels to fruits, this invention can save high label costs and labor costs incurred due to label customization, procurement, management, and affixing in large-scale fruit production;

[0031] 2. The fruit pattern code is derived from the fruit itself, and the function used in the invention has good anti-tampering properties, which greatly reduces the risk of fruit products being swapped during circulation.

[0032] 3. Because this invention uses desensitization processing, the method has good stability and ease of use;

[0033] 4. Since most fruits produce a self-preserving wax that is sufficient to ensure the stability of their appearance during the circulation period, this invention has good physical stability.

[0034] 5. This invention makes it possible to attach more proprietary data and information about fruit objects to the fruit pattern code. Attached Figure Description

[0035] Figure 1 This is a flowchart of the steps of the fruit pattern code generation method based on fruit spot texture recognition in an embodiment of the present invention;

[0036] Figure 2 This is an orthophoto of the top of an apple in an embodiment of the present invention;

[0037] Figure 3 It is to segment the bounding rectangle of the fruit image from the image;

[0038] Figure 4 This is a schematic diagram of the rectangle surrounding the spots in a fruit image;

[0039] Figure 5 This is a schematic diagram of the origin of the sample spot coordinates, the coordinates of the first spot, and the directrix in an embodiment of the present invention.

[0040] Figure 6 This is a schematic diagram showing the coordinates, distance, and angle with the directrix of any spot on the sample in an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the sample feature array X[] in an embodiment of the present invention;

[0042] Figure 8 This is a system structure diagram in an embodiment of the present invention;

[0043] Figure 9 This is a structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 This embodiment discloses a method for generating fruit pattern codes based on fruit spot texture recognition. This method can quickly extract all spots in the top image of a fruit and convert them into distance arrays from the image origin and angle arrays with the guideline. These are then used as input to a hash function to obtain a fruit pattern code that uniquely identifies the fruit. The specific steps of this method include:

[0046] S101. Obtain an image of the top of the fruit using a camera, such as... Figure 2 As shown;

[0047] In this embodiment, the basic parameters of the image of the top of the apple obtained by the mobile phone camera are: image width 3024 pixels, image height 4032 pixels, and image resolution 72 dpi;

[0048] S102. Using a depthwise separable convolutional neural network model, segment fruit images from an image;

[0049] In this embodiment, the MobileNet architecture is used to construct a deep learning network that can be applied to both mobile and embedded devices to segment fruit objects from images. Its structure includes 9 pointwise convolutional layers, 8 depthwise convolutional layers, 1 connection layer, 1 pooling layer, 1 fully connected layer, and 1 filter. The processing includes the following steps:

[0050] The input image is processed into a pixel matrix (length × width × channel × number of blocks) with a size of 227×227×3×18 by padding the width and height with zeros.

[0051] Convolution processing involves using several matrices of preset sizes as kernels to convolve the pixel matrix or the input matrix from the previous layer to obtain feature vectors.

[0052] The ReLU6 function is used to operate on the feature vectors obtained from the convolution process to obtain the nonlinear enhancement matrix;

[0053] Where ReLU6 = min(max(0,x),6);

[0054] Pooling processing: Overlapping pooling is performed on the nonlinear enhancement matrix calculated by the ReLU6 function using preset pooling units to obtain the pooling matrix.

[0055] Normalization processing: The pooling matrix obtained by pooling is normalized using the local response normalization function, and the image matrix is ​​output.

[0056] It should be noted that not every layer needs to execute all the above processing steps. Some convolutional layers may only need to execute a portion of the above processing steps, depending on the requirements. This invention provides a specific embodiment for illustration. For example, the convolutional neural network model in this embodiment has 5 convolutional layers, namely:

[0057] Convolutional layer C1, the processing flow of this layer is: convolution --> ReLU6 --> pooling --> normalization;

[0058] Convolution: M1 is convolved using 96 matrices of size 11×11×3 as convolution kernels to obtain feature vector M2 of size 55×55×96;

[0059] ReLU6: Perform the ReLU6 function operation on the eigenvector M2 to obtain matrix M3;

[0060] Pooling: Overlap pooling is performed on matrix M3 using 3×3 pooling units with a stride of 2, outputting matrix M4 with a size of 27×27×96;

[0061] Normalization: Local response normalization is performed on M4 using a local response normalization function, outputting an image matrix M5 with a size of 27×27×96. M5 is then divided into two matrices, M5-1 and M5-2, each with a size of 27×27×48.

[0062] Convolutional layer C2, the processing flow of this layer is: convolution --> ReLU6 --> pooling --> normalization;

[0063] Convolution: Two sets of 128 matrices of size 5×5×48 are used as convolution kernels to convolve the two input matrices M5-1 and M5-2 from C1. The stride of the convolution is 1, resulting in two sets of feature vectors M6-1 and M6-2 of size 27×27×128.

[0064] ReLU6: Perform the ReLU6 function operation on the eigenvectors M6-1 and M6-2 to obtain matrices M7-1 and M7-2;

[0065] Pooling: Overlap pooling is performed on matrix M3 using 3×3 pooling units with a stride of 2, outputting matrices M8-1 and M8-2 with a size of 13×13×256;

[0066] Normalization: Use the same local response normalization function with the same parameters as C1 layer to perform local response normalization on M9-1 and M9-2, and output two image matrices M10-1 and M10-2 with a size of 13×13×128.

[0067] Convolutional layer C3, the processing flow of this layer is: convolution --> ReLU6;

[0068] Convolution: Two sets of 192 matrices of size 3×3×128 are used as convolution kernels to convolve the two input matrices M10-1 and M10-2 from C2. The stride of the convolution is 1, resulting in two sets of feature vectors M11-1 and M11-2 of size 13×13×192.

[0069] ReLU6: Perform the ReLU6 function operation on the eigenvectors M11-1 and M11-2 to obtain matrices M12-1 and M12-2;

[0070] Convolutional layer C4, the processing flow of this layer is: convolution --> ReLU6;

[0071] Convolution: Two sets of 3×3×192 matrices are used as convolution kernels to convolve the two input matrices M12-1 and M12-2 from C3. The stride of the convolution is 1, resulting in two sets of feature vectors M13-1 and M13-2 with a size of 13×13×192.

[0072] ReLU6: Perform the ReLU6 function operation on the eigenvectors M13-1 and M13-2 to obtain matrices M14-1 and M14-2;

[0073] Convolutional layer C5, the processing flow of this layer is: convolution --> ReLU6 --> pooling;

[0074] Convolution: Two sets of 128 matrices of size 3×3×192 are used as convolution kernels to convolve the two input matrices M14-1 and M14-2 from C4. The stride of the convolution is 1, resulting in two sets of feature vectors M15-1 and M15-2 of size 13×13×128.

[0075] ReLU6: Perform the ReLU6 function operation on the eigenvectors M15-1 and M15-2 to obtain matrices M16-1 and M16-2;

[0076] Pooling: Overlap pooling is performed on matrices M16-1 and M16-2 using 3×3 pooling units with a stride of 2, resulting in a matrix M17 with a size of 6×6×256.

[0077] Specifically, the processing steps performed by the fully connected layer include:

[0078] The input matrix from the convolutional layer is convolved to obtain the neuron matrix; in order to increase the function fitting ability of the neuron matrix, it is also necessary to perform ReLU6 function operation on it.

[0079] The purpose of using the Dropout function to disable some neurons in the neuron matrix is ​​to prevent overfitting. The Dropout function randomly assigns zero values ​​to some neurons in the neuron matrix, thus making those neurons ineffective.

[0080] This invention provides a specific embodiment for illustration. For example, the convolutional neural network model in this embodiment has three fully connected layers, one of which is the final output layer:

[0081] The fully connected layer FC6 has the following processing flow: (convolution) fully connected --> ReLU6 --> Dropout;

[0082] Convolution -> Fully Connected: Using 4096 matrices of size 6×6×256 as convolution kernels, the input matrix M17 from C5 is convolved to obtain 4096 1×1 pixels, i.e. 4096 neurons [M18-1, M18-2, M18-3, ..., M18-4096];

[0083] ReLU6: Perform the ReLU6 function operation on [M18-1,M18-2,M18-3,...,M18-4096] to obtain neurons [M19-1,M19-2,M19-3,...,M19-4096];

[0084] Dropout: To prevent overfitting, a Dropout operation is performed on M19, that is, the Dropout function is used to set 0 to 10% of the members of matrix M19 (neurons are disabled), resulting in 4096 neurons M20. In this model, P is set to 50%;

[0085] The fully connected layer FC7 has the following processing flow: fully connected --> ReLU6 --> Dropout;

[0086] Convolution -> Fully Connected: Use a 1×1×4096 matrix as the convolution kernel to convolve the input matrix M20 from FC6 to obtain a 1×1×4096 neuron matrix M21;

[0087] ReLU6: Perform the ReLU6 function operation on M21 to obtain matrix M22;

[0088] Dropout: To prevent overfitting, a dropout operation is performed on M22, that is, the dropout function is used to set 0 to 50% of the members of matrix M22 (neurons are disabled), resulting in matrix M23. In this model, P is set to 50%.

[0089] The connection layer (output layer) FC8 processes the following steps: 1000 neurons are used to fully connect M23, and M23 is filtered through a Gaussian filter to obtain 1000 float-type prediction probabilities.

[0090] Preferably, the ReLU6 function is a piecewise linear function, and the output of the ReLU6 function is 0 when the output result is less than or equal to 0, and is equal to 0 when it is greater than 0.

[0091] Preferably, the expression for the local response normalization function is as follows:

[0092]

[0093] The local response normalization function parameters k, α, β, and n used are 2, 5, 10⁻⁴, and 0.75, respectively.

[0094] Preferably, the Dropout function is a random function that randomly selects neurons from the neuron matrix that meet a specified ratio according to the parameters and sets them to 0 (invalid). In this model, the invalidation ratio is 50%.

[0095] S103. Using a depthwise separable convolutional neural network model, identify and extract all the blob patterns from the segmented fruit images;

[0096] S104, please refer to Figure 3 Generate an outer rectangle for the fruit image, and use the center of the outer rectangle as the origin O of the coordinate system.

[0097] S105. Calculate the coordinates of the center point of the bounding rectangle of each spot, forming a coordinate array P[] = {p1, p2, p3, ..., p i ,…,p n};

[0098] S106. Calculate the distance between the center point of the bounding rectangle of each spot and the origin, forming a distance array L[] = {l1, l2, l3, ..., l 89 Arrange the distance array and its corresponding coordinate array in ascending order of distance value to obtain P′[] and L′[];

[0099] S107. Using the straight line p′1O between the first coordinate point p′1 of the coordinate array P′[] and the origin O as the directrix, calculate the current coordinate point p′ one by one. i The angle θ between the line connecting the origin O and the directrix i , that is, θ i =p′ i Op′1, forming the included angle array θ[];

[0100] S108. Desensitize L′[16,16.8,17.5,…,57] and θ[0,340,33,…,33] to obtain L″[16,16,17,…,57] and θ′[0,340,32,…,32].

[0101] In this embodiment, the following algorithm is used to implement the desensitization process:

[0102]

[0103] Where f() is the desensitization function, int() is the floor function, dpi = 72 is the image resolution, since the recognized image is an apple, the scale parameter x is set to 1, and S1 and S2 are the desensitization step sizes, which are set to 1 and 2 respectively.

[0104] S109. Combine L″[] and θ′[] into a new array X["160","16340","1732",...,"5732"], and use the hash function SM3 to find the hash code of X[], i.e., the hash code:

[0105] SM3(X[])=

[0106] "DBB68837C8FCF918AB49AE2BEBFFB69A54A98E7BE003FFD29DBA2D188149B7F6".

[0107] In this embodiment, the computing device can be a desktop computer, a laptop computer, a PDA handheld terminal, a tablet computer, or other terminal devices.

[0108] The above description represents the preferred embodiments of the present invention. However, the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for generating fruit pattern codes based on fruit spot texture recognition, characterized in that, Includes the following steps: S1. Obtain an image of the top of the fruit using a camera; S2. Use a depthwise separable convolutional neural network model to segment fruit images from an image; S3. Using a depthwise separable convolutional neural network model, identify and extract all the blob patterns from the segmented fruit images; S4. Generate an outer rectangle for the fruit image, and use the center of the outer rectangle as the origin O of the coordinate system. S5. Calculate the coordinates of the center point of the bounding rectangle of each spot, forming a coordinate array P[] = {p1, p2, p3, ..., p i ,…,p n }; S6. Calculate the distance between the center point of the bounding rectangle of each spot and the origin, forming a distance array L[] = {l1, l2, l3, ..., l i ,…,l n Arrange the distance array and its corresponding coordinate array in ascending order of distance value to obtain P′[] and L′[]; S7. Using the straight line p′1O between the first coordinate point p′1 of the coordinate array P′[] and the origin O as the directrix, calculate the current coordinate point p′ one by one. i The angle θ between the line connecting the origin O and the directrix i , that is, θ i =p′ i Op′1, forming the included angle array θ[]; S8. Desensitize L′[] and θ[] to obtain L″[] and θ′[]; S9. Combine L″[] and θ′[] into a new array X[], and use the Hash function to find the hash code of X[], which is the fruit pattern code; In step S8, L′[] and θ[] are desensitized, and the specific calculation method is as follows: Where f() is the desensitization function, int() is the floor function, dpi is the image resolution, x∈N is the scale parameter, the value of which is determined according to the identified fruit category, and S1∈N + S2∈N + The larger the desensitization step size, the lower the data resolution of the calculation results.

2. The fruit pattern code generation method based on fruit spot texture recognition as described in claim 1, characterized in that, Step S2, which uses a depthwise separable convolutional neural network model to segment fruit objects from an image, includes the following processing steps: The input image is processed into a multi-pixel matrix; By using MobileNet to build a deep separable convolutional neural network model, the pixel matrix is ​​convolved to segment fruit objects from the image.

3. The fruit pattern code generation method based on fruit spot texture recognition as described in claim 1, characterized in that, Step S3 describes using a depthwise separable convolutional neural network model to identify and extract all blotches from the segmented fruit image, including the following processing steps: The fruit image is processed into a multi-pixel matrix; By using MobileNet to build a deep separable convolutional neural network model, pixel matrix convolution is performed to segment blob objects from the image.

4. The fruit pattern code generation method based on fruit spot texture recognition as described in claim 1, characterized in that, Step S9 involves combining L″[] and θ′[] into a new array X[], and the specific method is as follows: Convert the array elements in arrays L″[] and θ′[] to string elements; Each element in the array X[] is assigned the value l″. i θ′ i , that is, X[]={l″1θ′1,l″2θ′2,…,l″ i θ′ i ,…l″ n θ′ n } 5. The fruit pattern code generation method based on fruit spot texture recognition as described in claim 1, characterized in that, The hash function described in step S9 uses the SM3 cryptographic hash algorithm promulgated by the State Cryptography Administration in December 2010.

6. A fruit pattern code generation system based on fruit spot texture recognition, characterized in that, The system includes: The acquisition module is used to acquire the top image of the fruit; A depthwise separable convolutional neural network module is used to segment fruit images from an image and to identify and extract all blob patterns from the segmented fruit images; The calculation module is used to calculate the generated fingerprint code. Generate an outer rectangle for the fruit image, and use the center of this outer rectangle as the origin O; Calculate the coordinates of the center point of the bounding rectangle of each spot, forming a coordinate array P[] = {p1, p2, p3, ..., p i ,…,p n }; Calculate the distance between the center point of the bounding rectangle of each spot and the origin, forming a distance array L[] = {l1, l2, l3, ..., l i ,…,l n Arrange the distance array and its corresponding coordinate array in ascending order of distance value to obtain P′[] and L′[]; Using the straight line p′1O between the first coordinate point p′1 in the coordinate array P′[] and the origin O as the directrix, calculate the current coordinate point p′ one by one. i The angle θ between the line connecting the origin O and the directrix i , that is, θ i =p′ i Op′1, forming the included angle array θ[]; Desensitizing L′[] and θ[] yields L″[] and θ′[], respectively. The specific calculation method for the desensitization is as follows: Where f() is the desensitization function, int() is the floor function, dpi is the image resolution, x∈N is the scale parameter, the value of which is determined according to the identified fruit category, and S1∈N + S2∈N + The larger the desensitization step size, the lower the data resolution of the calculation results; Combine L″[] and θ′[] into a new array X[], and use a hash function to find the hash code of X[], which is the fruit pattern code.

7. A fruit pattern code generation device based on fruit spot texture recognition, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the fruit pattern code generation method based on fruit spot texture recognition as described in any one of claims 1 to 5.

Images