Fruit and vegetable picking robot control system

Abstract

The invention discloses a fruit and vegetable picking robot control system, which relates to the field of fruit and vegetable picking and comprises a front-end acquisition device, a data integration module and a processing control center. A high-definition camera and a force sensor in the front-end acquisition device transmit acquired fruit and vegetable data to the data integration module; the data integration module integrates the fruit and vegetable data to obtain a fruit and vegetable data set U and transmits the fruit and vegetable data set U to the processing control center; and the processing control center performs integrated analysis calculation on the obtained fruit and vegetable data set U, calculates the ripeness coefficient and the quality coefficient of the fruits and vegetables, controls a mechanical arm of the fruit and vegetable picking robot based on the ripeness coefficient and the quality coefficient of the fruits and vegetables, and sorts the picked fruits and vegetables to a predetermined storage position. Accurate and effective data support can be provided for subsequent fruit and vegetable sorting according to the ripeness and quality, so that the sorting quality of picked fruits and vegetables is improved, secondary sorting of the fruits and vegetables by machines or manual work is not needed subsequently, and the process cost is saved.

Description

Technical Field

[0001] The present invention relates to the field of fruit and vegetable robots, and in particular to a fruit and vegetable picking robot control system. Background Art

[0002] As is known to all, with the development of society, these problems have been effectively solved. Digital agriculture and smart agriculture have become inevitable paths, and integrating information technology into agricultural production has become a major research topic. The implementation of the manufacturing power strategy has led to the rapid development of intelligent agricultural machinery and equipment. As part of intelligent agricultural machinery and equipment, agricultural robots have received widespread attention and have a bright future. Agricultural robots are a type of robot with the characteristics of service robots. They have the ability to perceive environmental information, understand the scene, and achieve precise control. They are used in agricultural application scenarios.

[0003] For example, the authorization announcement number is CN106105566B, the authorization announcement date is 2018.07.06, and the name is "An intelligent citrus picking robot and citrus picking method". The robot includes a walking mechanism, a robotic arm, a workbench, a recovery device, a control system and a power supply system. The robotic arm, the control system and the power supply system are installed on the workbench. The front end of the robotic arm is provided with an end effector and a visual system. The visual system is data-connected with the control system. The control system controls the movement of the robotic arm through the robotic arm drive unit. The control system controls the movement of the walking mechanism through the walking drive unit. The power supply system provides working power for the control system and the walking mechanism. The robotic arm includes a first arm, a second arm, a third arm, a fourth arm, a fifth arm and a sixth arm that are movably connected in sequence...

[0004] As in the above-mentioned application, most existing picking robots are lacking in the control of sorting fruits and vegetables. In some fruit and vegetable sorting control systems, the sorting and judgment of fruits and vegetables are merely simple image acquisition and recognition of fruits and vegetables, and the ripeness of fruits and vegetables is simply judged by color, and then simple sorting is performed based on the ripeness. After picking and sorting, they still need to be sorted a second time by manual or machine before they can be packaged and sold. There are many processes, the cost is increased, and the sorting is not accurate enough. Summary of the Invention

[0005] (1) Purpose of the invention

[0006] In view of this, the purpose of the present invention is to propose a fruit and vegetable picking robot control system, which can directly and accurately sort fruits and vegetables during the picking process, thereby improving the sorting quality of picked fruits and vegetables, and no longer requiring machines or manual labor to perform secondary sorting of fruits and vegetables, saving process costs.

[0007] (2) Technical solution

[0008] To achieve the above technical objectives, the present invention provides a fruit and vegetable picking robot control system, which is used to control the fruit and vegetable picking robot to sort fruits and vegetables according to their characteristics and place them in predetermined storage locations. The control system includes a front-end acquisition device, a data integration module, and a processing control center:

[0009] The front-end acquisition device includes a high-definition camera and multiple force sensors. The high-definition camera is installed on the fruit and vegetable picking robot, and the multiple force sensors are evenly distributed on the inner side of the gripper at the front end of the fruit and vegetable picking robot's mechanical arm;

[0010] The high-definition camera and various force sensors transmit the collected fruit and vegetable data to the data integration module, which integrates the fruit and vegetable data to obtain a fruit and vegetable data set U and transmits it to the processing and control center;

[0011] The processing control center performs integrated analysis and calculation on the acquired fruit and vegetable dataset U, calculates the ripeness coefficient and quality coefficient of the fruits and vegetables, and controls the robotic arm of the fruit and vegetable picking robot based on the ripeness coefficient and quality coefficient of the fruits and vegetables to sort the picked fruits and vegetables to a predetermined storage location.

[0012] As a further description of the above technical solution: the fruit and vegetable dataset U∈{A1(X1, Y1), A2(X2, Y2), A3(X3, Y3), ...A n (X n ,Y n )}, where A n Indicates the data of the nth collection location of fruits and vegetables; A n (X n ,Y n ), where X n Y represents the image of the nth acquisition position on the surface of fruits and vegetables captured by the high-definition camera, n Indicates the hardness data of the nth sampling position on the surface of fruits and vegetables collected by the force sensor.

[0013] As a further description of the above technical solution: the processing control center performs integrated analysis and calculation on the acquired fruit and vegetable data set, and the calculation of the ripeness of the fruits and vegetables specifically includes the following steps:

[0014] Identify the fruit and vegetable data collected at each location in the fruit and vegetable dataset U and extract the image and hardness data collected at each location;

[0015] Input the extracted images one by one into the fruit and vegetable color machine learning model to calculate the color values ​​of the fruits and vegetables in the fruit and vegetable images;

[0016] The color values ​​of each position of fruits and vegetables are integrated into the fruit and vegetable color value dataset P∈{b1, b2, b3, ...b n}, where b nRepresents the color value of the nth sampling position on the surface of fruits and vegetables, and calculates the color parameters of fruits and vegetables. The specific method is as follows:

[0017]

[0018] Among them, YS is the color parameter of fruits and vegetables, b max is the maximum value in the fruit and vegetable color value dataset P, b min is the minimum value in the fruit and vegetable color value dataset P;

[0019] The hardness data of fruits and vegetables at various positions are integrated into the fruit and vegetable hardness dataset K∈{c1, c2, c3, …c n}, where c n Represents the hardness data of the nth sampling position on the surface of fruits and vegetables, and calculates the hardness parameters of fruits and vegetables. The specific method is as follows:

[0020]

[0021] Among them, YD is the hardness parameter of fruits and vegetables, c max is the maximum value in the fruit and vegetable hardness dataset K, c min is the minimum value in the fruit and vegetable hardness dataset K;

[0022] The ripeness coefficient of fruits and vegetables is calculated based on their color and hardness parameters.

[0023] As a further description of the above technical solution: the training method of the fruit and vegetable color machine learning model is as follows:

[0024] Historical images of fruits and vegetables are collected and labeled, wherein the labels are color parameters of the fruits and vegetables. The fruit and vegetable images and the labels corresponding to the fruit and vegetable images constitute a set of training data, X sets of training data constitute a sample set, where X is an integer greater than 1, and the sample set is divided into a training set and a test set. The fruit and vegetable images in the training set are used as input of a fruit and vegetable color machine learning model, and the labels in the training set are used as output of the fruit and vegetable color machine learning model. The fruit and vegetable color machine learning model is trained to obtain a fruit and vegetable color judgment model, wherein minimizing the sum of prediction errors is the training objective. The fruit and vegetable color judgment model is evaluated using the test set, and the fruit and vegetable color judgment model when the sum of prediction errors reaches convergence is used as the constructed fruit and vegetable color machine learning model.

[0025] As a further description of the above technical solution: Based on the color parameters and hardness parameters of fruits and vegetables, the method for calculating the ripeness coefficient of fruits and vegetables is as follows:

[0026]

[0027] Among them, SD is the maturity coefficient of fruits and vegetables, YS is the color parameter of fruits and vegetables, and YD is the hardness parameter of fruits and vegetables. is the weight factor, Both are greater than 0.

[0028] As a further description of the above technical solution: the processing control center performs integrated analysis and calculation on the acquired fruit and vegetable data set, and calculates the quality coefficient of the fruits and vegetables, specifically including the following steps:

[0029] Identify the fruit and vegetable data collected at each location in the fruit and vegetable dataset U and extract the image and hardness data collected at each location;

[0030] The extracted images are input one by one into the machine learning model of fruit and vegetable representation difference parameters to calculate the representation difference parameters of fruits and vegetables in the fruit and vegetable images;

[0031] The hardness of standard quality fruits and vegetables is set as c b , the hardness data of fruits and vegetables at various positions are integrated into the fruit and vegetable hardness dataset K∈{c1, c2, c3, ...c n}, where c n represents the hardness data of the nth sampling position on the surface of fruits and vegetables. The calculation method of the hardness difference parameter of fruits and vegetables is as follows:

[0032]

[0033] Among them, YDC is the hardness difference parameter of fruits and vegetables;

[0034] The quality coefficient of fruits and vegetables is calculated based on the characterization difference parameters and hardness difference parameters of fruits and vegetables.

[0035] As a further description of the above technical solution: the training method of the fruit and vegetable difference machine learning model is as follows:

[0036] Historical images of fruits and vegetables are collected, and labels are set for the fruit and vegetable images, wherein the labels are representation difference parameters of the fruits and vegetables. The fruit and vegetable images and the labels corresponding to the fruit and vegetable images constitute a set of training data, and X groups of training data constitute a sample set, where X is an integer greater than 1. The sample set is divided into a training set and a test set. The fruit and vegetable images in the training set are used as input to a fruit and vegetable difference machine learning model, and the labels in the training set are used as output of the fruit and vegetable difference machine learning model. The fruit and vegetable difference machine learning model is trained to obtain a fruit and vegetable representation difference parameter calculation model, and minimizing the sum of prediction errors is used as the training goal. The fruit and vegetable representation difference parameter calculation model is evaluated using the test set, and the fruit and vegetable representation difference parameter calculation model when the sum of prediction errors reaches convergence is used as the constructed fruit and vegetable representation difference parameter machine learning model.

[0037] As a further description of the above technical solution: the processing control center calculates the fruit and vegetable quality coefficient based on the fruit and vegetable characterization difference parameter and the hardness difference parameter as follows:

[0038]

[0039] Among them, GZL is the quality coefficient of fruits and vegetables, BZC is the characterization difference parameter of fruits and vegetables, YDC is the hardness difference parameter of fruits and vegetables, β1 and β2 are weight factors, and β1 and β2 are both greater than 0.

[0040] As a further description of the above technical solution: the method for controlling the mechanical arm of the fruit and vegetable picking robot based on the ripeness coefficient and quality coefficient of fruits and vegetables to sort the picked fruits and vegetables to predetermined storage locations is as follows:

[0041] Set the maturity coefficient thresholds of fruits and vegetables [SD1, SD2], and set the quality coefficient thresholds of fruits and vegetables [GZL1, GZL2], where SD1 < SD2, GZL1 < GZL2;

[0042] If SD<SD1, GZL<GZL1, a first control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the first storage location according to the first control command;

[0043] If SD<SD1, GZL1≤GZL<GZL2, a second control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the second storage location according to the second control command;

[0044] If SD<SD1, GZL≥GZL2, a third control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the third storage location according to the third control command;

[0045] If SD1≤SD<SD2, GZL<GZL1, a fourth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the fourth storage location according to the fourth control command;

[0046] If SD1≤SD<SD2, GZL1≤GZL<GZL2, a fifth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the fifth storage location according to the fifth control command;

[0047] If SD1≤SD<SD2, GZL≥GZL2, a sixth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the sixth storage location according to the sixth control command;

[0048] If SD≥SD2 and GZL<GZL1, a seventh control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the seventh storage location according to the seventh control command;

[0049] If SD≥SD2, GZL1≤GZL<GZL2, then an eighth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the eighth storage position according to the eighth control command;

[0050] If SD≥SD2, GZL1≤GZL<GZL2, a ninth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the ninth storage position according to the ninth control command.

[0051] In the above technical solution, the present invention provides a fruit and vegetable picking robot control system, which first collects images of multiple parts of fruits and vegetables, and based on the images of multiple parts of fruits and vegetables, accurately analyzes and calculates the color parameters of fruits and vegetables and the characterization difference parameters of fruits and vegetables, and then collects the hardness of fruits and vegetables at various positions to accurately analyze and calculate the hardness parameters and hardness difference parameters of fruits and vegetables. Among them, by combining the color parameters and hardness parameters of fruits and vegetables for analysis, the ripeness coefficient of fruits and vegetables can be calculated, and the ripeness coefficient can fully and accurately reflect the ripeness of fruits and vegetables. Through the characterization difference parameters and hardness difference parameters of fruits and vegetables, the quality coefficient of fruits and vegetables can be calculated, and the quality coefficient can fully and accurately reflect the quality of fruits and vegetables, providing accurate and effective data support for subsequent sorting of fruits and vegetables, thereby improving the sorting quality of picked fruits and vegetables, and no longer requiring machines or manual labor to perform secondary sorting of fruits and vegetables, thereby saving process costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 This is an overall system block diagram of a fruit and vegetable picking robot control system provided by the present invention;

[0054] Figure 2 This is a control flow diagram of a fruit and vegetable picking robot control system provided by the present invention;

[0055] Figure 3 This is a flowchart of a method for calculating the ripeness coefficient of fruits and vegetables in a fruit and vegetable picking robot control system provided by the present invention;

[0056] Figure 4 This is a flow chart of a method for calculating the quality coefficient of fruits and vegetables in a fruit and vegetable picking robot control system provided by the present invention. DETAILED DESCRIPTION

[0057] The following description is merely illustrative in nature and is not intended to limit the present disclosure, its applications, or uses. It should be understood that throughout the drawings, identical or similar reference numerals indicate identical or similar parts and features. The drawings merely schematically illustrate the concepts and principles of the embodiments of the present disclosure and do not necessarily depict the specific dimensions and proportions of the various embodiments of the present disclosure. Certain portions of certain drawings may be exaggerated to illustrate relevant details or structures of the embodiments of the present disclosure.

[0058] Example 1

[0059] Reference Figure 1-2 This embodiment provides a technical solution: a fruit and vegetable picking robot control system, which is used to control the fruit and vegetable picking robot to sort fruits and vegetables according to their characteristics and place them in predetermined storage locations. The control system includes a front-end acquisition device, a data integration module, and a processing control center.

[0060] The front-end acquisition equipment includes a high-definition camera and multiple force sensors. The high-definition camera is installed on the fruit and vegetable picking robot, and the multiple force sensors are evenly distributed on the inner side of the gripper at the front end of the fruit and vegetable picking robot's mechanical arm.

[0061] The high-definition camera and various force sensors transmit the collected fruit and vegetable data to the data integration module, which integrates the fruit and vegetable data to obtain a fruit and vegetable data set U and transmits it to the processing and control center;

[0062] The processing and control center performs integrated analysis and calculation on the obtained fruit and vegetable dataset U, calculates the ripeness coefficient and quality coefficient of the fruits and vegetables, and controls the robotic arm of the fruit and vegetable picking robot based on the ripeness coefficient and quality coefficient of the fruits and vegetables to sort the picked fruits and vegetables to the predetermined storage location.

[0063] As a further description of the above technical solution: Fruit and vegetable dataset U∈{A1(X1,Y1), A2(X2,Y2), A3(X3,Y3), ...A n (X n ,Y n )}, where A n Indicates the data of the nth collection location of fruits and vegetables; A n (X n ,Y n ), where X n Y represents the image of the nth acquisition position on the surface of fruits and vegetables captured by the high-definition camera, n Indicates the hardness data of the nth sampling position on the surface of fruits and vegetables collected by the force sensor.

[0064] It should be noted that the method for capturing images of the nth capture position on the surface of fruits and vegetables with a high-definition camera is as follows: the gripper at the front end of the fruit and vegetable picking robot's mechanical arm is designed to be a rotatable structure. After the fruits and vegetables are picked, the rotation of the gripper can drive the fruits and vegetables to rotate, allowing the high-definition camera to capture images of multiple positions around the fruits and vegetables.

[0065] Specifically, the processing control center performs integrated analysis and calculation on the obtained fruit and vegetable data set. Calculating the ripeness of fruits and vegetables specifically includes the following steps:

[0066] Identify the fruit and vegetable data collected at each location in the fruit and vegetable dataset U and extract the image and hardness data collected at each location;

[0067] The extracted images are input one by one into the fruit and vegetable color machine learning model to calculate the color values ​​of the fruits and vegetables in the fruit and vegetable images. It should be noted that the color values ​​of fruits and vegetables are directly related to their maturity. Taking tomatoes as an example, during their growth process, their color gradually changes from green to red. At this time, according to the change in their growth time, their corresponding color changes, and the color value gradually increases. In summary, the larger the color value of fruits and vegetables, the higher the maturity of fruits and vegetables;

[0068] The color values ​​of each position of fruits and vegetables are integrated into the fruit and vegetable color value dataset P∈{b1, b2, b3, …b n}, where b n Represents the color value of the nth sampling position on the surface of fruits and vegetables, and calculates the color parameters of fruits and vegetables. The specific method is as follows:

[0069]

[0070] Among them, YS is the color parameter of fruits and vegetables, b max is the maximum value in the fruit and vegetable color value dataset P, b min is the minimum value in the fruit and vegetable color value dataset P. It should be noted that the color parameters of fruits and vegetables are directly related to their maturity, but the color of fruits and vegetables is also directly related to light. The color of parts exposed to light for a long time is generally darker than that of parts exposed to light for a short time. Therefore, when calculating the color parameters of fruits and vegetables, the maximum and minimum values ​​in a color value are first removed. In this way, the color parameters of fruits and vegetables can better reflect the maturity of fruits and vegetables, and the judgment of fruits and vegetables is more accurate.

[0071] The hardness data of fruits and vegetables at various positions are integrated into the fruit and vegetable hardness dataset K∈{c1, c2, c3, ...c n}, where c n Represents the hardness data of the nth sampling position on the surface of fruits and vegetables, and calculates the hardness parameters of fruits and vegetables. The specific method is as follows:

[0072]

[0073] Among them, YD is the hardness parameter of fruits and vegetables, c max is the maximum value in the fruit and vegetable hardness dataset K, c min is the minimum value in the fruit and vegetable hardness dataset K. It should be noted that the hardness parameter of fruits and vegetables is directly related to their maturity. That is, the hardness of fruits and vegetables gradually decreases with increasing maturity. However, considering that fruits and vegetables with local abnormalities (such as damage) often have a large difference in hardness from normal hardness, when calculating the hardness parameter of fruits and vegetables, the maximum and minimum values ​​in the hardness data are first removed. In this way, the hardness parameter of fruits and vegetables can better reflect the maturity of fruits and vegetables, and the judgment of fruits and vegetables is more accurate.

[0074] The ripeness coefficient of fruits and vegetables is calculated based on their color and hardness parameters.

[0075] Specifically, the training method of the fruit and vegetable color machine learning model is as follows:

[0076] Historical images of fruits and vegetables are collected and labeled with the color parameters of the fruits and vegetables. The fruit and vegetable images and the labels corresponding to the fruit and vegetable images constitute a set of training data. X sets of training data constitute a sample set, where X is an integer greater than 1. The sample set is divided into a training set and a test set. The fruit and vegetable images in the training set are used as the input of a fruit and vegetable color machine learning model, and the labels in the training set are used as the output of the fruit and vegetable color machine learning model. The fruit and vegetable color machine learning model is trained to obtain a fruit and vegetable color judgment model. The training goal is to minimize the sum of prediction errors. The fruit and vegetable color judgment model is evaluated using the test set. The fruit and vegetable color judgment model when the sum of prediction errors reaches convergence is used as the constructed fruit and vegetable color machine learning model.

[0077] Furthermore, based on the color parameters and hardness parameters of fruits and vegetables, the method for calculating the ripeness coefficient of fruits and vegetables is as follows:

[0078]

[0079] Among them, SD is the maturity coefficient of fruits and vegetables, YS is the color parameter of fruits and vegetables, and YD is the hardness parameter of fruits and vegetables. is the weight factor, The weight factors are all greater than 0. The weight factors are collected by a technician in this field and the corresponding weight factors are set for each set of comprehensive parameters. The set weight factors and the collected comprehensive parameters are substituted into the formula. Any three formulas constitute a three-variable linear equation system. The calculated weight factors are screened and averaged to obtain The mean of , optionally, is 0.42, is 0.58;

[0080] To summarize, in this embodiment, by collecting images of multiple parts of fruits and vegetables, the color parameters of fruits and vegetables are accurately analyzed and calculated. By collecting the hardness of fruits and vegetables at various positions, the hardness parameters of fruits and vegetables are accurately analyzed and calculated. Then, the ripeness coefficient of fruits and vegetables is calculated by combining the color parameters and hardness parameters of fruits and vegetables. This ripeness coefficient can fully and accurately reflect the ripeness of fruits and vegetables, making subsequent sorting more accurate, thereby improving the sorting quality of picked fruits and vegetables.

[0081] Example 2

[0082] Reference Figure 3 Based on Example 1, this embodiment provides a technical solution: the processing control center performs integrated analysis and calculation on the acquired fruit and vegetable data set, and the calculation of the quality coefficient of the fruits and vegetables specifically includes the following steps:

[0083] Identify the fruit and vegetable data collected at each location in the fruit and vegetable dataset U and extract the image and hardness data collected at each location;

[0084] The extracted images are input one by one into the fruit and vegetable representation difference parameter machine learning model to calculate the representation difference parameters of the fruits and vegetables in the fruit and vegetable images. It should be noted that when calculating the representation difference parameters of the fruits and vegetables, difference pictures of the fruits and vegetables are input into the fruit and vegetable representation difference parameter machine learning model in advance, that is, normal fruit and vegetable pictures and pictures with difference features in the fruits and vegetables. The pictures with difference features in the fruits and vegetables include but are not limited to: pictures with bad spots on the surface of fruits and vegetables, and pictures with cracks on the surface of fruits and vegetables. Based on this, the larger the representation difference parameter of the fruits and vegetables, the worse the quality of the fruits and vegetables.

[0085] The hardness of standard quality fruits and vegetables is set as c b , the hardness data of fruits and vegetables at various positions are integrated into the fruit and vegetable hardness dataset K∈{c1, c2, c3, ...c n}, where c n represents the hardness data of the nth sampling position on the surface of fruits and vegetables. The calculation method of the hardness difference parameter of fruits and vegetables is as follows:

[0086]

[0087] Among them, YDC is the hardness difference parameter of fruits and vegetables. It should be noted that the quality of fruits and vegetables is directly related to their hardness. Taking tomatoes as an example, the greater the difference between the hardness of tomatoes and mature standard tomatoes, the lower their quality. Conversely, the smaller the difference between the hardness of tomatoes and mature standard tomatoes, the higher their quality. That is, the larger the YDC value, the lower the quality of fruits and vegetables, and the smaller the YDC value, the higher the quality of fruits and vegetables.

[0088] The quality coefficient of fruits and vegetables is calculated based on the characterization difference parameters and hardness difference parameters of fruits and vegetables.

[0089] Specifically, the training method of the fruit and vegetable difference machine learning model is as follows:

[0090] Historical images of fruits and vegetables are collected and labeled with labels representing the representation difference parameters of the fruits and vegetables. The fruit and vegetable images and the labels corresponding to the fruit and vegetable images constitute a set of training data. X sets of training data constitute a sample set, where X is an integer greater than 1. The sample set is divided into a training set and a test set. The fruit and vegetable images in the training set are used as the input of a fruit and vegetable difference machine learning model, and the labels in the training set are used as the output of the fruit and vegetable difference machine learning model. The fruit and vegetable difference machine learning model is trained to obtain a fruit and vegetable representation difference parameter calculation model. The training goal is to minimize the sum of prediction errors. The test set is used to evaluate the fruit and vegetable representation difference parameter calculation model. The fruit and vegetable representation difference parameter calculation model when the sum of prediction errors reaches convergence is used as the constructed fruit and vegetable representation difference parameter machine learning model.

[0091] Specifically, based on the characterization difference parameters and hardness difference parameters of fruits and vegetables, the method for calculating the quality coefficient of fruits and vegetables is as follows:

[0092]

[0093] Among them, GZL is the fruit and vegetable quality coefficient, BZC is the characterization difference parameter of fruits and vegetables, YDC is the hardness difference parameter of fruits and vegetables, β1 and β2 are weight factors, β1 and β2 are both greater than 0, and the weight factors are collected by technical personnel in this field. Multiple sets of comprehensive parameters are collected, and corresponding weight factors are set for each set of comprehensive parameters. The set weight factors and the collected comprehensive parameters are substituted into the formula. Any three formulas constitute a set of three-variable linear equations. The calculated weight factors are screened and averaged to obtain the average of β1 and β2. Optionally, β1+β2=1, β1 is 0.66, and β1 is 0.34.

[0094] To sum up, in this embodiment, by collecting images of multiple parts of fruits and vegetables, the characterization difference parameters of fruits and vegetables are accurately analyzed and calculated. By collecting the hardness of fruits and vegetables at various positions, the hardness difference parameters of fruits and vegetables are accurately analyzed and calculated. Then, the quality coefficient of fruits and vegetables is calculated by combining the characterization difference parameters and the hardness difference parameters of fruits and vegetables. This quality coefficient can fully and accurately reflect the quality of fruits and vegetables, provide accurate data support for subsequent sorting, and thus further improve the sorting quality of picked fruits and vegetables.

[0095] Example 3

[0096] Reference Figure 4 Based on Example 2, this embodiment provides a technical solution: a processing control center controls the robotic arm of a fruit and vegetable picking robot based on the ripeness coefficient and quality coefficient of the fruits and vegetables, and sorts the picked fruits and vegetables to a predetermined storage location as follows:

[0097] Set the maturity coefficient thresholds of fruits and vegetables [SD1, SD2], and set the quality coefficient thresholds of fruits and vegetables [GZL1, GZL2], where SD1 < SD2, GZL1 < GZL2;

[0098] If SD<SD1, GZL<GZL1, a first control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the first storage location according to the first control command;

[0099] If SD<SD1, GZL1≤GZL<GZL2, a second control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the second storage location according to the second control command;

[0100] If SD<SD1, GZL≥GZL2, a third control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the third storage location according to the third control command;

[0101] If SD1≤SD<SD2, GZL<GZL1, a fourth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the fourth storage location according to the fourth control command;

[0102] If SD1≤SD<SD2, GZL1≤GZL<GZL2, a fifth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the fifth storage location according to the fifth control command;

[0103] If SD1≤SD<SD2, GZL≥GZL2, a sixth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the sixth storage location according to the sixth control command;

[0104] If SD≥SD2 and GZL<GZL1, a seventh control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the seventh storage location according to the seventh control command;

[0105] If SD≥SD2, GZL1≤GZL<GZL2, then an eighth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the eighth storage location according to the eighth control command;

[0106] If SD≥SD2, GZL1≤GZL<GZL2, then a ninth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the ninth storage position according to the ninth control command.

[0107] In addition, it should be noted that the size of the weight factor is a specific numerical value obtained by quantifying each data to facilitate subsequent comparison. The size of the weight factor depends on the number of comprehensive parameters and the preliminary setting of the corresponding weight factor for each set of comprehensive parameters by technical personnel in this field.

[0108] The exemplary implementation schemes proposed in the present disclosure are described in detail above with reference to preferred embodiments. However, it will be understood by those skilled in the art that, without departing from the concept of the present disclosure, various modifications and variations can be made to the above-mentioned specific embodiments, and various technical features and structures proposed in the present disclosure can be combined in various ways without exceeding the scope of protection of the present disclosure, which is determined by the appended claims.

Claims

1. A fruit and vegetable picking robot control system, which is used to control the fruit and vegetable picking robot to sort fruits and vegetables according to their characteristics and place them in predetermined storage locations, characterized in that: It includes front-end acquisition equipment, data integration module and processing control center: The front-end acquisition device includes a high-definition camera and multiple force sensors. The high-definition camera is installed on the fruit and vegetable picking robot, and the multiple force sensors are evenly distributed on the inner side of the gripper at the front end of the fruit and vegetable picking robot's mechanical arm; The high-definition camera and various force sensors transmit the collected fruit and vegetable data to the data integration module; The data integration module integrates the fruit and vegetable data to obtain the fruit and vegetable data set U and transmits it to the processing control center; The processing control center performs integrated analysis and calculation on the acquired fruit and vegetable dataset U, calculates the ripeness coefficient and quality coefficient of the fruits and vegetables, and controls the robotic arm of the fruit and vegetable picking robot based on the ripeness coefficient and quality coefficient of the fruits and vegetables to sort the picked fruits and vegetables to a predetermined storage location.

2. A fruit and vegetable picking robot control system according to claim 1, characterized in that: The fruit and vegetable dataset U∈{A1(X1, Y1), A2(X2, Y2), A3(X3, Y3), ...A n (X n ,Y n )}, where A n Indicates the data of the nth collection location of fruits and vegetables; A n (X n ,Y n ), where X n Y represents the image of the nth acquisition position on the surface of fruits and vegetables captured by the high-definition camera, n Indicates the hardness data of the nth sampling position on the surface of fruits and vegetables collected by the force sensor.

3. A fruit and vegetable picking robot control system according to claim 1, characterized in that: The processing control center performs integrated analysis and calculation on the acquired fruit and vegetable data set. The calculation of the ripeness of the fruits and vegetables specifically includes the following steps: Identify the fruit and vegetable data collected at each location in the fruit and vegetable dataset U and extract the image and hardness data collected at each location; Input the extracted images one by one into the fruit and vegetable color machine learning model to calculate the color values ​​of the fruits and vegetables in the fruit and vegetable images; The color values ​​of each position of fruits and vegetables are integrated into the fruit and vegetable color value dataset P∈{b1, b2, b3, …b n }, where b n Represents the color value of the nth sampling position on the surface of fruits and vegetables, and calculates the color parameters of fruits and vegetables. The specific method is as follows: Among them, YS is the color parameter of fruits and vegetables, b max is the maximum value in the fruit and vegetable color value dataset P, b min is the minimum value in the fruit and vegetable color value dataset P; The hardness data of fruits and vegetables at various positions are integrated into the fruit and vegetable hardness dataset K∈{c1, c2, c3, …c n }, where c n Represents the hardness data of the nth sampling position on the surface of fruits and vegetables, and calculates the hardness parameters of fruits and vegetables. The specific method is as follows: Among them, YD is the hardness parameter of fruits and vegetables, c max is the maximum value in the fruit and vegetable hardness dataset K, c min is the minimum value in the fruit and vegetable hardness dataset K; The ripeness coefficient of fruits and vegetables is calculated based on their color and hardness parameters.

4. A fruit and vegetable picking robot control system according to claim 3, characterized in that: The training method of the fruit and vegetable color machine learning model is as follows: Historical images of fruits and vegetables are collected and labeled, wherein the labels are color parameters of the fruits and vegetables. The fruit and vegetable images and the labels corresponding to the fruit and vegetable images constitute a set of training data, X sets of training data constitute a sample set, where X is an integer greater than 1, and the sample set is divided into a training set and a test set. The fruit and vegetable images in the training set are used as input of a fruit and vegetable color machine learning model, and the labels in the training set are used as output of the fruit and vegetable color machine learning model. The fruit and vegetable color machine learning model is trained to obtain a fruit and vegetable color judgment model, wherein minimizing the sum of prediction errors is the training objective. The fruit and vegetable color judgment model is evaluated using the test set, and the fruit and vegetable color judgment model when the sum of prediction errors reaches convergence is used as the constructed fruit and vegetable color machine learning model.

5. The fruit and vegetable picking robot control system according to claim 3, characterized in that: Based on the color and firmness parameters of fruits and vegetables, the method for calculating the ripeness coefficient of fruits and vegetables is as follows: Among them, SD is the maturity coefficient of fruits and vegetables, YS is the color parameter of fruits and vegetables, and YD is the hardness parameter of fruits and vegetables. is the weight factor, Both are greater than 0.

6. The fruit and vegetable picking robot control system according to claim 2, characterized in that: The processing control center performs integrated analysis and calculation on the acquired fruit and vegetable data set, and calculating the quality coefficient of the fruits and vegetables specifically includes the following steps: Identify the fruit and vegetable data collected at each location in the fruit and vegetable dataset U and extract the image and hardness data collected at each location; The extracted images are input one by one into the machine learning model of fruit and vegetable representation difference parameters to calculate the representation difference parameters of fruits and vegetables in the fruit and vegetable images; The hardness of standard quality fruits and vegetables is set as c b , the hardness data of fruits and vegetables at various positions are integrated into the fruit and vegetable hardness dataset K∈{c1, c2, c3, …c n }, where c n represents the hardness data of the nth sampling position on the surface of fruits and vegetables. The calculation method of the hardness difference parameter of fruits and vegetables is as follows: Among them, YDC is the hardness difference parameter of fruits and vegetables; The quality coefficient of fruits and vegetables is calculated based on the characterization difference parameters and hardness difference parameters of fruits and vegetables.

7. The fruit and vegetable picking robot control system according to claim 6, characterized in that: The training method of the fruit and vegetable difference machine learning model is as follows: Historical images of fruits and vegetables are collected, and labels are set for the fruit and vegetable images, wherein the labels are representation difference parameters of the fruits and vegetables. The fruit and vegetable images and the labels corresponding to the fruit and vegetable images constitute a set of training data, and X groups of training data constitute a sample set, where X is an integer greater than 1. The sample set is divided into a training set and a test set. The fruit and vegetable images in the training set are used as input to a fruit and vegetable difference machine learning model, and the labels in the training set are used as output of the fruit and vegetable difference machine learning model. The fruit and vegetable difference machine learning model is trained to obtain a fruit and vegetable representation difference parameter calculation model, and minimizing the sum of prediction errors is used as the training goal. The fruit and vegetable representation difference parameter calculation model is evaluated using the test set, and the fruit and vegetable representation difference parameter calculation model when the sum of prediction errors reaches convergence is used as the constructed fruit and vegetable representation difference parameter machine learning model.

8. The fruit and vegetable picking robot control system according to claim 6, characterized in that: Based on the characterization difference parameters and hardness difference parameters of fruits and vegetables, the method for calculating the quality coefficient of fruits and vegetables is as follows: Among them, GZL is the quality coefficient of fruits and vegetables, BZC is the characterization difference parameter of fruits and vegetables, YDC is the hardness difference parameter of fruits and vegetables, β1 and β2 are weight factors, and β1 and β2 are both greater than 0.

9. The fruit and vegetable picking robot control system according to claim 8, characterized in that: The processing control center controls the robotic arm of the fruit and vegetable picking robot based on the maturity coefficient and quality coefficient of the fruits and vegetables to sort the picked fruits and vegetables to predetermined storage locations as follows: Set the maturity coefficient thresholds of fruits and vegetables [SD1, SD2], and set the quality coefficient thresholds of fruits and vegetables [GZL1, GZL2], where SD1 < SD2, GZL1 < GZL2; If SD<SD1, GZL<GZL1, a first control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the first storage location according to the first control command; If SD<SD1, GZL1≤GZL<GZL2, a second control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the second storage location according to the second control command; If SD<SD1, GZL≥GZL2, a third control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the third storage location according to the third control command; If SD1≤SD<SD2, GZL<GZL1, a fourth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the fourth storage location according to the fourth control command; If SD1≤SD<SD2, GZL1≤GZL<GZL2, a fifth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the fifth storage location according to the fifth control command; If SD1≤SD<SD2, GZL≥GZL2, a sixth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the sixth storage location according to the sixth control command; If SD≥SD2 and GZL<GZL1, a seventh control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the seventh storage location according to the seventh control command; If SD≥SD2, GZL1≤GZL<GZL2, then an eighth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the eighth storage position according to the eighth control command; If SD≥SD2, GZL1≤GZL<GZL2, a ninth control command is generated, and the processing control center controls the robotic arm to grab the fruits and vegetables to the ninth storage position according to the ninth control command.

Images