Tomato region recognition method and system based on deep learning

Abstract

This invention discloses a deep learning-based method and system for tomato region recognition, comprising: acquiring historical images of the tomato unloading process and performing hybrid enhancement to generate an enhanced image set; annotating the generated enhanced image set to obtain instance annotations, and identifying the color and stacking features of the tomatoes within it; constructing a deep learning model based on the color features, stacking features, and instance annotations of the enhanced image set, and training and testing it to obtain a tomato region recognition model; determining real-time images of the tomato unloading process and inputting them into the tomato region recognition model for output, thereby obtaining the tomato regions in the tomato unloading images. This invention enhances the original images by simulating multi-dimensional real-world scenarios and integrates color and stacking features to construct a model, enabling tomato region recognition to adapt to dynamic lighting and stacking patterns, improving recognition accuracy, and solving the problem of identifying and segmenting individual and group tomato regions under complex unloading conditions.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a tomato region recognition method and system based on deep learning. Background Technology

[0002] As a widely cultivated and consumed fruit and vegetable crop globally, the automated sorting and quality grading of tomatoes in their post-harvest processing has become an important direction for the development of modern agricultural equipment. During the unloading process of tomatoes, it is necessary to quickly identify and locate the large number of scattered and piled tomatoes from the conveyor belt or turnover box, which places extremely high demands on the accuracy, robustness, and real-time performance of visual perception.

[0003] However, traditional methods, based on manually designed color thresholds, edge detection, or morphological operators, can achieve certain results in simple scenarios with controlled lighting and single-layer tiling. In real unloading conditions, tomatoes exhibit complex characteristics such as dense stacking, mutual occlusion, motion blur, and drastic changes in lighting, making it difficult to achieve stable and accurate instance-level segmentation. Since they have not been specifically optimized for the special challenges of tomato unloading scenarios, they are prone to undersegmentation or oversegmentation in stacked and sticky areas, and their generalization ability to environmental changes such as lighting and motion is insufficient. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a tomato region recognition method and system based on deep learning, comprising:

[0005] Acquire historical images of the tomato unloading process, and perform hybrid enhancement on these images to accommodate the unloading scene and dynamic lighting, generating an enhanced image set. The tomato colors and tomato stacks in the generated enhanced image set are labeled to obtain instance labels for the enhanced image set, and the color features and stacking features of tomatoes in the enhanced image set are determined. A deep learning model is constructed based on the color features, stacking features, and instance annotations of the enhanced image set. The deep learning model is then trained and tested to obtain a tomato region recognition model. The real-time tomato unloading process image is determined and input into the tomato region recognition model for output, thus obtaining the tomato region in the tomato unloading image.

[0006] Furthermore, the process of acquiring historical tomato unloading process images and performing hybrid enhancement on these images, tailored to the unloading scene and dynamic lighting, to generate an enhanced image set includes: Images of the unloading process are collected from multiple fixed locations at the unloading port to construct a historical tomato unloading process image database, and historical tomato unloading process images containing tomatoes are obtained from the historical tomato unloading process image database; Identify the illumination features in historical tomato unloading process images, and simulate the image changes of tomatoes under different lighting conditions based on the illumination features to generate illumination-enhanced images; The motion features in historical tomato unloading process images were identified, and the image changes of tomatoes under different motion scene conditions were simulated based on the motion features to generate motion-enhanced images; The stacking features in historical tomato unloading process images were identified, and image changes of tomatoes under different stacking scenarios were simulated based on the stacking features to generate stacking-enhanced images; An enhanced image set is generated based on the illumination-enhanced image, motion-enhanced image, and stacked enhanced image, and noise is injected into the enhanced image set, including conveyor belt vibration noise, ambient dust noise, and illumination flicker noise.

[0007] Furthermore, the step of determining the illumination features in historical tomato unloading process images and simulating image changes of tomatoes under different lighting conditions based on the illumination features to generate illumination-enhanced images includes: Illumination conditions were analyzed on historical images of tomato unloading processes, and illumination features of each image were extracted. Illumination features include average brightness, global contrast, color temperature distribution, shadow ratio, and highlight area ratio. A classification model for lighting conditions was established based on lighting features, and historical images of tomato unloading processes were divided into multiple lighting scene categories. For each lighting scene category, a generative adversarial network is used to learn the lighting distribution of that category and generate image variants that are consistent with the original image content but have different lighting conditions. The generated image variants are physically verified, and images that pass the verification are included in the illumination-enhanced image set.

[0008] Furthermore, the step of determining the motion features in historical tomato unloading process images and simulating image changes of tomatoes under different motion scene conditions based on the motion features to generate motion-enhanced images includes: Dynamic motion analysis was performed on historical images of tomato unloading processes to identify the motion states of the tomatoes in the images, including static, free fall, collision and bouncing, and rolling accumulation. The corresponding motion characteristics are determined based on the motion state. The motion characteristics include motion direction, instantaneous velocity, acceleration, and motion trajectory. A motion blur kernel is generated based on motion features, and the motion blur kernel is convolved with historical tomato unloading process images to simulate the motion blur effect of tomatoes during the unloading process. The generated blurred images are subjected to temporal continuity verification, and the images that pass the verification are included in the motion-enhanced image set.

[0009] Furthermore, the step of determining the stacking features in historical tomato unloading process images and simulating image changes of tomatoes under different stacking scenario conditions based on the stacking features to generate stack-enhanced images includes: Stacking contact condition analysis was performed on historical tomato unloading process images to extract stacking features between tomato instances. Stacking features include contact type, occlusion ratio, stacking level, and contact boundary morphology. A tomato instance template library is constructed based on historical tomato unloading process images. The tomato instance template library stores the individual tomato instance masks segmented from historical tomato unloading process images, as well as their corresponding texture and color information. Based on stacking features, multiple tomato instances are selected from the template library, and a synthetic stacked image with specified stacking density, contact angle, and occlusion relationship is generated on the actual unloading background through physical simulation rendering. The synthesized stacked images are checked for illumination consistency, and images that pass the check are included in the stacked enhanced image set.

[0010] Furthermore, the annotation of tomato colors and tomato stacks in the generated enhanced image set to obtain instance annotations for the enhanced image set includes: Tomato color grades are determined based on tomato maturity, and tomato colors in the generated enhanced image set are labeled based on these tomato color grades. The boundary of each tomato instance in the image set is determined by the boundary recognition algorithm, and the tomato stacks in the generated enhanced image set are labeled based on the boundary of the tomato instances.

[0011] Furthermore, determining the color features and stacking features of tomatoes in the enhanced image set includes: Calculate the color statistical features of each tomato in the enhanced image set. The color statistical features include mean, variance, skewness and kurtosis. The color statistical features are then used as the color features of the tomatoes in the enhanced image set. The spatial relationship features of each tomato in the enhanced image set are calculated. The spatial relationship features include adjacent distance, overlapping area, relative angle and contact length. The spatial relationship features are then determined as the stacking features of tomatoes in the enhanced image set.

[0012] Furthermore, the process of constructing a deep learning model based on the color features, stacking features, and instance annotations of the enhanced image set, and training and testing the deep learning model to obtain a tomato region recognition model, includes: Determine the color and stacking features of the enhanced image set, and use the color and stacking features as input data for the model; Instance annotations are used as model output data, and a dataset is constructed based on the model input data and model output data; The dataset is divided into a training set and a test set according to a preset ratio, and the training set is input into a preset graph neural network model for training to build a deep learning model; The dataset is divided into a training set and a test set according to a preset ratio. The training set is then input into a deep learning model for training to obtain an initial tomato region recognition model. The test set is input into the initial tomato region recognition model for testing until the initial tomato region recognition model meets the preset convergence condition, thus obtaining the tomato region recognition model.

[0013] Furthermore, the step of determining the real-time tomato unloading process image and inputting it into the tomato region recognition model for output, to obtain the tomato region in the tomato unloading image, includes: The real-time tomato unloading process image was determined and preprocessed, including illumination normalization, contrast enhancement and noise filtering. The preprocessed real-time tomato unloading process image is input into the tomato region recognition model to obtain the initial tomato region; Morphological optimization and stacking relationship segmentation are performed on the initial tomato region, and the processing results are superimposed on the real-time tomato unloading process image to obtain the tomato region in the tomato unloading image.

[0014] This invention also provides a tomato region recognition system based on deep learning, comprising: The acquisition module is used to acquire historical images of the tomato unloading process and perform hybrid enhancement on these images, taking into account both the unloading scene and dynamic lighting, to generate an enhanced image set. The determination module is used to annotate the tomato colors and tomato stacks in the generated enhanced image set, obtain instance annotations of the enhanced image set, and determine the color features and stacking features of tomatoes in the enhanced image set; The modeling module is used to build a deep learning model based on the color features, stacking features, and instance annotations of the enhanced image set, and to train and test the deep learning model to obtain the tomato region recognition model. The recognition module is used to obtain real-time images of the tomato unloading process and input these images into the tomato region recognition model for output, thereby obtaining the tomato region in the tomato unloading image.

[0015] Compared with existing technologies, the tomato region recognition method and system based on deep learning proposed in this invention have the following advantages: This invention uses a hybrid enhancement strategy targeting unloading scenarios and dynamic lighting to simulate the complex and varied lighting conditions, motion blur, and stacking contact patterns in real unloading processes. This significantly expands the diversity and coverage of training data, enabling the model to maintain stable color feature extraction capabilities under harsh lighting conditions such as strong light, shadow, and backlight. It can accurately restore instance boundaries in dynamic scenarios such as high-speed movement and collision rolling of tomatoes, and accurately separate individual contours under high-density stacking and severe occlusion. This invention uses graph neural network modeling of multi-spatial fusion and stacked features of color features to deeply mine the ripeness information and spatial position relationship of tomatoes, effectively solves the problem of blurred boundaries of the sticky area, and greatly reduces the missed detection and false detection rates caused by occlusion and contact. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process structure of the tomato region recognition method based on deep learning in an embodiment of the present invention; Figure 2 This is a schematic diagram of the composition of the tomato region recognition system based on deep learning in an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0018] like Figure 1 As shown in the embodiments of this application, a tomato region recognition method based on deep learning is provided, including: S100: acquiring historical tomato unloading process images, and performing hybrid enhancement on the historical tomato unloading process images to be oriented towards the unloading scene and dynamic lighting, generating an enhanced image set; S200: labeling the tomato colors and tomato stacking in the generated enhanced image set to obtain instance labels of the enhanced image set, and determining the color features and stacking features of the tomatoes in the enhanced image set; S300: constructing a deep learning model based on the color features, stacking features and instance labels of the enhanced image set, and training and testing the deep learning model to obtain a tomato region recognition model; S400: determining real-time tomato unloading process images, and inputting the real-time tomato unloading process images into the tomato region recognition model for output, to obtain the tomato regions in the tomato unloading images.

[0019] Furthermore, this invention employs a hybrid enhancement strategy targeting unloading scenarios and dynamic lighting to simulate the complex and varied lighting conditions, motion blur, and stacking contact patterns during real unloading processes. This significantly expands the diversity and coverage of training data, enabling the model to maintain stable color feature extraction capabilities even under harsh lighting conditions such as strong light, shadow, and backlight. It accurately restores instance boundaries in dynamic scenarios such as high-speed tomato movement and collision / rolling, and precisely separates individual contours under high-density stacking and severe occlusion. Through multi-spatial fusion of color features and graph neural network modeling of stacking features, this invention allows the model to deeply mine the ripeness information and spatial relationship of tomatoes, effectively solving the problem of boundary blurring in adhered areas and significantly reducing the false negative and false positive rates caused by occlusion and contact.

[0020] In an embodiment of this application, a deep learning-based tomato region recognition method is provided. The method involves acquiring historical tomato unloading process images and performing hybrid enhancement on these images, considering both the unloading scenario and dynamic lighting, to generate an enhanced image set. This includes: acquiring unloading process images from multiple fixed locations at the unloading port to construct a historical tomato unloading process image database; obtaining historical tomato unloading process images containing tomatoes from the database; determining lighting features in the historical tomato unloading process images and simulating image changes of tomatoes under different lighting conditions based on these features to generate lighting-enhanced images; determining motion features in the historical tomato unloading process images and simulating image changes of tomatoes under different motion conditions based on these features to generate motion-enhanced images; determining stacking features in the historical tomato unloading process images and simulating image changes of tomatoes under different stacking conditions based on these features to generate stacked-enhanced images; generating an enhanced image set based on the lighting-enhanced images, motion-enhanced images, and stacked-enhanced images; and injecting noise into the enhanced image set, including conveyor belt vibration noise, environmental dust noise, and light flicker noise.

[0021] Specifically, by simultaneously deploying industrial cameras at multiple fixed positions such as directly above, diagonally above, and to the side of the unloading port, a multi-view historical image database of the tomato unloading process is constructed to ensure comprehensive coverage of the imaging characteristics of tomatoes from different angles and distances. Illumination condition analysis, dynamic motion analysis, and stacking contact condition analysis are performed on the images to extract illumination feature vectors including brightness, color temperature, and shadow ratio; motion feature parameters including motion direction, speed, and acceleration; and stacking features including contact type, occlusion ratio, and stacking level. Based on these features, a generative adversarial network is used for illumination style transfer, motion blur kernel convolution is used to simulate dynamic trailing effects, and 3D physical rendering or instance template synthesis is used to simulate high-density stacking and contact deformation. This generates large-scale enhanced images covering extreme conditions from direct sunlight to low-light shadows, from static tiling to high-speed falling, and from single-layer scattering to multi-layer stacking. To address the unique sensor noise environment at the unloading site, motion artifact noise caused by high-frequency vibration of the conveyor belt, local occlusion noise caused by environmental dust adhesion, and light flicker noise caused by artificial light source flicker are injected into the enhanced images, making the training data distribution highly approximate the real industrial scene. This step, through the organic combination of multi-view acquisition and multi-dimensional physical enhancement, improves the scene coverage of training data by more than two orders of magnitude without the need for additional manual annotation costs, enabling the model to fully learn various extreme imaging conditions of the entire unloading process during the training phase; the injected domain-specific noise significantly enhances the model's adaptability to the non-ideal characteristics of sensors and effectively suppresses the performance degradation caused by equipment differences or environmental interference during actual deployment.

[0022] In an embodiment of this application, a deep learning-based tomato region recognition method is provided. The method involves determining the illumination features in historical tomato unloading process images and simulating image changes of tomatoes under different lighting conditions based on these features to generate illumination-enhanced images. This includes: analyzing the illumination conditions of historical tomato unloading process images and extracting illumination features from each image, including average brightness, global contrast, color temperature distribution, shadow ratio, and highlight area ratio; establishing an illumination condition classification model based on the illumination features and dividing the historical tomato unloading process images into multiple illumination scene categories; for each illumination scene category, learning the illumination distribution of that category using a generative adversarial network and generating image variants that are identical to the original image content but have different illumination conditions; performing physical consistency verification on the generated image variants and incorporating the verified images into the illumination-enhanced image set.

[0023] Specifically, a refined illumination condition analysis is performed on historical images of the tomato unloading process. By calculating key illumination features such as average brightness, global contrast, color temperature distribution, shadow ratio, and highlight area proportion for each image, a multi-dimensional feature vector representing the illumination state of the image is constructed. Based on these illumination features, a illumination condition classification model is established using clustering or classification algorithms to automatically divide historical images into several representative illumination scene categories, such as strong direct light, uniform diffused light, low-light shadows, and backlighting with high contrast. For each illumination scene category, a dedicated generative adversarial network is trained to fully learn the illumination distribution patterns of that category. Using the content features of the original image as a condition, diverse image variants are generated that are completely consistent with the content of the source image but have different illumination styles. This allows a single image to be expanded into corresponding versions under multiple illumination conditions without manual re-annotation. The generated image variants undergo physical consistency verification to ensure that their shadow direction, light source color temperature, and highlight position conform to the real physical imaging laws. Abnormal samples that do not conform to the physical constraints of illumination are filtered out, and high-quality images that pass the verification are included in the illumination enhancement image set. This step systematically models and generalizes the complex lighting changes in the unloading scenario, expanding limited historical image data into a large-scale training sample covering the entire lighting spectrum. This significantly enhances the model's adaptability to actual working conditions such as drastic fluctuations in on-site lighting, local shadow interference, and light source color temperature shifts. The physical consistency verification mechanism ensures the authenticity and usability of the generated images, avoiding model overfitting or mislearning caused by unreasonable lighting synthesis. Ultimately, it achieves stable and high-precision segmentation of the tomato region recognition model under different time periods, weather conditions, and light source configurations, greatly improving the deployment success rate and long-term operational reliability of the automated unloading system in complex lighting environments.

[0024] In the embodiments of this application, a tomato region recognition method based on deep learning is provided. The method involves determining motion features in historical tomato unloading process images and simulating image changes of tomatoes under different motion scenarios based on these motion features to generate motion-enhanced images. This includes: performing dynamic motion analysis on historical tomato unloading process images to identify the motion state of the tomatoes in the images, including static, free fall, collision and bouncing, and rolling accumulation; determining corresponding motion features based on the motion state, including motion direction, instantaneous velocity, acceleration, and motion trajectory; generating a motion blur kernel based on the motion features and performing a convolution operation between the motion blur kernel and historical tomato unloading process images to simulate the motion blur effect of tomatoes during the unloading process; performing temporal continuity verification on the generated blurred images and incorporating the verified images into a motion-enhanced image set.

[0025] Specifically, a time-series dynamic motion analysis is performed on historical tomato unloading process images. Through inter-frame differencing, optical flow estimation, or target tracking algorithms, the instantaneous motion state of individual tomatoes in the image sequence is accurately identified, categorized into four typical unloading motion modes: stationary, free fall, collision and bouncing, and rolling accumulation. For each motion state, corresponding quantitative motion feature parameters are further decoupled, including motion direction vector, instantaneous velocity amplitude, acceleration vector, and motion trajectory coordinate sequence between consecutive frames. Based on these physical motion parameters, a motion blur kernel with anisotropic orientation, velocity correlation kernel length, and non-uniform acceleration modulation weights is adaptively generated. When the tomato falls at high speed, a large-scale linear blur kernel is generated to simulate the trailing trajectory. During collisions and bounces, a polygonal blur kernel with abrupt changes in direction is generated to simulate motion steering. During rolling and stacking, a curved motion blur kernel is generated to simulate arc rolling. This motion blur kernel is then convolved with a static historical image in two dimensions. While preserving the tomato's identity features and texture details, a blurred image with a realistic motion trail effect is accurately synthesized, enabling the originally static single-frame image to achieve a dynamic imaging effect equivalent to high-speed photography. The generated blurred image sequence undergoes temporal continuity verification to check whether the tomato's positional changes between adjacent frames conform to the physical laws of the preset motion trajectory. Abnormal samples with abrupt motion changes, discontinuous trajectories, or blur directions inconsistent with the motion direction are removed. High-quality images that pass the rigorous verification are then included in the motion-enhanced image set. This step transforms the complex dynamics of the tomato unloading process into a quantifiable and programmable motion fuzz synthesis operator, enabling the model to anticipate target imaging features under dynamic conditions such as high-speed descent, collision rebound, and stacking roll during the training phase. This fundamentally solves the problems of target edge blurring, feature annihilation, and instance identity loss caused by motion fuzziness. Through temporal continuity verification of physical constraints, it ensures that the generated dynamic samples conform to real kinematic laws, avoiding the interference of unreasonable fuzz artifacts on model training. Ultimately, it endows the tomato region recognition model with powerful dynamic scene perception capabilities, enabling it to accurately infer the motion state of the target from a single frame of static image, recover the precise position of the fuzzy boundary, and achieve stable detection and instance segmentation of moving targets on a high-speed unloading production line.

[0026] In the embodiments of this application, a deep learning-based tomato region recognition method is provided. The method involves determining stacking features in historical tomato unloading process images and simulating image changes of tomatoes under different stacking scenarios based on these features to generate stacked enhanced images. This includes: analyzing stacking contact conditions in historical tomato unloading process images and extracting stacking features between tomato instances, including contact type, occlusion ratio, stacking level, and contact boundary morphology; constructing a tomato instance template library based on historical tomato unloading process images, storing individual tomato instance masks segmented from the historical images and their corresponding texture and color information; selecting multiple tomato instances from the template library based on the stacking features and generating a synthetic stacked image with specified stacking density, contact angle, and occlusion relationship on an actual unloading background through physical simulation rendering; performing illumination consistency adjustment and verification on the synthetic stacked image and incorporating the verified images into a stacked enhanced image set.

[0027] Specifically, a deep stacking contact condition analysis is performed on historical tomato unloading process images. Through contour curvature detection, concave point recognition, and spatial position relationship inference, the stacking features between tomato instances are accurately extracted, including contact type (point contact, line contact, surface contact), occlusion ratio, stacking level (upper and lower layer relationship), and contact boundary morphology (smooth transition or extrusion deformation). An instance template library containing a large number of high-quality individual tomatoes is segmented and constructed from historical images. Each template stores accurate instance masks, surface texture details, and multi-color space distribution features. Based on the extracted stacking feature parameters, the template library... The system intelligently selects tomato instances of varying shapes, sizes, and colors, and uses a 3D physics engine to simulate rigid body collisions. It then renders in real-time on actual unloading background images to generate composite stacked images with realistic physical stacking postures, precise contact angles, and natural occlusion relationships. The system automatically outputs pixel-level labeled masks and stacking relationship maps for each instance. Global illumination consistency is verified on the composite stacked images to ensure that the light source direction, shadow projection, and ambient light occlusion of each instance conform to a unified physical scene. Samples with inconsistent illumination or unnatural contact are filtered out, and high-fidelity images that pass the verification are included in the stacking enhancement image set. This step overcomes the bottleneck of traditional data acquisition, which struggles to obtain large-scale and diverse stacked samples. Through an innovative paradigm of template-based stack synthesis, it exponentially expands the training data coverage of stacked scenarios with zero manual annotation costs, covering extreme conditions such as sparse contact, slight occlusion, high-density multi-layer stacking, and severe compression deformation. The physical simulation rendering mechanism ensures the high realism of the synthesized stacked images in terms of geometry, lighting relationships, and contact deformation, allowing the stack separation capabilities learned by the model to be directly transferred to real unloading scenarios. This method endows the tomato region recognition model with powerful stack perception and instance decoupling capabilities, enabling it to accurately distinguish individual boundaries, restore the outline of occluded parts, and reconstruct the complete stacking hierarchy even under dense stacking and severe occlusion conditions, significantly improving the adaptability of automated sorting systems to high-density unloading conditions.

[0028] In an embodiment of this application, a tomato region recognition method based on deep learning is provided. The step of annotating the tomato colors and tomato stacks in the generated enhanced image set to obtain instance annotations of the enhanced image set includes: determining the tomato color level according to the tomato maturity and annotating the tomato colors in the generated enhanced image set based on the tomato color level; determining the boundary of each tomato instance in the image set according to a boundary recognition algorithm and annotating the tomato stacks in the generated enhanced image set based on the boundary of the tomato instances.

[0029] Specifically, based on the phased color changes observed during tomato ripening, tomato color grades are scientifically divided into five refined maturity levels: dark green, light green, yellow-green, orange-yellow, and red. For each level, a multi-dimensional threshold judgment standard and color distribution statistical model based on HSV and Lab color spaces are established. On this basis, pixel-level color grade annotations are applied to each tomato instance in the enhanced image set to ensure the model can accurately learn the subtle color differences and transition features from immature to fully mature stages. A boundary recognition algorithm based on depth contour detection and curvature analysis is used to accurately extract edges from each tomato instance in the enhanced image set. Especially in stacked areas where tomatoes are in contact and deformed, a composite strategy combining concave point detection, convex hull analysis, and distance transformation is used to accurately locate the physical segmentation boundaries between different instances. Independent, pixel-level precise binary mask annotations are generated for each tomato instance, while simultaneously recording the contact type, occlusion ratio, and stacking hierarchy between instances. This step, through maturity-driven fine-grained color level labeling, not only distinguishes tomatoes from the background but also achieves accurate maturity discrimination. Furthermore, by precisely locating the stacked contact boundary at the sub-pixel level and generating independent instance masks, it fundamentally solves the labeling problem of blurred instance boundaries and confused attribution in densely stacked scenes. This provides the model with clear and unambiguous supervision signals, enabling it to learn the feature boundaries of different instances at the pixel level. The organic combination of these two aspects allows the trained tomato region recognition model to simultaneously possess maturity sensitivity and stack decoupling capabilities, enabling it to output the precise spatial location, complete outline shape, and maturity level of each tomato instance in a single forward inference.

[0030] In an embodiment of this application, a deep learning-based tomato region recognition method is provided. The method for determining the color features and stacking features of tomatoes in an enhanced image set includes: calculating the color statistical features of each tomato in the enhanced image set, wherein the color statistical features include mean, variance, skewness, and kurtosis, and determining the color statistical features as the color features of tomatoes in the enhanced image set; and calculating the spatial relationship features of each tomato in the enhanced image set, wherein the spatial relationship features include adjacent distance, overlapping area, relative angle, and contact length, and determining the spatial relationship features as the stacking features of tomatoes in the enhanced image set.

[0031] Specifically, by performing multi-dimensional color statistical modeling on each tomato instance in the enhanced image set, based on the first moment (mean) representing the overall hue tendency and the second moment (variance) quantifying the color uniformity, a third moment (skewness) is introduced to describe the asymmetry of the color distribution, used to capture the highlight shift characteristics of the tomato surface caused by the directionality of light, and a fourth moment (kurtosis) to measure the sharpness of the color distribution, used to distinguish between ripe tomatoes with smooth surfaces and concentrated colors and unripe or damaged tomatoes with mottled surfaces and dispersed colors, thus constructing a four-dimensional color feature vector that can completely characterize the ripeness and surface quality of tomatoes. For the spatial configuration between stacked tomato instances, the Euclidean distance between adjacent instances is accurately calculated to assess the stacking tightness, the area of ​​the overlapping region to quantify the severity of occlusion, the relative azimuth angle to describe the contact orientation, and the linear length of the contact boundary to characterize the interaction strength. These four geometric measures are used as quantitative feature representations of the stacking morphology. This step transforms the subtle, ineffable sense of maturity and compactness of tomatoes into calculable and optimizable numerical features, enabling deep learning models to obtain explicit supervisory signals for high-level semantic features beyond pixel feature extraction. The introduction of high-order color statistical features allows the model to decouple the inherent color attributes of tomatoes from complex mixed lighting environments, effectively suppressing the interference of highlights and shadows on maturity judgment. The quantitative representation of spatial relationship features allows the model to leap from simple pixel classification to scene geometry understanding, accurately perceiving the mechanical contact state and spatial occupancy structure between tomatoes.

[0032] In an embodiment of this application, a tomato region recognition method based on deep learning is provided. The method involves constructing a deep learning model based on color features, stacking features, and instance annotations of an enhanced image set, and training and testing the deep learning model to obtain a tomato region recognition model. The method includes: determining the color features and stacking features of the enhanced image set and using these features as model input data; using instance annotations as model output data and constructing a dataset based on the model input and output data; dividing the dataset into training and testing sets according to a preset ratio, and inputting the training set into a preset graph neural network model for training to construct the deep learning model; dividing the dataset into training and testing sets according to a preset ratio, and inputting the training set into the deep learning model for training to obtain an initial tomato region recognition model; and inputting the testing set into the initial tomato region recognition model for testing until the initial tomato region recognition model meets a preset convergence condition to obtain the tomato region recognition model.

[0033] Specifically, the refined color and stacking features of each tomato instance in the enhanced image set are vectorized, concatenated, and normalized to form an instance-level feature vector with strong semantic expressive power, which serves as the model input. The pixel-level segmentation mask, maturity level label, and stacking relationship annotation of the corresponding instance are encoded as multi-dimensional output targets, thus constructing a structured dataset with feature-label as sample pairs. Subsequently, the dataset is randomly divided into training and testing sets according to a preset ratio, and a graph neural network is innovatively introduced as the core learning framework. Each tomato instance is defined as a graph node, and the contact and occlusion relationships between instances are defined as graph edges. The node features are the aforementioned color and stacking feature vectors, and the edge features encode relative distance and contact angle. Multiple layers of graph convolutional networks facilitate information transfer and feature aggregation between instances, enabling the model to perceive the overall configuration and local interactions of stacked groups in the global scene. During the training phase, the model simultaneously optimizes four sub-tasks—instance classification, bounding box regression, mask segmentation, and stacking relationship prediction—in an end-to-end manner, iteratively updating network parameters through backpropagation. After initial training, the model undergoes rigorous performance evaluation using a test set, monitoring key metrics such as intersection-union ratio, average precision, and stacking separation accuracy on unseen samples. Based on the validation results, hyperparameters are dynamically adjusted or training epochs are increased until the model's performance on the test set no longer shows significant improvement and converges stably, ultimately yielding a tomato region recognition model with strong generalization capabilities. This step extends the pixel-level features relied upon by traditional instance segmentation networks to a dual-driven paradigm of instance-level semantic features and relational features. The introduction of graph neural networks enables the model to not only focus on what each tomato is, but also to deeply understand how tomatoes are related. By explicitly modeling the spatial graph structure of stacked groups, the model can use the feature information of neighboring instances to help infer the contour of the occluded part and use the geometric constraints of contact edges to accurately locate the adhesion boundary, fundamentally overcoming the structural defects of convolutional neural networks, such as limited receptive field and difficulty in modeling long-distance dependencies.

[0034] In an embodiment of this application, a tomato region recognition method based on deep learning is provided. The method involves determining a real-time tomato unloading process image and inputting it into a tomato region recognition model to obtain the tomato region in the tomato unloading image. The method includes: determining the real-time tomato unloading process image and preprocessing it, including illumination normalization, contrast enhancement, and noise filtering; inputting the preprocessed real-time tomato unloading process image into the tomato region recognition model to obtain an initial tomato region; performing morphological optimization and stacking relationship segmentation on the initial tomato region, and superimposing the processing result onto the real-time tomato unloading process image to obtain the tomato region in the tomato unloading image.

[0035] Specifically, the raw images captured by the industrial camera undergo multi-level preprocessing. An illumination normalization algorithm maps the overall brightness and color temperature of the image to a standard distribution space, eliminating global color shifts caused by flickering artificial light sources, time-varying natural light, and the mixing of multiple light sources. Adaptive contrast enhancement technology, based on local histogram equalization, improves the texture visibility in shadow and overexposed areas, clearly revealing the contact boundaries and surface details of the stacked tomatoes. Non-local mean filtering or 3D block matching filtering algorithms effectively suppress Gaussian noise and impulse noise introduced by sensor thermal noise, conveyor belt vibration, and environmental dust, while fully preserving the sharpness and texture structure of the tomato edges. The preprocessed high-quality images are then input into a pre-trained... The tomato region recognition model outputs initial segmentation masks, maturity levels, and confidence scores for all candidate tomato instances in parallel through a single forward inference. Deep post-processing optimization is then applied to the initial segmentation results. Conditional random fields or morphological closing operations are used to fill small holes and cracks within the mask. A stacking relationship segmentation technique based on concave point detection and watershed algorithms is employed to perform secondary fine-tuning of the adhered regions in the initial mask, accurately restoring the independent boundaries and complete contours of each tomato instance. Finally, the optimized instance segmentation mask, instance number, maturity label, and stacking hierarchy information are rendered in real-time onto the original image in a pseudo-color overlay format, generating a visual recognition result and simultaneously outputting a structured data interface. This step constructs an end-to-end real-time processing pipeline from raw image acquisition to industrial decision output. The preprocessing module uses an adaptive correction strategy driven by a physical model to minimize the impact of complex and variable lighting environments and sensor noise on model inference performance, ensuring that the model can stably reproduce the accuracy level during training under different production lines, time periods, and climate conditions. The postprocessing optimization module effectively compensates for the prediction uncertainty of the model at extreme stacking and complex occlusion edges through the collaborative correction of geometric priors and depth prediction, improving the boundary fitting accuracy of instance segmentation.

[0036] like Figure 2As shown in the embodiments of this application, a tomato region recognition system based on deep learning is provided, comprising: an acquisition module, used to acquire historical tomato unloading process images, and perform hybrid enhancement of the historical tomato unloading process images with respect to unloading scene and dynamic lighting to generate an enhanced image set; a determination module, used to annotate the tomato colors and tomato stacks in the generated enhanced image set to obtain instance annotations of the enhanced image set, and determine the color features and stacking features of the tomatoes in the enhanced image set; a modeling module, used to construct a deep learning model based on the color features, stacking features and instance annotations of the enhanced image set, and train and test the deep learning model to obtain a tomato region recognition model; and a recognition module, used to acquire real-time tomato unloading process images, and input the real-time tomato unloading process images into the tomato region recognition model for output to obtain the tomato regions in the tomato unloading images.

[0037] In summary, this invention provides a deep learning-based method and system for tomato region recognition, comprising: acquiring historical images of the tomato unloading process and performing hybrid enhancement to generate an enhanced image set; annotating the generated enhanced image set to obtain instance annotations, and identifying the color and stacking features of the tomatoes within it; constructing a deep learning model based on the color features, stacking features, and instance annotations of the enhanced image set, and training and testing it to obtain a tomato region recognition model; determining real-time images of the tomato unloading process and inputting them into the tomato region recognition model for output, thereby obtaining the tomato regions in the tomato unloading images. This invention enhances the original image by simulating multi-dimensional real-world scenarios and integrates color and stacking features to construct a model, enabling tomato region recognition to adapt to dynamic lighting and stacking patterns, improving recognition accuracy, and solving the problem of identifying and segmenting individual and group tomato regions under complex unloading conditions.

[0038] Finally, it should be noted that those skilled in the art can obviously make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A tomato region recognition method based on deep learning, characterized in that, include: Acquire historical images of the tomato unloading process, and perform hybrid enhancement on these images to accommodate the unloading scene and dynamic lighting, generating an enhanced image set. The tomato colors and tomato stacks in the generated enhanced image set are labeled to obtain instance labels for the enhanced image set, and the color features and stacking features of tomatoes in the enhanced image set are determined. A deep learning model is constructed based on the color features, stacking features, and instance annotations of the enhanced image set. The deep learning model is then trained and tested to obtain a tomato region recognition model. The real-time tomato unloading process image is determined and input into the tomato region recognition model for output, thus obtaining the tomato region in the tomato unloading image.

2. The tomato region recognition method based on deep learning according to claim 1, characterized in that, The process involves acquiring historical images of the tomato unloading process and performing hybrid enhancement on these images, tailored to the unloading scene and dynamic lighting, to generate an enhanced image set, including: Images of the unloading process are collected from multiple fixed locations at the unloading port to construct a historical tomato unloading process image database, and historical tomato unloading process images containing tomatoes are obtained from the historical tomato unloading process image database; Identify the illumination features in historical tomato unloading process images, and simulate the image changes of tomatoes under different lighting conditions based on the illumination features to generate illumination-enhanced images; The motion features in historical tomato unloading process images were identified, and the image changes of tomatoes under different motion scene conditions were simulated based on the motion features to generate motion-enhanced images; The stacking features in historical tomato unloading process images were identified, and image changes of tomatoes under different stacking scenarios were simulated based on the stacking features to generate stacking-enhanced images; An enhanced image set is generated based on the illumination-enhanced image, motion-enhanced image, and stacked enhanced image, and noise is injected into the enhanced image set, including conveyor belt vibration noise, ambient dust noise, and illumination flicker noise.

3. The tomato region recognition method based on deep learning according to claim 2, characterized in that, The process of determining the illumination features in historical tomato unloading process images and simulating image changes of tomatoes under different lighting conditions based on these features to generate illumination-enhanced images includes: Illumination conditions were analyzed on historical images of tomato unloading processes, and illumination features of each image were extracted. Illumination features include average brightness, global contrast, color temperature distribution, shadow ratio, and highlight area ratio. A classification model for lighting conditions was established based on lighting features, and historical images of tomato unloading processes were divided into multiple lighting scene categories. For each lighting scene category, a generative adversarial network is used to learn the lighting distribution of that category and generate image variants that are consistent with the original image content but have different lighting conditions. The generated image variants are physically verified, and images that pass the verification are included in the illumination-enhanced image set.

4. The tomato region recognition method based on deep learning according to claim 2, characterized in that, The process of determining motion features in historical tomato unloading process images and simulating image changes of tomatoes under different motion scenarios based on these motion features to generate motion-enhanced images includes: Dynamic motion analysis was performed on historical images of tomato unloading processes to identify the motion states of the tomatoes in the images, including static, free fall, collision and bouncing, and rolling accumulation. The corresponding motion characteristics are determined based on the motion state. The motion characteristics include motion direction, instantaneous velocity, acceleration, and motion trajectory. A motion blur kernel is generated based on motion features, and the motion blur kernel is convolved with historical tomato unloading process images to simulate the motion blur effect of tomatoes during the unloading process. The generated blurred images are subjected to temporal continuity verification, and the images that pass the verification are included in the motion-enhanced image set.

5. The tomato region recognition method based on deep learning according to claim 2, characterized in that, The process of determining stacking features in historical tomato unloading process images and simulating image changes of tomatoes under different stacking scenarios based on these features to generate stack-enhanced images includes: Stacking contact condition analysis was performed on historical tomato unloading process images to extract stacking features between tomato instances. Stacking features include contact type, occlusion ratio, stacking level, and contact boundary morphology. A tomato instance template library is constructed based on historical tomato unloading process images. The tomato instance template library stores the individual tomato instance masks segmented from historical tomato unloading process images, as well as their corresponding texture and color information. Based on stacking features, multiple tomato instances are selected from the template library, and a synthetic stacked image with specified stacking density, contact angle, and occlusion relationship is generated on the actual unloading background through physical simulation rendering. The synthesized stacked images are checked for illumination consistency, and images that pass the check are included in the stacked enhanced image set.

6. The tomato region recognition method based on deep learning according to claim 1, characterized in that, The step of annotating the tomato colors and tomato stacks in the generated enhanced image set to obtain instance annotations for the enhanced image set includes: Tomato color grades are determined based on tomato maturity, and tomato colors in the generated enhanced image set are labeled based on these tomato color grades. The boundary of each tomato instance in the image set is determined by the boundary recognition algorithm, and the tomato stacks in the generated enhanced image set are labeled based on the boundary of the tomato instances.

7. The tomato region recognition method based on deep learning according to claim 1, characterized in that, The determination of the color features and stacking features of tomatoes in the enhanced image set includes: Calculate the color statistical features of each tomato in the enhanced image set. The color statistical features include mean, variance, skewness and kurtosis. The color statistical features are then used as the color features of the tomatoes in the enhanced image set. The spatial relationship features of each tomato in the enhanced image set are calculated. The spatial relationship features include adjacent distance, overlapping area, relative angle and contact length. The spatial relationship features are then determined as the stacking features of tomatoes in the enhanced image set.

8. The tomato region recognition method based on deep learning according to claim 1, characterized in that, The tomato region recognition model is obtained by constructing a deep learning model based on the color features, stacking features, and instance annotations of the enhanced image set, and by training and testing the deep learning model. This model includes: Determine the color and stacking features of the enhanced image set, and use the color and stacking features as input data for the model; Instance annotations are used as model output data, and a dataset is constructed based on the model input data and model output data; The dataset is divided into a training set and a test set according to a preset ratio, and the training set is input into a preset graph neural network model for training to build a deep learning model; The dataset is divided into a training set and a test set according to a preset ratio. The training set is then input into a deep learning model for training to obtain an initial tomato region recognition model. The test set is input into the initial tomato region recognition model for testing until the initial tomato region recognition model meets the preset convergence condition, thus obtaining the tomato region recognition model.

9. The tomato region recognition method based on deep learning according to claim 1, characterized in that, The process of determining the real-time tomato unloading process image and inputting it into the tomato region recognition model for output, to obtain the tomato region in the tomato unloading image, includes: The real-time tomato unloading process image was determined and preprocessed, including illumination normalization, contrast enhancement and noise filtering. The preprocessed real-time tomato unloading process image is input into the tomato region recognition model to obtain the initial tomato region; Morphological optimization and stacking relationship segmentation are performed on the initial tomato region, and the processing results are superimposed on the real-time tomato unloading process image to obtain the tomato region in the tomato unloading image.

10. A tomato region recognition system based on deep learning, characterized in that, include: The acquisition module is used to acquire historical images of the tomato unloading process and perform hybrid enhancement on these images, taking into account both the unloading scene and dynamic lighting, to generate an enhanced image set. The determination module is used to annotate the tomato colors and tomato stacks in the generated enhanced image set, obtain instance annotations of the enhanced image set, and determine the color features and stacking features of tomatoes in the enhanced image set; The modeling module is used to build a deep learning model based on the color features, stacking features, and instance annotations of the enhanced image set, and to train and test the deep learning model to obtain the tomato region recognition model. The recognition module is used to obtain real-time images of the tomato unloading process and input these images into the tomato region recognition model for output, thereby obtaining the tomato region in the tomato unloading image.

Images