PROCEDURE FOR MONITORING THE OPERATION OF A ROOT CROPS HARVESTING MACHINE AND MONITORING SYSTEM

Abstract

Method for monitoring the operation of a root crop harvester (1) which includes the following: a. obtaining images (15) from an image capture unit (11), of harvested material (16, 17, 18), which is transported in the root crop harvester (1). b. receiving the images (15) into one or more neural network algorithms (28) provided for applying segmentation and executing the one or more neural network algorithms (28) to analyze the received images (15) in order to detect at least one part of the harvested material (16, 17, 18) in the images (15); c. determining one or more harvest parameters (29, 30, 31, 32) associated with the detected at least one part of the harvested material (16, 17, 18). The invention further relates to a monitoring system (34) designed for carrying out this method and a harvester (1) equipped with such a monitoring system (34).

Description

2 nosinecure.Tothisend,inaharvester,oneormoreoperatingparametersaretypically adjustablebyaharvesteroperatorbytakingoperationactionsinordertoensurethat theharvesterremainsefficientinvariousoperatingcircumstances.Forexample,the speedofaconveyorbeltcanbesetand / oragitatorsand / oranintervalbetweencleaning unitsand / oraslopeofahedgehog,…5 Knownharvesterstypicallyincludeimagecapturingunitsforimagingoneormoreof theconveyingmeansintheharvester,withtheharvestedmaterialfedthereon.Images fromthesecapturingunitsarevisualisedonascreeninthecabinsothattheoperator isabletoseehowefficientlythedifferentconveyingmeansareoperatingandhowthe10 harvestedmaterialmovesthroughtheharvester.Basedonwhattheoperatorobserves ontheseimages,theoperatorcanadjustoneormoreoperatingparameters,suchase.g. aconveyorangle,aconveyingspeed,...Forexample,whentheoperatorobservesthat thereisalotofsoilonasievingwebofapotatoharvester,theoperatorcanincrease thesievingwebspeedsothatmoresoilfallsthroughthesievingweb.Byallowingthe15 operatortoadjusttheoperatingparametersbasedonwhatheseesontheimages,the harvestercanbesetmoreefficientinvariousoperatingcircumstances. Inordertoaidtheoperatorwiththisproblem,itise.g.inresearchprojectsande.g.in WO2021160607A1proposedtouseevaluationdevicesforautomaticallyanalysing20 imagesobtainedbysuchimagecapturingunitsfordeterminingrelatedharvest parametersandgeneratingoperatingparametersoftherootcropharvesterbasedon thisanalysis.Alsoine.g.EP3557972B1itisdescribedtoautomaticallyderiveoneor moreharvestparameterswithaprocessingunitanalysingsuchimages. Intheory,theoperatorwillbeabletomakemoreinformeddecisionsusingsuch25 harvestparametersandpossiblyoperatingparametersgeneratedinfunctionthereof,to improvetheefficiencyevenfurther.Suchharvestparameterscane.g.bethenumber ofrootcrops,thesizeofrootcrops,theamountofhaulmand / orclodswithinthe harvestedmaterial,qualityoftherootcrops,…Inpractice,however,thecomputer visionsystemswhichareproposedinthisrespecthaveamyriadofissueswhichmake30 themnotpracticallyusableinreal-lifeconditions.Mostimportantly,itisverydifficult BE2025 / 7042 3 toensurethatthecomputerprogramforanalysingtheimagesworksrobustlyinall kindsofoperatingcircumstances.Furthermore,theharvestedmaterialwhichis conveyedontheconveyingmeansisverydisorganisedandchaotic,makingitmore difficulttodetectduringoperation. 5 Inotherapplications,objectdetectionbyartificialintelligence(deeplearning)using boundingboxeshasbecomeverywidelyusedinimagedetectiontechniques.Deep learningorneuralnetworkalgorithmsconsistofvariouslayersofartificialneuronsor artificialnodesthatworktogethertolearnandprocessinformation,similartothe neuralnetworkofahumanmind.Thegrandadvantageofsaiddeeplearning10 algorithmscomparedtomoreclassiccomputervisionprogramsisthatthesedeep learningalgorithmsareself-learning,sothatthealgorithmsareabletoadapttovarious conditionsandcanbemorerobustinsituationswheretheoperatingcircumstancescan changeveryquickly.Withclassiccomputervisionprograms,perharvestparameterto detect,itmustbedecidedwhichfeaturesaretobedetectedinanimage.Thisquickly15 becomescumbersomewhenthenumberofharvestparametertobedetectedgrows. Dependentonthenumberoffeaturesused,lotsofparametersalsohavetobemanually fine-tunedbytheengineer. Thecommonlyusedneuralnetworkalgorithmsusingboundingboxes,aretrainedby feedingtheneuralnetworkalgorithmwithimagesinwhicheachobjectisinsidea20 knownlabelledboundarybox,whichisoftenrectangularorsquare.Thelabelled boundaryboxesareusuallynotactuallydrawnontheimages,butareinsteadprovided inaseparatedatafile,comprisingdatawhichreferstospecificpixelsoftheimage,so thattheneuralnetworkalgorithmcancorrectlyunderstandwhereeachboundarybox wouldbeplacedifitwasdrawnontheimage.Thealgorithmwilllearntoautomatically25 detectobjectsintheimagesthataresimilartotheobjectboxeswhichthealgorithmis trainedtorecognize.However,applicantnoticedthatapplyingthismethodfor detectingharvestedmaterialinrootcropharvestersispracticallynotthatfeasible. Althoughitispossibletodetecttherootcropsinidealcircumstances,thismethodis notrobustenoughtobeabletotracktherootcropsinallofthevariousoperating30 conditions.Especiallyduringwetconditions,whentherootcropsaremostlycovered BE2025 / 7042 4 bywetsoilthatstickstotherootcrops,thesealgorithmsarenotabletorobustlyand accuratelymakethedifferencebetweenaclumpofsoilandarootcropcoveredby soil.Anobjectofthepresentinventionistoprovideamethodformonitoringtheoperation5 ofarootcropharvesterbyautomaticallydetectingharvestedmaterialwhereinthis methodisbothmorepracticallyfeasibleandmoreefficientindetectingharvested materialthancurrentknownmethods. Thisobjectisfirstlyobtainedbyprovidingamethodformonitoringtheoperationofa10 rootcropharvester,comprising: a.obtainingimagesfromanimagecapturingunit,ofharvestedmaterial, conveyedonconveyingmeansintheharvester; b.detectingatleastpartoftheharvestedmaterialintheimages; c.determiningoneormoreharvestparametersassociatedwiththedetectedat15 leastpartoftheharvestedmaterial; whereinstepbcomprisesreceivingtheimagesinoneormoreneuralnetwork algorithmsprovidedforapplyingsegmentation,preferablysemanticsegmentation,for detectingtheatleastpartoftheharvestedmaterialintheimagesandcomprises executingtheoneormoreneuralnetworkalgorithmsforanalysingthereceivedimages20 inordertodetecttheatleastpartoftheharvestedmaterial.Byusingoneormoreneuralnetworkalgorithmsprovidedforapplyingsegmentation, itispossibletodetectatleastpartoftheharvestedmaterialinawaythatisbothmore practicallyfeasibleandmoreefficientthanwiththecurrentlyknownmethods.25 Thismethodutilisesoneormoreneuralnetworksalgorithms,withabovementioned advantagesoverclassiccomputervisionprograms. Theoneormoreneuralnetworkalgorithmsarenowmoreoverprovidedforapplying segmentation,whichdiffersalotfromtheearliermentionedboundingboxesmodel. Insteadofdetectingeachobjectindividually,thisneuralnetworkalgorithminstead30 detectsimageregionsbylocatingtheboundariesofsaidobjects,byselectingthepixels BE2025 / 7042 5 that‘belong’togetherinaregion.Fortheatleastpartoftheharvestedmateriala respectivesegmentationmaskisthusappliedtotheimage,thissegmentationmask specifying,forthisatleastpartoftheharvestedmaterial,alikelihoodthatthesceneat thepixelshowsanobjectbelongingtotheatleastpartoftheharvestedmaterial.These pixelswhobelongtogetherinaregionsharesimilarcharacteristicssuchascolouror5 brightnessandareseparatedfromotherpixelsbelongingtootherregionsby boundariesthataredetectedbytheneuralnetworkalgorithm.Thismethodis especiallyrobustcomparedtoothertypesofneuralnetworkalgorithmsandcompared toclassicalcomputervisionprograms,ande.g.makesitpossibletodetectrootcrops and / oroneormoreby-productssuchase.g.haulm,soil,stones,…invariousoperating10 circumstances.Dependingontheharvestparametertobedetermined,itise.g.possible todetectrootcropsand / oroneormoreby-productssuchase.g.haulm,soil,stones,… aswillbeexplainedfurtheron. Insteadofotherdetectionmethodssuchase.g.describedinEP3557972B1wherein detectionisbasedonspecificallychosenaspectsoftheharvestedmaterialtobe15 detected,suchase.g.shape,colourand / orsizedetection,usingsegmentation,nowa neuralnetworkalgorithmisused,whichistrainedwithlabelledimages(whereinthe harvestedmaterialtobedetectedislabelled),wherethisalgorithmitselfdecideswhich aspectstousefordetection.Thisensuresthatamuchmoregeneralapproachcanbe takenandthedatacanbeannotatedmoreaccurately,aslongassufficientdataare20 provided. Morepreferentially,theoneormoreneuralnetworkalgorithmswillbeprovidedfor applyingsemanticsegmentation.Semanticsegmentationisaspecifictypeof segmentationthatdoesnotdistinguishbetweentheindividualobjectswithinaregion.25 Thisnotonlyallowstheoneormoreneuralnetworkalgorithmstoworkmuchmore efficientlysinceitnolongerneedstorecognizeeachindividualobject,butitalso allowstodetectby-productssuchashaulmandsoilusingthismodelsincethereisno needtorecognizeeachindividual‘soil’or‘haulm’-particlebutitisinsteadonly necessarytorecognizeregionsof‘soil’andregionsof‘haulm’.Sinceeachindividual30 BE2025 / 7042 6 objectisnolongerrecognized,itisnolongerpossibletocounttheexactnumberof objects,butthisisquiteirrelevantforthehaulmandthesoil. Fortheoneormoresegmentationneuralnetworkalgorithms,preferentiallya convolutionalneuralnetworkalgorithmischosenandmorepreferablyaneural5 networkalgorithmwithacontractingpathandanexpansivepath,suchasforexample theU-Netalgorithm.Inregularneuralnetworks,neuronsaresplitintovariouslayers, suchastheinput / outputlayerandmultiple‘hidden’layerswhichapplyfiltersonthe inputdata,whichisinthiscaseaninputimage.Theneuronsofonelayerareall interconnectedtotheneuronsofthenextlayer.Incontrast,aconvolutionalneural10 networkalgorithmtakesadvantageofthelocalspatialcorrelationswithinanimage andinsteadonlylinkstheneuronsofonelayertothe‘locallyconnected’neuronsof thenextlayer,sothattheneuralnetworkalgorithmcanworkmuchmoreefficiently andrequireslessmemory.Theseconvolutionalneuralnetworkshavecharacteristic convolutionallayers,whichusemulti-dimensionalfilters(e.g.3x3,4x4)thatare15 convolvedovertheinputimage,sothatfeaturesintheimagesuchasedgesaredetected whilemaintainingthelocalcorrelationswithinanimage.Somefiltersdetectedges whileothersmightdetectcolours,textures,…Bylinkingseveraloftheseconvolutional layers,verycomplicatedfeaturescanbedetectedefficientlybytheconvolutional neuralnetworkalgorithm,thusmakingsuchaneuralnetworkalgorithmanexcellent20 choicefor(semantic)segmentation,aslongasthefeaturesarerelativelylocalwithin animage,suchasisthecasefortheharvestedmaterialwithintheharvester. TheU-Netalgorithmisaspecifictypeofconvolutionalneuralnetworkalgorithmthat isespeciallyfastandefficientforsemanticsegmentation.TheU-Netalgorithmhas twopaths:acontractingpathandanexpansivepath.Thecontracting“convolutional”25 pathisanormalconvolutionalneuralnetworkwithmultiplelayersthatrecognizes featureswhilereducingthespatialresolutionoftheinputimage.Afterthispath,afinal convolutionoperationisperformedtogenerateafeaturemap,containingthesemantic segmentationresults,withverysmallspatialresolutions.Thesecondexpansive decodingpaththenagainusesconvolutingfilterswhileincreasingthespatial30 resolutionsofthefeaturemapuntilafull-sizedfeaturemapisreturnedattheendof BE2025 / 7042 7 theneuralnetworkalgorithm.Inaddition,therearesomeadditionalconnections betweenequallysizedspatialresolutionswithinthefirstandsecondpath.Theseso- calledskipconnectionsensurethatwhilethesecondpathisup-scalingthefeaturemap, thefeaturemapremainsasaccurateaspossibletotheinputimage. 5 Inordertoadaptthisneuralnetworkalgorithmtothespecificapplicationwithinaroot cropharvester,theobtainedimageswillpreferablyfirstbepre-processedwithdata augmentationbyclassicmachineoperationstoimproveforexamplelightning,noise andsaturation.Additionally,theimagescouldberescaledand / orrotatedand / or croppedorsimilarlyedited.Thisdataaugmentationcanprovideasignificant10 improvementoftheperformanceoftheneuralnetworkalgorithm. Theimagecapturingunitcanbearegularcameraoravideocameraoranyothertype ofimagecapturingdevice.Preferentially,thisimagecapturingunitisadjustablebythe operatorandcanpanand / ortiltand / orzoominaccordancewiththepreferencesofthe15 operator. Sinceitcanbeverydarkwithintherootcropharvester,itisnecessaryforimage capturingunitswithshuttertimestoprovidesuchshuttertimessufficientlylongso thattheimagesarenottoodark.However,theshuttertimescanofcoursenotbetoo long,sinceotherwisetheimageswillbecomeveryblurry,whichwouldbedetrimental20 fortheperformanceoftheoneormoreneuralnetworkalgorithms.Inspecific embodiments,therootcropharvestercomprisessupplementalinternallightssothat theshuttertimesoftheimagescanbecomesmaller. Inspecificembodiments,theimagecapturingunitcomprisesadepthcameraora LiDARcamerasystem,sothatthecapturedimagesare3Dimages,whichprovidea25 lotmoreinformationthan2Dimages.These3Dimagesadditionallyprovidedepth infowhichallowsanaccuratevolumemeasurementtobetakenasanadditional harvestparametersothattheoneormoreneuralnetworkalgorithmsareabletomore efficientlydetecttheharvestedmaterial.Formostapplicationssuchdepthinformation willhowevernotberequired.Inotherspecificembodiments,theimagecapturingunit30 BE2025 / 7042 8 comprisesahyperspectralcamera,sothattheproductqualityoftherootcropscanbe determinedasaharvestparameter. Oncetheoneormoreharvestparametersaredetermined,theseharvestparameterscan becommunicatedtotheoperatoroftheharvesterviaauserinterface,theycanbeused5 byothersystemswithintheharvesterand / ortheycanbestoredwithinadatabank. Theoneormoreneuralnetworkalgorithmsarepreferentiallyexecutedbyoneormore processingunitsthatarespecialisedinhighperformanceforneuralnetwork applications,suchasforexampleacomputercircuitthatincludesgoodCPU10 performance,goodGPUperformanceandhigh-speedmemoryconnections.Sucha processingunitpreferentiallyalsoincludesoneormorehardwareand / orsoftware GPUaccelerators.Inspecificembodiments,suchaprocessingunitcanbeintegrated withintheimagecapturingunit.Preferentiallysuchaprocessingunitisalocalcontrol unitwhichisprovidedintherootcropharvester.15 Preferentially,therootcropharvesterisadjustablewithatleastoneadjustable operatingparameter.Forexample,thespeedofaconveyorbeltcanbesetand / or agitatorsand / oradistancebetweencleaningunitsand / oraslopeofahedgehog,… Themethodthenpreferablycomprisesgeneratingatleastoneoperatingparameter20 signalbasedontheoneormoreharvestparameters,foradjustingtheatleastone operatingparameteroftherootcropharvester. Thisatleastoneoperatingparametersignalcanforexamplebeanumberindicating howmuchtheatleastoneoperatingparametershouldbechanged.Evenmore preferably,themethodcomprisesautomaticallygeneratingthisatleastoneoperating25 parametersignalbasedontheoneormoreharvestparameters,foradjustingatleast oneoperatingparameteroftherootcropharvester.Theoperatingparametersignalcanincertainembodimentsbeprovidedtotheoperator oftheharvester,suchasforexampleviaagraphicaluserinterfaceorviaanaudibleor visualsignal,…allowingtheoperatortomakemoreinformeddecisionsastohowto30 adjusttheatleastoneoperatingparameter.Alternatively,atleastoneoperating BE2025 / 7042 9 parametercanbeadjustedbyacontrolunitthatadjuststheatleastoneoperating parameterbasedontheatleastoneoperatingparametersignal.Inaddition,thiscontrol unitpreferablyincludesmeansfortheoperatortooverridethecontrolunitsothatthe operatorisalwaysabletoadjusttheatleastoneoperatingparameter.Inaddition,the atleastoneadjustableoperatingparameterpreferablyremainswithinasafetyinterval.5 Thissafetyintervalcouldbeimmutableincertainembodiments,andcouldbe changeablebytheoperatorinotherembodiments.Preferably,theoperatorisnotified viatheaforementionedgraphicaluserinterfaceorviaaudibleorvisualsignalwhenan operatingparameterapproachesoneoftheendpointsofthesafetyinterval.10 Theobtainedimagesarepreferablytakenwithaspecificfieldofviewoftheimage capturingunit,i.e.aspecificshot.Suchfieldofviewcoulde.g.bealteredbyturning suchimagecapturingunitorbymovingitotherwisewithrespecttoatarget,orwhen suchtargetmoveswithrespecttotheimagecapturingunitorbyzooming,etc. Preferably,themethodcomprisesdeterminingfortheobtainedimagesadetectionsub-15 areawithinthespecificfieldofview,asaregionofinterestforfurtheranalysis,and theoneormoreneuralnetworkalgorithmsareprovidedfordetectingtheatleastpart oftheharvestedmaterialwithinthedetectionsub-areaasaregionofinterest. Determiningsuchdetectionsub-areainthisway(andallfurtherusesthereofas describedbelow)isalsousefulinalternativemethodsformonitoringtheoperationof20 arootcropharvester,wheretheimagesarenotsimilarlyreceivedinoneormoreneural networkalgorithmsforapplyingsegmentation,butareinsteade.g.analysedwith classicmachinevisionsystemsorwithothertypesofneuralnetworkalgorithms.Thepossibilityofchoosingadetectionsub-area,forexample,incaseswherethefield ofviewisadaptable,suchasbymovingtheimagecapturingunitand / orthesubject25 withrespecttotheimagecapturingunitand / orbyzooming,allowstheoperatorto chooseafieldofviewfortheimagecapturingunitthatgiveshimvisualinformation inthecabin,beyondtheinformationwhichisrequiredfortheanalysisoftheimages, e.g.bytheoneormoreneuralnetworkalgorithms.Choosingthedetectionsub-area ensuresthatthisdoesnotaffectthequalityoftheanalysisoftheoneormoreneural30 networkalgorithms.Thiscanforexamplebedesiredwhentheoperatorwantstohave BE2025 / 7042 10 afieldofviewwhereinaconveyorisobservedfromanangleeventhoughthisangle islesssuitablefortheoneormoreneuralnetworkalgorithms,whilethechosen detectionsub-areaiseasiertoanalysefortheoneormoreneuralnetworkalgorithms. Bychoosingadetectionsub-areaitisalsopossibletopurposefullyneglectazoneof thefieldofviewinwhichharvestedmaterialwouldnormallynotbepresentandwould5 thuswithhighcertaintynotbedetected.Itisfurthermoree.g.possibletopurposefully neglectthedetectionofharvestedmaterialatplaceswheresuchharvestedmaterialcan getstuck,suchasforexamplebesidestheelevators,sothattheharvestparametersare notadverselyaffectedbythis‘stuck’harvestedmaterial. 10 Inaddition,themethodfurtherpreferablycomprisesoneormoretransformationsof theobtainedimages,suchasaperspectivetransformationbasedonthedetectionsub- area,inordertoincreasetherobustness. Thisdetectionsub-areacaninsomeembodimentsbechosenbytheoperatororcanin someembodimentsbeautomaticallytrackedbyatrackingalgorithm.Thedetection15 sub-areacanbeanyshape,butispreferablyeitherdefinedbyalineordefinedbya polygonandmorepreferentiallyaquadrilateralsuchasarectangleoratrapezium.By choosingatrapezoidalshape,forexampleaperspectivetransformationcane.g.be appliedtothistrapezoidalshape. Additionally,themethodmorespecificallycomprisesdeterminingfortheobtained20 imagesmultipledetectionsub-areaswithinthespecificfieldofview.Theneachofthe multipledetectionsub-areascanbeusedforaspecificpurpose,suchasforexamplea firstdetectionsub-areabeingprovidedforaspecificneuralnetworkalgorithm,e.g.for detectingstuckhaulm,whileasecondsub-areaisprovidedforanotherneuralnetwork algorithm,e.g.fordetectingamountofsoil,orforexampletogivevisualinformation25 totheoperator.Themethodpreferablyfurthercomprisesselectingaspecificdetection sub-areaofthemultiplesub-areasasaregionofinterestbasedontheoneormore operatingparameters. Inspecificembodiments,theimagecapturingunitisprovidedforobtainingimagesof30 harvestedmaterialconveyedonahedgehogoftheharvester.Saiddetectionsub-area BE2025 / 7042 11 isthenpreferablysituatedatthetopofthehedgehog.Thisdetectionsub-areaisthen preferentiallydeterminedbyabaseline,whichcanmorespecificallybeprovided settablebytheoperatororcane.g.beautomaticallytrackedbyatrackingalgorithm. Suchbaselinecandefineonedetectionsub-areaataspecifiedsidethereof,orcan definetwodetectionsub-areasatbothsidesthereof.Multiplesuchbaselinescanbeset5 fordeterminingmultiplecorrespondingsub-areas.Insteadofprovidingabaseline,the detectionzonecane.g.bedeterminedintheimageasarectangleorasatopedgeof thehedgehog. Themethodfurtherpreferablycomprisesdeterminingsaiddetectionsub-areamore10 thanonce.Sincetheimagecapturingunitispreferablyadjustablebytheoperator,the fieldofviewoftheobtainedimagescanvarydependingonhowtheimagecapturing unitisadjusted.Inaddition,variousadjustmentstotheoperatingparameters,for exampletheangleofaconveyoroftheconveyingmeans,alsochangethefieldofview oftheobtainedimageswidely.Byallowingtodeterminethedetectionsub-areamore15 thanonce,thesevariousadjustmentsminimallyaffecttherobustnessandthe performanceoftheoneormoreneuralnetworkalgorithms.Tothisend,inspecificembodiments,afirstsaidsub-areaisdeterminedatafirstfield ofviewoftheimagecapturingunit,asecondsaidsub-areaisdeterminedatasecond fieldofviewoftheimagecapturingunit,themethodfurthercomprisesdetermining20 thespecificfieldofviewatwhichtheobtainedimagesaretakenanddeterminingthe detectionsub-areabasedon: -thedeterminedspecificfieldofview; -saidfirstsub-area;and -saidsecondsub-area.25 Morespecifically,wheretheimagecapturingunitisprovidedforobtainingimagesof harvestedmaterialconveyedonaconveyoroftheconveyingmeans,forwhichthe imagecapturingunitandthisconveyorarearrangedinanadjustablemutualposition, thefirstsub-areaisthenpreferablydeterminedatafirstmutualpositionandthesecond sub-areaatasecondmutualposition.Inordertodeterminethespecificfieldofview30 BE2025 / 7042 12 themutualpositionoftheimagecapturingunitandtheconveyoristhendeterminedat takingoftheobtainedimages.Preferably,theoneormoreneuralnetworkalgorithmsareprovidedfordetectingroot cropsasafirstsubpartoftheatleastpartoftheharvestedmaterialintheimagessuch5 thatwhenexecutingtheoneormoreneuralnetworkalgorithms,thereceivedimages areanalysedfordetectingthefirstsubpartoftheatleastpartoftheharvestedmaterial andfurthermorearootcropparameterisdeterminedasoneoftheharvestparameters associatedwiththedetectedfirstsubpart.Inembodimentswhereinatleastone operatingparametersignalisgenerated,theatleastoneoperatingparametersignalcan10 thenbegeneratedbasedontherootcropparameter.Possiblerootcropparametercan e.g.beanamountofdetectedrootcrops,asize,suchase.g.anaveragesizeofthe detectedrootcrops,anestimatedvolumeoftherootcropparameters,acolourofroot crops,aspeedoraccelerationoftherootcrops,thepresenceofrootcropsonaspecific areaoftheconveyingmeans,…15 Alternativelyand / oradditionally,theoneormoreneuralnetworkalgorithmsare preferablyprovidedfordetectinghaulmasasecondsubpartoftheatleastpartofthe harvestedmaterialintheimagessuchthatwhenexecutingtheoneormoreneural networkalgorithms,thereceivedimagesareanalysedfordetectingthesecondsubpart20 oftheatleastpartoftheharvestedmaterialandfurthermoreahaulmparameteris determinedasoneoftheharvestparametersassociatedwiththedetectedsecond subpart.Inembodimentswhereinatleastoneoperatingparametersignalisgenerated, theatleastoneoperatingparametersignalcanthenbegeneratedbasedonthehaulm parameter.Possiblehaulmparameterscane.g.beanamountofhaulm,anestimated25 volume,thepresenceofhaulmonaspecificareaoftheconveyingmeans,… Furtheralternativeand / orcomplementary,theoneormoreneuralnetworkalgorithms arepreferablyprovidedfordetectingsoil(whichmayalsoincludeclods)asathird subpartoftheatleastpartoftheharvestedmaterialintheimagessuchthatwhen30 executingtheoneormoreneuralnetworkalgorithms,thereceivedimagesareanalysed BE2025 / 7042 13 fordetectingthethirdsubpartoftheatleastpartoftheharvestedmaterialand furthermoreasoilparameterisdeterminedasoneoftheharvestparametersassociated withthedetectedthirdsubpart.Inembodimentswhereinatleastoneoperating parametersignalisgenerated,theatleastoneoperatingparametersignalcanthenbe generatedbasedonthesoilparameter.Possiblesoilparameterscane.g.beanamount5 ofsoil,anestimatedvolume,thepresenceofsoilonaspecificareaoftheconveying means,...Evenfurtheralternativeand / oradditionally,theoneormoreneuralnetworkalgorithms arepreferablyprovidedfordetectingoneormoreotherharvestedmaterialthanroot10 crop,haulmandsoilasafourthsubpartoftheatleastpartoftheharvestedmaterial, suchase.g.stones,orunexpectedobjectssuchase.g.afootball,etc.suchthatwhen executingtheoneormoreneuralnetworkalgorithms,thereceivedimagesareanalysed fordetectingthefourthsubpartoftheatleastpartoftheharvestedmaterialand furthermoreafurtherby-productparameterisdeterminedasoneoftheharvest15 parametersassociatedwiththedetectedfourthsubpart.Inembodimentswhereinat leastoneoperatingparametersignalisgenerated,theatleastoneoperatingparameter signalcanthenbegeneratedbasedonthefurtherby-productparameter.Possible furtherby-productparameterscane.g.beanamountofby-products,anestimated volume,thepresenceofby-productsonaspecificareaoftheconveyor,…20 Sincethefeaturesforeachtypeofharvestedmateriale.g.rootcrops,haulm,soil,… arequitedifferentfromeachother,eachtypeofharvestedmaterialwillpreferablybe detectedbyarespectiveneuralnetworkalgorithmoftheoneormoreneuralnetwork algorithms.Alternativelyhowever,asingleneuralnetworkalgorithmcanbeexecuted25 thatisprovidedfordetectingseveraltypesofharvestedmaterial. Preferably,theoneormoreneuralnetworkalgorithmsareprovidedfordetectingall typesofharvestedmaterial.Theoneormoreneuralnetworkalgorithmscanthusbe providedfordetectingthefirstsubpartandfordetectingthesecondsubpartand30 possiblyfordetectingthethirdsubpart,whensuchthirdsubpartispresentandpossibly BE2025 / 7042 14 fordetectingthefourthsubpartifoneormoreothertypesofharvestedmaterialare present.Atleastoneneuralnetworkalgorithmisthenpreferablyprovidedfordetecting thefirstsubpart,atleastoneneuralnetworkalgorithmfordetectingthesecondsubpart andpossiblyatleastoneneuralnetworkalgorithmfordetectingthethirdsubpartand possiblyatleastoneneuralnetworkalgorithmfordetectingthefourthsubpart.5 Theoneormoreneuralnetworkalgorithmsarepreferablyadditionallyprovidedfor applyingsegmentation,preferablysemanticsegmentation,fordetectingatleastpartof theconveyingmeansintheimagesandthemethodpreferablyfurthercomprises checkingwhetherforareceivedimagethedetectedfirstsubpart,thedetectedsecond subpart,possiblythedetectedthirdsubpart,possiblythedetectedfourthsubpartand10 thedetectedatleastpartoftheconveyingmeansamounttoapredeterminedpercentage ofthisreceivedimage. Preferablyatleastoneneuralnetworkalgorithmisspeciallyprovidedforthedetection oftheatleastpartoftheconveyingmeansintheimages.Alternativelyhowevera singleneuralnetworkalgorithmcanbeprovidedfordetectingtheatleastpartofthe15 conveyingmeansandoneormoretypesoftheharvestedmaterial. Bydetectingtheconveyingmeansandcheckingwhetherthesumofthedetected harvestedmaterialandtheconveyingmeansamounttoapredeterminedpercentage,it ispossibletotesttherobustnessofthedetectionswiththeoneormoreneuralnetwork algorithms.Especiallyinembodimentswhereinadetectionsub-areacanbe20 determinedsothatthedetectionsub-areacomprisessolelyconveyingmeansand harvestedmaterial,thesumofthepercentagesofthedetectedatleastpartofthe conveyingmeansandthedetectedatleastpartoftheharvestedmaterialwithinthis detectionsub-areashouldbeclosetoorapproximatelyequalto100%incasethatthe oneormoreneuralnetworkalgorithmsaresufficientlyrobust.Itispossiblethatthe25 sumofthesedetectedpartsisgreaterthan100%,e.g.somewheresomethingisseen bothasrootscropandhaulm,sothatthereisoverlapinthedetections.Itisalso possiblethatthesumislessthan100%.Ifthedeviationissmall,thedetectionscanbe consideredascorrect,whereastheycanbeconsideredasincorrectwithlarger deviations.Ifthecombinedpercentagedeviatesfrom100%itisalsoe.g.possibleto30 correctthepercentagesinproportionuntiltheyaddupto100%.Wherethedetection BE2025 / 7042 15 sub-areacomprisespartswhichcannotbeconsideredasconveyingmeansorharvested materials,suchase.g.ashieldplate,compensationcanbetakenintoaccountforsuch parts,e.g.takingalowerreferencepercentageforthesumofthedetectionsof conveyingmeansandharvestedmaterialaspredeterminedpercentage.Alsowhere conveyingmeansarenotdetectedand / orwhennotallharvestedmaterialisdetected,5 thiscanbetakenintoaccountwhendeterminingthepredeterminedpercentage. Detectingalloftheharvestedmaterialandatleastpartoftheconveyingmeansinan obtainedimageandcheckingwhetherthedetectedharvestedmaterialandatleastpart oftheconveyingmeansamounttoapredeterminedpercentageoftheimage,isalso usefulinalternativemethodsformonitoringtheoperationofarootcropharvester,10 wheretheimagesarenotsimilarlyreceivedinoneormoreneuralnetworkalgorithms forapplyingsegmentation,butareinsteade.g.analysedwithclassicmachinevision systemsorwithothertypesofneuralnetworkalgorithms.Furthermore,theoneormoreneuralnetworkalgorithmsarepreferablyadditionally15 providedforapplyingboundingboxesfordetectingasecondatleastpartofthe harvestedmaterialintheimages,suchthatwhenexecutingtheoneormoreneural networkalgorithms,thereceivedimagesareanalysedbyapplyingboundingboxesfor detectingthesecondatleastpartoftheharvestedmaterial.Thissecondatleastpartof theharvestedmaterialcanbethesameatleastpartoftheharvestedmaterialasdetected20 applyingsegmentation,orcandifferfromtheatleastpartoftheharvestedmaterialas detectedapplyingsegmentation. Preferablyatleastoneneuralnetworkalgorithmisspeciallyprovidedforapplying boundingboxesforthedetectionofthesecondatleastpartoftheharvestedmaterial intheimages.25 Thecombinationof(semantic)segmentationandboundingboxesisespeciallyuseful when(semantic)segmentationisusedtodetermineasaidharvestparameterassociated withoneormoreby-productsandwhereboundingboxesareusedtodetermineasaid harvestparameterassociatedwiththerootcrops.Eventhoughusingboundingboxes islessrobustandlessperformantthanusing(semantic)segmentation,usingbounding30 boxestheexactcountofrootcropscanbedetermined,whilewith(semantic) BE2025 / 7042 16 segmentationtheexactcountofrootcropscanonlybeapproximatedasitisdifficult todifferentiatebetweenoverlappingrootcrops.Sinceby-productscanhavehighly irregularshapes,itislesspreferredtouseboundingboxesinordertodetectsuchby- products. Alternativelyitise.g.possibletouseanalternativewayofsegmentation,suchase.g.5 instancesegmentationasanalternativeforapplyingboundingboxesinordertodetect thesecondatleastpartoftheharvestedmaterialifsomecountofrootcropsisrequired wheresemanticsegmentationisusedinordertodetectthefirstsaidatleastpartofthe harvestedmaterialinordertosavecomputingcapacitywhereanexactcountisof lesserimportance.Instancesegmentationisamorespecifictypeofsegmentationfor10 whichalsotheseparatedetectedinstancesaretobelabelled.Asaresulttheoutput doesnotonlyshowtheareawhereforexampletherootcropsaresituated,butalsothe numberofrootcrops.Inordertoperforminstancesegmentationincontrastwith semanticsegmentationanewneuralnetworkalgorithmneedstobetrainedandthe datarequiresfurtherlabellingoftheinstances.15 Theoneormoreneuralnetworkalgorithmsarepreferentiallytrainedwithlabelled imagesofharvestedmaterial,conveyedonconveyingmeansinarootcropharvester. Inorderforaneuralnetworkalgorithmtobetrainedforaspecifictypeofharvested materialorconveyingmeans,atleastthisspecifictypeofharvestedmaterialor20 conveyingmeansistobelabelled.Sincetheothertypesofharvestedmaterialcanbe ignoredwhilelabellingaspecifictypeofharvestedmaterial,thelabellingiseasierand faster.Otherelementsintheimagesarepreferablylabelledasnotbeingthisspecific typeofharvestedmaterialorconveyingmeans.Whereoneormoreneuralnetwork algorithmsaretobetrainedfordifferenttypesofharvestedmaterialand / orconveying25 means,inorderfortheoneormoreneuralnetworkalgorithmstobetrained successfully,examplesarepreferablyusedwhichhavebeenlabelledtoindicateallof thedifferentelementsontheimagessuchaswhereeachtypeoftheharvestedmaterial isandwheretheconveyingmeansare.Thislabellingcanbeperformedmanually,or canadditionallybeperformedsemi-automaticallywhereapreviousversionofaneural30 networkalgorithmtobetrainedisusedtolabeltheelementstobelabelled,which BE2025 / 7042 17 labellingbythepreviousversionoftheoneormoreneuralnetworkalgorithmsare thencorrectedmanually.Theoneormoreneuralnetworkalgorithmsitselfcanthenbe trainedinthecloudoronalocaldevice.Inspecificembodiments,theoneormoreneuralnetworkalgorithmsaretrainedin5 severaltrainingswithdifferentsetsoflabelledimagesfordifferentharvesting conditionsandtheoneormoreneuralnetworkalgorithmsarepreferablyprovidedfor detectingtheatleastpartoftheharvestedmaterialaccordingtosuchaharvesting condition.Themethodpreferablyadditionallycomprisesobtainingasaidharvesting condition,stepbthenfurthercomprisingreceivingtheobtainedharvestingcondition10 intheoneormoreneuralnetworkalgorithms,suchthatwhenexecutingtheoneor moreneuralnetworkalgorithms,thereceivedimagesareanalysedfordetectingtheat leastpartoftheharvestedmaterialindependenceoftheobtainedharvestingcondition. Theobtainedharvestingconditioncane.g.bethesalesmarketoftherootcropstobe15 harvested,suchaswhethertherootcropsaretobesoldfresh,aretobeprovidedfor preservationoraretobeprovidedforfurtherprocessinginafactory,.... Theobtainedharvestingconditioncanfurthermoree.g.reflectthedifferencebetween ‘wet’and‘dry’soil,and / orcanreflectdifferentsoiltypesand / ordifferent climatologicalcircumstances,changesinharvestrequirementsforspecificmarkets,20 differentcroptypesandcropvarieties,amountandsizeofstonesinground,inclination ofgroundand / orlightingconditionsinwhichharvestingcantakeplace,suchasa differencebetweendayornight,orcloudyorfoggy,etc.whichwillcauseimagestaken withimagecapturingunits(possiblyswitchingtonightview,infrared,blackandwhite, …)tovary,…althoughitisevenmorepreferentialtotraintheoneormoreneural25 networkalgorithmsinordertohavesufficientrobustnessinsuchsaidcircumstances. Itcanbeprovidedthattheharvestingconditionismanuallysubmittedbyanoperator inordertoobtaintheharvestingcondition.Alternativelyoradditionally,theharvesting conditioncanbeautomaticallydetectedbyadetectionalgorithmusingimagesfrom theimagecapturingunitand / oroneormoresensors,suchase.g.ahumiditysensor,a30 BE2025 / 7042 18 pressuresensor,….Alternativelyoradditionallytheoneormoreneuralnetwork algorithmscandetecttheharvestingcondition. Theoneormoreneuralnetworkalgorithmscanmorespecificallycomprisemultiple sub-algorithmsforthedifferentharvestingconditions.Alternatively,respectiveneural networkalgorithmsforeachofthedifferentharvestingconditionsmaybeused.In5 embodimentswherethemethodcomprisesthegenerationofatleastoneoperating parametersignalbasedontheoneormoreharvestparametersforadjustingtheatleast oneoperatingparameteroftherootcropharvester,theatleastoneoperatingparameter signalispreferablyfurtherbasedontheharvestingcondition,sothattheoperating parametersarealsodependentontheobtainedharvestingcondition,suchase.g.a10 qualityparameter. Furtherinspecificembodiments,theimagecapturingunitisprovidedforobtaining imagesofharvestedmaterialconveyedonaconveyoroftheconveyingmeans,and stepafurthercomprisesobtainingimagesfromasecondimagecapturingunitwhich15 isprovidedforobtainingimagesofharvestedmaterialconveyedonthesame conveyor.Stepbthenpreferablyadditionallycomprisessimilarlydetectingatleast partoftheharvestedmaterialintheimagesobtainedfromthissecondimagecapturing unitandsynchronisingthedetectionofatleastpartoftheharvestedmaterialinthe imagesfromthefirstsaidimagecapturingunitandthedetectionofatleastpartofthe20 harvestedmaterialintheimagesfromthesecondimagecapturingunit. Itismorespecificallypossiblethatstepafurthercomprisesobtainingimagesfroma third,fourth,…imagecapturingunit,althoughtwoimagecapturingunitsaremost preferential. Foranalysis,analysingimagesofoneimagecapturingunitcanbemuchsimpler,but25 anoperatorcane.g.preferhavingdifferentviewsofaconveyorforwhichdifferent imagecapturingunitsaretobeprovided.Insuchcase,theprovidedimagecapturing unitscanalsobeusedforthemethodaccordingtotheinvention. Moreover,byusingbothafirstandasecondimagecapturingunit,itise.g.possibleto havemultipleviewanglesattheconveyingmeans,sothattheharvestedmaterialcan30 bedetectedmoreaccurately.Ifonlyafirstimagecapturingunitisusedthenitisoften BE2025 / 7042 19 notpossibletodetectalloftheharvestedmaterialsincetherecanbesignificantoverlap ofthevariousharvestedproductswithinoneviewangle. Inordertosynchronisethedetectionofthefirstsaidimagecapturingunitandsecond imagecapturingunit,therearemultiplepossibilities.Themostpreferentialpossibility5 istodetecttheharvestedmaterialintheimageofthefirstimagecapturingunit separatelyfromtheharvestedmaterialintheimageofthesecondimagecapturingunit andthentocombinethetwodeterminedharvestparametersbyforexampletakinga maximumfunctionoftheseharvestparameters.Alesspreferentialpossibilityistofirst mergethetwoimagestoacombinedimageandthentoexecutetheoneormoreneural10 networkalgorithmstoanalysethiscombinedimage.Themaindrawbackwiththis methodisthatthetwoimagesdooftennothavethesamespatialresolutions,especially whendetectionsub-areasareusedorthatimagesaretakenfromadifferentangledue todifferentsettingsofthedifferentimagecapturingunits,sothatitisdifficulttomerge themintooneimage.15 Stillfurtherinspecificembodiments,thefirstsaidimagecapturingunitisprovided forobtainingimagesofharvestedmaterialconveyedonafirstconveyorofthe conveyingmeansandstepafurthercomprisesobtainingimagesfromanadditional imagecapturingunitwhichisprovidedforobtainingimagesofharvestedmaterial20 conveyedonanadditionalconveyor,differentfromthefirstconveyor.Stepbthen preferablyadditionallycomprisessimilarlydetectingatleastpartoftheharvested materialintheimagesobtainedfromthisadditionalimagecapturingunitandstepc additionallycomprisessimilarlydeterminingoneormoreadditionalharvest parametersassociatedwiththedetectedatleastpartoftheharvestedmaterialinthe25 imagesfromtheadditionalimagecapturingunit.Morepreferably,atleastone operatingparametersignalisgeneratedbasedontheoneormoreharvestparameters associatedwiththedetectedatleastpartoftheharvestedmaterialintheimagesfrom thefirstsaidimagecapturingunitandbasedontheoneormoreadditionalharvest parametersassociatedwiththedetectedatleastpartoftheharvestedmaterialinthe30 imagesfromtheadditionalimagecapturingunit. BE2025 / 7042 20 Preferably,themethodfurthercomprisesobtainingsensoroutputdatafromatleast onesensorintherootcropharvesterandtheatleastoneoperatingparametersignalis generatedbasedontheobtainedsensoroutputdata.Theatleastonesensorcan compriseapressuresensor,amoisturesensor,ananglesensor,awheelspeedsensor,5 atyrepressuresensor,avehiclespeedsensor,abrakepedalpositionsensor,a suspensionarticulationsensor,… Themethodfurtherpreferablycomprisescalibratingtheobtainedsensoroutputdata, eitherdependentorindependentoftheoperatingparameters.10 Theobjectoftheinventionisfurtherobtainedbyamonitoringsystemformonitoring theoperationofarootcropharvester,whereinthemonitoringsystemcomprisesmeans forcarryingoutthemethodasdiscussedabove. Themonitoringsystempreferablycomprisesoneormorelocalcontrolunitsonthe rootcropharvesterforexecutingtheoneormoreneuralnetworkalgorithms.In15 addition,themonitoringsystempreferablycompriseoneormorenetworkconnections whichallowaccesstoservices,datastorageand / orapplicationsviatheinternetorany othertypeofnetworksuchasforexamplealocalareanetwork.Byutilisingtheseone ormorelocalcontrolunits,itispossibletousethismonitoringsystemincircumstances wherethereisnogoodnetworkconnectione.g.inareaswithbadinternetconnection,20 andthusensuresthatthemonitoringsystemismorerobust.Additionally,theoneor morelocalcontrolunitsarepreferablyadaptedforusewithinarootcropharvesters, forexamplethattheyaretemperatureresistantand / orvibrationresistantand / orwater resistantand / ordustresistant.Anetworkconnectionallowsforanoperatorto downloadand / oruploadoneormoreoftheoneormoreneuralnetworkalgorithmsto25 and / orfromtheoneormorelocalcontrolunits. Thecontrolsystempreferablyfurthercomprisesoneormoredatabanks,whichcan storethedatawithinthemonitoringsystemsuchastheimagesand / orsensoroutput dataand / ortheoneormoreharvestparametersand / orsettingsofthegraphicaluser30 interface,…Thenetworkconnectionallowsforanoperatortodownloadand / or BE2025 / 7042 21 uploadthisdatatoand / orfromtheoneormoredatabanks.Thedatawhichisstored insuchadatabankcanberetrievedlater.Thesedatabankscancompriseanytypeof meansforstoringdata,e.g.flashdrives,Sdcards,harddrives,…Thedatabankscan bedirectlyintegratedwithintheuserinterfaceand / oroneormorelocalcontrolunits. 5 Inaddition,theobjectoftheinventionisobtainedbyacomputerprogram,comprising instructions,which,whenthecomputerprogramisexecutedonamonitoringsystem asdiscussedabovecausesthemonitoringsystemtocarryoutthemethodasdiscussed above.10 Furthermore,theobjectoftheinventionisobtainedbyacomputer-readabledata carrierhavingstoredthereonthecomputerprogramasdiscussedabove. Lastly,theobjectoftheinventionisobtainedbyaharvestercomprisingamonitoring systemasdiscussedabove.15 Thisharvesterpreferablycomprisesafirstimagecapturingunitandinaddition preferablyadditionalimagecapturingunits.Theharvesterfurthermorepreferably comprisesatleastonesensorwhichgeneratessensoroutputdata.Theharvester preferablycomprisesCAN-busconnectionsand / orethernetconnectionsbetweenits differentcomponents.Themonitoringsystemandtheoneormorelocalcontrolunits,20 ifpresent,arepreferablyadaptedtobeconnectabletotheCAN-busconnectionsand / or theethernetconnectionstotransmit / obtaintheimagesand / orthesensoroutputdata and / ortheoperatingparametersand / ortheoperatingparametersignals.Thepresentinventionwillnowbeexplainedinmoredetailbymeansofthefollowing25 detaileddescriptionofmethodsformonitoringtheoperationofarootcropharvester, correspondingmonitoringsystems,computerprograms,computer-readabledataand harvestersaccordingtothepresentinvention.Thesoleaimofthisdescriptionisto giveexplanatoryexamplesandtoindicatefurtheradvantagesandparticularsofthe presentinvention,andcanthusbynomeansbeinterpretedasalimitationofthearea30 ofapplicationoftheinventionorofthepatentrightsdefinedintheclaims. BE2025 / 7042 22 Inthisdetaileddescription,referencenumeralsareusedtorefertotheattached drawings,inwhichin: -Fig.1schematicallyillustratesapossibleembodimentofamonitoringsystem accordingtotheinvention,implementedinarootcropharvester(1);5 -Fig.2illustratesthepossiblefieldofviewofacamerapositionedabovea sievingconveyor,withpossiblesetdetectionsub-areasandwithpossible detectedharvestedmaterialandpossibledetectedconveyingmeansusing neuralnetworkalgorithmsofamonitoringsystemaccordingtotheinvention; -Fig.3illustratesthepossiblefieldofviewofacamerapositionedabovea10 hedgehog,withpossiblesetdetectionsub-areasandwithpossibledetected harvestedmaterialandpossibledetectedconveyingmeansusingneural networkalgorithmsofamonitoringsystemaccordingtotheinvention; -Fig.4schematicallyillustratesapossiblemonitoringsystemaccordingtothe invention.15 Figure1schematicallyillustratesapossibleembodimentofamonitoringsystem accordingtotheinvention,implementedinarootcropharvester(1).Sucharootcrop harvester(1)comprisesacrop-diggersectionwithe.g.oneormoreharvestingshares, whicharenotillustrated.Asievingunitcomprisessievingconveyors(3),haulmhooks20 (10)afeedingconveyor(4),ahedgehog(5)andaxialrollers(6).Thissievingunitis positioneddownstreamoftheharvestingsharesfortransportingrootcropstowardsthe ringelevator(7)andinthemeantime,tosievedirtfromtheharvestedrootcrops.With theelevator(7),therootcropsarethenliftedtowardsadischargeconveyor(8)ontoa transferelevator(9).Thefeedingconveyor(4)preferablydoesnothaveasieve25 functionwhenpassingthroughtheringelevator(7). Otherconfigurationsareofcourseconceivablewhereine.g.therootcropharvester(1) isprovidedwithabunkerand / orwithoutaxialrollers(6)and / orwithhaulmrollers and / orwithouthedgehog(5)and / orwiththeringelevator(7)beingpositionedfullyat30 BE2025 / 7042 23 thebackoftherootcropharvester(1)and / orwithoutsuchringelevator(7),and / or withadditionalorlessconveyors(3,4,8,9)and / orothertransfermeansetc. Thepartsoftherootcropharvester(1)canbeofanyknowndesign.Infigure1,theharvester(1)isprovidedwithdifferentcameras(11)asimagecapturing5 units: -2cameras(11)arepositionedabovethefirstsievingconveyor(3); -2cameras(11)arepositionedabovethethirdsievingconveyor(3); -2cameras(11)arepositionedabovethefeedingconveyor(4); -1camera(11)ispositionedabovethehedgehog(5).10 Acamera(11)canalsobepositionedabovethereadingtable(notillustrated). Preferablyatleast1camera(11)isprovidedpersaidmodule(3,4,5).2cameras(11) canbeplacedaboveasaidmodule(3,4,5)e.g.toprovideanoperatorwithamore pleasantview.Ifdesired,evenmorecameras(11)couldbeprovidedpermodule(3,4, 5).Alternatively,itisalsopossibletoprovide1camera(11)forseveralconveyors(3,15 4,5)nexttoeachother.Suchconveyors(3,4,5)arethenpreferablydrivenatasame speed.Itishoweveralsopossibletosplitanalysisoftheobtainedimages(15)for conveyors(3,4,5)whicharesetupnexttoeachotherbutwhicharerunningat differentspeeds.E.g.conveyorswhichfillanelevatorevenlycanbecontrolled separately.20 Withfewercameras(11)relatedcostswillbereduced.Withmultiplecameras(11) complexityofanalysisofcorrespondingimages(15)increases. Infact,theharvester(1)canbeprovidedwithasmanycameras(11)asdesired.In practice,preferablye.g.8to18cameras(11)canbeimplementedstandardlyanda customercouldrequestmorecameras(11)asanoption.Notallcameras(11)willthen25 necessarilybeusedformonitoring,somemightonlybepresenttoprovideanoperator withadesiredview,e.g.fordriving,suchasareversecameraorahaulmtopper camera,etc. Whereacamera(11)isprovided,thelocationthereofispreferablyascentralas possibleontherespectivemodule(3,4,5)abovewhichitisplaced.30 BE2025 / 7042 24 Therootcropharvester(1)isnowaccordingtotheinventionprovidedwitha monitoringsystem(34),whichappliesartificialintelligenceformonitoringthe operationoftherootcropharvester(1),thiswithoneormoreneuralnetwork algorithms(28)providedforapplyingsegmentation.5 Themonitoringsystem(34)ispreferablyprovidedsothatitcanbesethowmany cameras(11)arepresentsothattheoneormoreneuralnetworkalgorithm(28)will processimages(15)takingthissettingintoaccount.Themonitoringsystem(34)can beprovidedautomaticallytodetectwhichcameras(11)arepresentand / oranoperator couldsetwhichcameras(11)aretobetakenintoaccountusinge.g.auserinterface10 (13).Itcanthenpreferablybesetwhichcamera(11)ispresentatwhichpositioninthe harvester(1),thispreferablypermodule(3,4,5)andpossiblyalsowithinsaidmodule (3,4,5). Theoneormorecameras(11)couldbesupplementedwithotherimagecapturingunits15 (11).Inadditiontooneormoreimagecapturingunits(11)theharvester(1)is preferablyprovidedwithadditionalsensors(26),suchasoneormorepressuresensors and / oroneormoreanglesensorsand / oroneormorespeedsensorsand / oroneormore valvesettingsensors,…Thus,preferablyapressuresensorisprovidedonthefirst sievingconveyor(3).Thefillingoftheringelevator(7)couldalsobemonitoredusing20 animagecapturingunitorusingapressuresensor.Speedsensorsformonitoringthe speedoftheconveyorscouldfurtherbeprovided.Itisalsoe.g.possibletoprovidea moisturesensor. Furthermoreasshowninfigure1,themonitoringsystem(34)withintheharvester(1)25 isfurtherpreferablyprovidedwithauserinterfaces(13)withinthecabin(2)which allowstheoperatortomakesettingsand / oronwhichobtainedimages(15)and possiblyothersensoroutputdata(27)canbeshowntotheoperatorand / oronwhicha generatedoperatingparametersignal(33)canbepresentedtotheoperator,etc. Preferably,thecontrolsystem(30)withintheharvester(1)comprisesoneormore30 suchuserinterfaces(13).Suchuserinterfaces(13)cane.g.compriseamonitorand / or BE2025 / 7042 25 atouch-screenand / orakeyboardand / oradialand / orvoiceactivation,etc.Suchuser interfaces(13)canbeprovidedinthecabin(2)oftheharvester(1),butcouldalso comprisemobiledevicesand / orremotedevices. Themonitoringsystem(34)canfurtherbeprovidedwithanetworkconnection(14)5 whichallowsaccesstoservices,datastorageand / orapplicationse.g.forobtaining settingsviatheinternet.Incertainembodimentsofcontrolsystems(30)accordingto theinvention,therecouldalsobenonetworkconnection(14)ormultiplenetwork connections(14)presentwithinthecontrolsystem(30). Thecontrolsystem(30)isfurtherprovidedwithoneormoredatabankswhichcan10 storethedatawithinthecontrolsystem(30)suchasthesensoroutputdata(15,17) and / orthecurrentoperationactions(αt)and / orsettingswithauserinterface(13). Thesedatabankscancompriseanytypeofmeansforstoringdata,e.g.flashdrives, Sdcards,harddrives,… Theoneormoreuserinterfaces(13)and / ortheoneormorenetworkconnections(14)15 and / ortheoneormoredatabanksarepreferablyadaptedtobeconnectabletoadata connectionwithinthecontrolsystem(30)suchasaCAN-busconnectionand / oran Ethernetconnection. Themonitoringsystem(34)showninfigure1furthercomprisesalocalcontrolunit20 (12)forexecutingtheoneormoreneuralnetworkalgorithms(22).Preferably,the monitoringsystem(34)comprisesoneormoresuchlocalcontrolunits(12),also knownasaso-callededgedevice.Theoneormorelocalcontrolunits(12)are preferablyadaptedtobeconnectabletoadataconnectionwithinthemonitoringsystem (34)suchasaCAN-busconnectionand / oranEthernetconnection.Eachlocalcontrol25 unit(12)ispreferablyspecialisedinhighperformanceforneuralnetworkapplications, andpreferablycomprisesaprocessingunitwhichcontainsacomputercircuitthat includesgoodCPUperformance,goodGPUperformanceandhigh-speedmemory connections.Suchaprocessingunitpreferentiallyalsoincludesoneormorehardware and / orsoftwareGPUaccelerators.Inspecificembodiments,suchaprocessingunit30 BE2025 / 7042 26 canbeintegratedwithinanimagecapturingunit(11).Preferentiallysuchaprocessing unitisalocalprocessingunitwhichisprovidedwithintherootcropharvester(1). Formonitoringoftheoperationoftheharvester(1),images(15)areobtainedwithone ormoreofsaidpossiblecameras(11)intheharvester(1).Morespecificallye.g.25 images(15)persecond(thisispreferablysettable)canbetakenbyeachcamera(11) soasnottooverloadthelocalcontrolunit(12)whichisprovidedforprocessingsaid images(15). Asillustratedinfigure4,theimages(15)areloadedinoneormoreneuralnetwork algorithms(28)ononeormorelocalcontrolunits(12).Eachneuralnetworkalgorithm10 (28)isexecutedfordetectingatleastpartoftheharvestedmaterial(16,17,18)inthe images(15)andpossiblyfordetectingatleastpartoftheconveyingmeans(19). Suchacontrolunit(12)canbeintegratedwithintherespectivecamera(11)butis preferablyprovidedinthecabin(2).MorespecificallyaNvidiajetsoncanbechosen assuchcontrolunit(12).15 Images(15)takenbysuchcameras(11)aretakenwithaspecificfieldofview,which -asillustratedinfigures2and3-typicallycomprisespartsoftheharvester(1)which arenotrelevantforthedesiredanalysis,e.g.whentheoperatorwishestohaveaview onimages(15)takenbysuchcameras(11)whichisnotoptimalfortherequired20 analysis.Inordertosimplifytheanalysis,themonitoringsystem(34)isprovidedtoallowthe operatortosetviatheuserinterface(13)oneormoredetectionsub-area(20,21,22, 24)ofthespecificfieldofview,suchase.g.illustratedinfigures2and3.Alternatively and / oradditionallythemonitoringsystem(34)couldbeprovidedtohavesuch25 detectionsub-area(20,21,22,24)automaticallydeterminede.g.byaneuralnetwork algorithm. Infigure2,afirstdetectionsub-area(20)ise.g.defined,selectinganareaofthesieving conveyor(3)onwhichtheharvestedmaterial(16,17,18)isconveyed,tobeanalysed byoneormorecorrespondingalgorithms(28),byselectingfourcorners(23)defining30 aquadrangle.Peripheralareasoftheconveyor(3)whicharenotrelevantforthefurther BE2025 / 7042 27 analysisaretherebyexcluded.Aseconddetectionsub-area(21)isanalogouslydefined infigure2,byselectingfourcorners(23)onthehaulmhooks(10)abovethesieving conveyor(3). Infigure3,afirstdetectionsub-area(22)isanalogouslydefined,byselectingfour corners(23)onthehedgehog(5).Furthermoreinfigure3,aseconddetectionsub-area5 (24)isdefined,bysettingabaseline(25)onthehedgehog(5)abovewhichtheimage (15)ismorespecificallyfurthertobeanalysed.Thisseconddetectionsub-area(24)is therebysituatedatthetopofthehedgehog(5). Moreover,suchcameras(11)willtypicallybeadjustablebytheoperatorandcanpan and / ortiltand / orzoominaccordancewiththepreferencesoftheoperator.10 Additionally,oralternatively,itispossiblethatmodules(3,4,5)abovewhichsuch cameras(11)areplaced,canbesetindifferentpositions.Insuchcase,itisnot desirabletohavesaiddetectionsub-areas(20,21,22,24)foreachpossiblesettingof thecamera(11)and / ortherespectivemodule(3,4,5).Insuchcases,preferablya limitednumberofpossiblesettingsischosen,havingdifferentfieldsofview.Fora15 firstsaidfieldofview,afirstsaidsub-areaisthenset(e.g.afirstpolygon(22)asin figure3orafirstbaseline(25)asinfigure3)andforasecondsaidfieldofview,a secondsaidsub-area(e.g.asecondpolygon(22)asinfigure3orasecondbaseline (25)asinfigure3)isset.Thereafterateachactualsettingofthecamera(11)and / or eachactualsettingoftherespectivemodule(3,4,5)thespecificfieldofviewis20 determinedinrelationtosaidfirstfieldofviewandsaidsecondfieldofview.Forthis specificfieldofview,therespectivedetectionsub-area(20,21,22,24)isthennotset byanoperatorordeterminedbyatrackingalgorithm,butobtainedinfunctionofthe setfirstsub-areaandthesetsecondsub-area.Morespecificallye.g.forthehedgehog (5),afirstpolygonalsub-areacanbesetwiththehedgehog(5)setatafirstangle,a25 secondpolygonalsub-areacanbesetwiththehedgehog(5)setatasecondangleand ateachthirdangleatwhichsaidhedgehog(5)isset,determiningthespecificfieldof viewatwhichtheimages(15)aretakenforanalysis,thedetectionsub-areaisfurther determinedasapolygonbyinterpolationofthefirstpolygonalsub-areaandthesecond polygonalsub-area,inaccordancewiththeratioofthethirdangletothefirstangleand30 thesecondangle. BE2025 / 7042 28 Theimages(15)takenbythecameras(11)canbetakenwithdifferentshuttertimes.Suchashuttertime,alsoknownastheexposuretime,isthelengthoftimethatthe camera(11)isexposedtolightwhilecapturinganimage(15).Thisshuttertimecan preferablybeconfiguredbytheoperatorand / orcouldbeautomaticallydetermined5 basedonthebrightnessconditionswithinthemodule,e.g.viaalightsensor.Havinga slowershuttertimemakestheimages(15)morevaguewhiledisplayingthem,but increasestherobustnessoftheimages(15)invariousbrightnessconditions.Inspecific embodiments,itisalsopossiblethatthecameras(11)cyclebetweendifferentshutter times.Forexample,acamera(11)couldcapture2images(15)persecondwithlow10 exposuretimesand2images(15)persecondwithhighexposuretimes,sothatthe images(15)withlowexposuretimescanbeusedfortheneuralnetworkalgorithm (22)andtheotherimages(15)canbeusedfortheuserinterface(13). Theimages(15)(oronlyoneormoreofsaiddetectionsub-area(20,21,22,24)15 thereof,whichfortheanalysisthereofarefurthermorealsoreferredtoasimages(15)) arepre-processedwithdataaugmentationtoimprovee.g.lightning,noise,saturation, …Itise.g.possibletoswitchtonightview,infrared,blackandwhite,… Additionally,theimages(15)canberescaled(e.g.images(15)obtainwith704x400 pixelscanberesizeto208x208pixelsinordernottooverloadthecontrolunit(12))20 and / orrotatedand / orcroppedand / orsimilarlyedited. Usingoneormoreneuralnetworkalgorithms(28),theimagesofacamera(11)are analysedfordeterminingoneormoreharvestparameters(29,30,31,32)associated withthedetectedatleastpartoftheharvestedmaterial(16,17,18)andpossiblytaking thedetectedconveyingmeans(19)intoaccount.25 Afirstneuralnetworkalgorithm(28)cane.g.beprovidedfordetectingrootcrops(16) asafirstsubpartoftheharvestedmaterial(16,17,18)withinaspecificdetectionsub- area(20,21,22,24)andcanpossiblyalsobeprovidedfordeterminingoneormore rootcropparameters(29)basedthereon.Asecondneuralnetworkalgorithm(28)can e.g.beprovidedfordetectinghaulm(17)asasecondsubpartandpossiblyfor30 determiningoneormorehaulmparameters(30)basedthereonand / orathirdneural BE2025 / 7042 29 networkalgorithm(28)cane.g.beprovidedfordetectingsoil(18)asathirdsubpart andpossiblyfordeterminingoneormoresoilparameters(31)basedthereonand / ora furtherneuralnetworkalgorithm(28)cane.g.beprovidedfordetectingother harvestedmaterialthansaidrootcrops(16),haulm(17)orsoil(18)asafurthersubpart oftheharvestedmaterial(16,17,18)andfordeterminingoneormorefurtherby-5 productparameter(32)basedthereon.Anotherneuralnetworkalgorithm(28)cane.g. beprovidedfordetectingatleastpartoftheconveyingmeans(19)withinthespecific detectionsub-area(20,21,22,24).Itisalsopossibletohavetheoneormoreneural networkalgorithms(28)tosendthedetectedsubparts(16,17,18,19)toafurther algorithm(35)fordeterminingoneormorerespectiveharvestparameters(29,30,31,10 32)basedthereon,asillustratedinfigure4.Alternatively,itisalsopossibletohaveonesingleneuralnetworkalgorithm(28) whichisprovidedfordetectingseveraltypesofharvestedmaterial(16,17,18)and possiblyalsofordetectingconveyingmeans(19).Preferablyhoweverseparateneural networkalgorithms(28)areprovidedforeachofthedifferenttypesofharvested15 material(16,17,18)tobedetectedandfordetectingtheconveyingmeans(19)asthis simplifiesthetrainingofsuchneuralnetworkalgorithm(28).Should1neuralnetwork algorithm(28)beusedfordetectingalltypesofharvestedmaterial(16,17,18)and theconveyingmeans(19),thentheseshouldallbelabelledforallimages(15)used fortrainingpurposes,alsowhenretrainingisrequired.Usingdifferentneuralnetwork20 algorithms(16,17,18)forthedifferenttypes,allowstouseimages(15)withspecific conditionsand,forexample,onlytoretraintheneuralnetworkalgorithm(28)for detectingsoilonthoseimages(15).Otherwise,itwouldbeobligedtoretrainforall typesand,forthespecificimagesforsoil,alsotolabeltheotherdetectedtypes.25 Inordertotraintheoneormoreneuralnetworkalgorithms(28)images(15)arefirst collectedofharvestedmaterial(16,17,18)conveyedinsucharootcropharvester(1) andconveyingmeans(19)withwhichsaidharvestedmaterial(16,17,18)isconveyed inthisharvester(1).Theseimages(15)arethenlabelledfortraininganeuralnetwork algorithm(28)forapplyingsegmentation,thispreferablyonthefullfieldofview30 BE2025 / 7042 30 thereof.Onlyaftertraining,partsoftheimages(15)willbeexcludedbysettingofsaid detectionsub-area(20,21,22,24). Lightning,noiseandsaturationareusedasdataaugmentationmethodsduringtraining oftheneuralnetworkalgorithms(28)tomakethemodelsperformbetter.Dataused fortrainingcanbeartificiallyenlargedbye.g.changeimages(15)applyingsome5 classicaloperationsthereon,soase.g.mirroring,rotating,cropping... Labellingcanbedonemanuallyorsemi-automatically.Itisalsopossibletouseself- adversarialtraining. Partoftheneuralnetworkalgorithms(28)isprovidedforapplyingsegmentation,10 preferablysemanticsegmentation.Forsegmentation,themodelcane.g.bebasedon U-Netasdescribedabove. Someneuralnetworkalgorithms(28)canbeprovidedforapplyingboundingboxes. Forboundingboxes,themodelcane.g.morespecificallybebasedonyolo(youonly lookonce),butisthenpreferablyspecificallyadaptedforsmallitems.Theamountof15 filtersandtheamountofconvolutionallayerscanbechosenlessthantypicalforsuch modelsandarelativelysmallgridcanbesettodetectsmallerobjects. Itisalsopossibletoprovidesomeneuralnetworkalgorithms(28)forapplyingother typesofanalysis. 20 Inembodimentsofamonitoringsystem(34)oftheinventionwhereinallofthe harvestedmaterial(16,17,18)aredetectedandwhereintheconveyingmeansare detectedinthesetdetectionsub-area(20,21,22,24),itisfurtherpreferablyprovided tocheckwhetherforareceivedimage(15)thedetectedfirstsubpart,thedetected secondsubpart,thepossiblydetectedthirdsubpart,thepossiblydetectedfourth25 subpartandthedetectedatleastpartoftheconveyingmeans(19)amounttoa predeterminedpercentageofthisreceivedimage(15).Ifdeviationsaresmall,the detectionscanbeconsideredcorrect,whereastheycanbeconsideredincorrectwith largerdeviationsandpossiblyrescaledinordertoobtainthepredetermined percentage.30 BE2025 / 7042 31 Itisfurthermorepossibletotraintheneuralnetworkalgorithms(28)inseveral trainingswithdifferentsetsoflabelledimages(15)fordifferentharvestingconditions (37).Itisthenpossibleasillustratedinfigure4,tousee.g.sensorvalues(37)ofsensors (26)providedinthisrespectand / orparameters(37)readovertheinternet(14),which areameasureforsuchharvestingconditions(37),asfurtherinputfortheneural5 networkalgorithms(28)sothatthereceivedimages(15)canbeanalysedbythe respectiveneuralnetworkalgorithm(28)independenceoftheobtainedharvesting condition(37),fordetectingtheatleastpartoftheharvestedmaterial(16,17,18),e.g. with‘wet’ / ’dry’soilasharvestingconditionorclimatologicalcircumstancesas harvestingcondition,...Insteadofusingsuchsensorvalues(37)orparameters(37)10 readovertheinternet(14),itisalsopossibletoprovidethemonitoringsystem(34)so thatanoperatorcansetsuchharvestingconditionasuserinput(38)usingauser interface(13),e.g.withasalesmarketorqualityparameterasharvestingcondition,... Theharvestingconditioncandepend,forexample,onthedifferentsoiltypes,climatic conditions,drynessofthesoil,qualityrequired,...whereitisnowexplicitlypassedto15 theneuralnetworkalgorithms(28)asaparametertobetakenintoaccountinorderto analysetheimages(15)accordingly.E.g.whereat“harvestingcondition”Apotatoes arenicelywashedandat“harvestingcondition”Btheyareverydirty,different evaluationcriteriaforsegmentationareapplied. 20 Itisalsopossibletohavethefurtherdeterminationofharvestparameters(29,30,31, 32)e.g.bythealgorithm(35)providedinthisrespect,dependonsensorvalues(27) and / orparameters(27)readviatheinternet(14)and / orparametersenteredbythe operatorasuserinput(38),…asillustratedinfigure4.25 Itisfurthermorepossibletohavethefurtherdeterminationofharvestparameters(29, 30,31,32)dependontheanalysisofimages(15)ofdifferentcameras(11). Inthisrespectitise.g.possibletoprovideseveralimagecapturingunits(11),eachfor obtainingimages(15)ofharvestedmaterial(16,17,18)conveyedonasameconveyor (3,4,5,6,7,8,9)oftheharvester(1).Thesameoradditionalneuralnetwork30 algorithms(28)canthenbeprovidedforsimilarlydetectingatleastpartofthe BE2025 / 7042 32 harvestedmaterial(16,17,18)intheimages(15)obtainedfromthedifferentimage capturingunits(11),asillustratedinfigure4.Thedetectionsofatleastpartofthe harvestedmaterial(16,17,18)intheimages(15)fromthedifferentimagecapturing units(11)arethenpreferablysynchronised.Itisinthisrespecte.g.possibletocombine respectivelydeterminedharvestparameters(29,30,31,32)byforexampletakinga5 maximumfunctionoftheseharvestparameters(29,30,31,32).Alternativelyitise.g.possiblefirsttomergetheimages(15)ofthedifferentcameras(11)toacombined imageandthentoexecutetheoneormoreneuralnetworkalgorithms(28)toanalyse thiscombinedimage. Fordeterminingharvestparameter(29,30,31,32)dependentontheanalysisofimages10 (15)ofdifferentcameras(11),itisfurthermorepossibletoprovideafirstimage capturingunit(11)forobtainingimages(15)ofharvestedmaterial(16,17,18) conveyedonafirstconveyor(3,4,5,6,7,8,9)oftheharvester(1)andoneormore additionalimagecapturingunits(11)forobtainingimages(15)ofharvestedmaterial (16,17,18)conveyedononeormoreadditionalconveyors(3,4,5,6,7,8,9)ofthe15 harvester(1),differentfromthefirstconveyor(3,4,5,6,7,8,9).Thesameor additionalneuralnetworkalgorithms(28)canthenbeprovidedforsimilarlydetecting atleastpartoftheharvestedmaterial(16,17,18)intheimages(15)obtainedfromthe oneormoreadditionalimagecapturingunits(11),asillustratedinfigure4.Thesame oradditionalneuralnetworkalgorithms(28)canthenbeprovidedforsimilarly20 detectingatleastpartoftheharvestedmaterial(16,17,18)intheimages(15)obtained fromthedifferentimagecapturingunits(11),asillustratedinfigure4,beforesimilarly determiningoneormoreharvestparameters(29,30,31,32). Possiblerootcropparameter(29)cane.g.beanamountofdetectedrootcrops,asize,25 suchase.g.anaveragesizeofthedetectedrootcrops,anestimatedvolumeoftheroot cropparameters,acolourofrootcrops,aspeedoraccelerationoftherootcrops,the presenceofrootcropsonaspecificareaoftheconveyingmeans,… Possiblehaulmparameters(30)cane.g.beanamountofhaulm,anestimatedvolume, thepresenceofhaulmonaspecificareaoftheconveyingmeans,…30 BE2025 / 7042 33 Possiblesoilparameters(31)cane.g.beanamountofsoil,anestimatedvolume,the presenceofsoilonaspecificareaoftheconveyingmeans,... Possiblefurtherby-productparameters(32)cane.g.beanamountofby-products,an estimatedvolume,thepresenceofby-productsonaspecificareaoftheconveyor,… Obtainedharvestparameters(29,30,31,32)canbecommunicatedtotheoperatorvia5 asaiduserinterface(13)and / orcanbeusedbyothersystemswithintheharvester(1) and / orcanbestoredwithinadatabankand / orcanbesharedremotelyand / orcanbe usedtodetermineoperatingparametersignals(33),… Whentheharvester(1)isadjustablewithatleastoneadjustableoperatingparameter,10 themonitoringsystem(34)canfurtherbeprovidedwithanadditionalalgorithm(36) forgeneratingatleastoneoperatorparametersignal(33),basedontheoneormore harvestparameters(29,30,31,32)(associatedwithimages(15)fromoneormore imagecapturingunits(11))foradjustingtheatleastoneoperatingparameterofthe rootcropharvester(1),asillustratedinfigure4.15 Suchoperatorparametersignal(33)cane.g.beusedtoset: -thespeedofconveyingmeans(3,4,6,7,8,9); -agitators; -anintervalbetweensievingconveyors(3); -aslopeofahedgehog(5);20 -adjustingofthespindles; -dropheight; -adjustdifferentialbypassesofe.g.aso-calledflexyclean; -intensity,angle,position,pressureand / orspeedofahaulmrollerandincase ofablockagechangedirectionofrotationthereof;25 -adjustmentofaxialrollers; -openingadjustment,angle,rotationaldirectionand / orpositionofacleaning unit; -fanblowercontrolandintensity; -haulmtopperintensity,speedandpositioning;30 -heightofcropdiggingsection. BE2025 / 7042 34 Theoperatorparametersignal(33)canalsobebasedonsensorvalues(27)and / or parametersenteredbytheoperatorasuserinput(38),…asillustratedinfigure4. Itisthuse.g.possibletotakeintoaccountchangesinsoiltypeorsoilhumidityor changesinharvestrequirementsforspecificmarkets,ordifferentcroptypesandcrop varieties,amountandsizeofstonesintheground,inclinationoftheground,…5 Itise.g.possibletoprovidethemonitoringsystem(34)sothattheoperatorcanseta qualityparameter,thise.g.bothforthewholeharvester(1)ase.g.perpossiblemodule (3,4,5),sothattheoperatorcandeterminethequalitywithwhichhewishestoharvest. Thequalityparametercanbesetmanuallybytheuserintheuserinterface(13).This canbeageneraloverallqualityparameter.Furtheralternativeand / orcomplementary10 aqualityparametercanbesetforeachmoduleseparately.Theoperatingparameter signal(33)canthene.g.varydependingonthequalityande.g.theamountofhaulm detected. Asanextrasafety,themonitoringsystem(3)canfurthermorebeprovidedsothatthe operatorcansetaminimumvalueandamaximumvaluebetweenwhichtheoperating15 parametercanvary. Theoperatorparametersignal(33)canbeprovidedtotheoperatore.g.viatheuser interface(13)forsettingtheoperatingparameterand / orcanbeusedtoautomatically settheoperatingparameterbythemonitoringsystem(34).Inthelattercase,the monitoringsystem(34)ispreferablyprovidedwithapossibilityfortheoperatorto20 overridethesetting.Themonitoringsystem(34)oftheinventioncane.g.beusedtomonitorthehedgehog (5)inapotatoharvester(1). Inthisrespect,itise.g.possibletodetecttheamountofpotatoes(16)abovethe25 baseline(25)(seefigure3)e.g.usingboundingboxesand / ortodetectotherharvested material(17,18)and / ortheconveyingmeans(19)usingsemanticsegmentation.The differentharvestedmaterial(16,17,18)canbedetectedinseriesorparallel. Whenpotatoes(16)riseabovethebaseline(25),theycancollidewiththecounterroll, whichcausesdamage.Alternativelyorinaddition,ifthereisalotofsoil(18)above30 thebaseline(25),thennotenoughsoilcangetridof.Thehedgehog(5)canthenbe BE2025 / 7042 35 putmoreverticalandtothisend,ahedgehoganglesettingparametercanbegenerated asoperatingparametersignal(36).Thishedgehoganglesettingparameter(36)can e.g.furtherbedeterminedindependenceofthemoistureconditions.E.g.whereindry conditionstheangleofthehedgehog(5)istobeincreasedbasedonthedetectionof potatoes(16)abovethebaseline(25),inwetconditions,whenthereistoomuchdirt5 abovethebaseline,thisdetectionofpotatoes(16)canbedisregardedandthefillingof thehedgehogabovethebaseline(25)ingeneralcanbeusedtogeneratethehedgehog anglesettingparameter(36)instead. Itisalsopossibletouseacombinationoftheamountofhaulm(17)andsoil(18)10 detectedabovethebaseline(25)tosetthespeedofthehedgehog(5)andpossiblyalso settingthespeedofotherconveyingmeans(3,4,6,7,8,9)intheharvester(1).Inthis respectoneormorespeedsettingsignalscancorrespondinglybegeneratedas operatingparametersignal(36).E.g.whenthereisalotofsoil(18),oneormorespeeds canbeincreased,sothatmoresoilfallsthroughthesievingconveyors(3)andwhen15 hereislesssoil(18),suchspeedscanbelowered. Suchspeedsettingsignal(36)canfurthermoree.g.bedetermineddependentone.g. saidqualityparameter,setbytheoperator.E.g.whenqualityislessimportant,cleaning canbemaximisedwithhigherspeeds,whereaswhenqualityismoreimportant, cleaningisminimisedwithlowerspeeds.20 Preferablyalsothepressureofthefirstsievingconveyor(3),e.g.detectedusinga pressuresensor(26),istakenintoaccountforgeneratingsuchspeedsettingsignal(36) forsettingthespeedofthehedgehog(5)orotherconveyingmeans(3,4,6,7,8,9). Furthermore,itispossibletotrackwhetherdetectedpotatoes(16)remainhanging abovethebaseline(25),e.g.becausehaulm(17)gotstucktosuchpotato(16).Insuch25 case,itise.g.possibletoshowapopuponascreen(13)ortogeneratesomeother alert,toalerttheoperatoraccordingly. Themonitoringsystem(34)oftheinventioncanfurthermoree.g.beusedtomonitor thethirdsievingconveyor(3)(orasecondorafurthersievingconveyor(3)).30 BE2025 / 7042 36 Intheillustratedembodimenttwocameras(11)areprovidedabovethisthirdsieving conveyor(3).Agitatorsforthisconveyor(3)arenotseparatelycontrollable.Inorder tocontrolbeatingoftheconveyor(3)withtheagitators,thesoil(18)onthisthird sievingconveyor(3)cane.g.bemonitoredforgeneratinganagitatoractivationsignal asoperatingparametersignal(36).Sincethesidewiththemostsoilisthemostlikely5 toclog,inordertomonitorthesoil(18)onthisthirdsievingconveyor(3)the maximumofsoil(18)detectedwiththeleftcamera(11)andthemaximumofsoil(18) detectedwiththecamera(11)totherightcane.g.betakentodeterminethefinalsoil percentageforgeneratingtheagitatoractivationsignal(36).Inadditiontothesoil(18) alsotheamountofdetectedconveyingmeans(19)canbeusedtogeneratetheagitator10 activationsignal(36).Furthermoresuchagitatoractivationsignal(36)canalsobe determinedindependenceofsaidqualityparameter,setbytheoperator.E.g.when qualityislessimportant,cleaningcanbemaximisedwithhigheragitation,whereas whenqualityismoreimportant,cleaningisminimisedwithloweragitation.In addition,itisalsopossibletousethedetectionofhaulm(17)stucktopotatoes(16)on15 thehedgehog(5)asmentionedabovetocheckwhetheragitatorsaretobeactivatedin generatingsuchagitatoractivationsignal(36).Potatoes(16)withhaulm(17)stuck thereoncanmoreeasilybeseenonthehedgehog(5). Acombinationofdetectedsoil(18),amountofdetectedconveyingmeans(19)and haulm(17)detectedcane.g.beusedtogenerateaspeedsettingsignal(36)forsetting20 thespeedofthisthird(orsecondorfurther)sievingconveyor(3)andpossiblyother conveyingmeans(3,4,6,7,8,9).Asmentionedabove,preferablyalsothepressure ofthefirstsievingconveyor(3),e.g.detectedusingapressuresensor(26)and / ore.g. asetqualityparameter,…istakenintoaccountforgeneratingsuchspeedsetting signal(36).Forgeneratingaspeedsettingsignal(36)forsettingthespeedofthe25 hedgehog(5)preferablyalsotheamountofhaulm(17)onthefeedingconveyor(4)is takenintoaccount.Whenthereisalotofhaulm(17)thehedgehog(5)canthusbe movedfastersoitcanbeprocessed.Itisalsopossibletousethemonitoringsystem(34)oftheinventione.g.tomonitor:30 -thehaulmhooks(10),similartothethirdsievingconveyor(3); BE2025 / 7042 37 -thehaulmhooks(10)tocheckwhetherhaulm(17)getsstuckthereon,inorder e.g.togenerateapopuponascreen(13)ortogeneratesomeotheralert,to alerttheoperatoraccordingly; -thereadingtable,e.g.todeterminethesizeofpotatoes(16)enteringthebunker, e.g.usingboundingboxes,e.g.forclassificationintodifferentclassesand / or5 todetermineyield,… Itisfurthermoree.g.possibletodetectablockageintheharvester(1)usingthe monitoringsystem.E.g.ifhaulm(17)isdetectedwhichremainsconstantlydetected inthesameplaceoveracertaintime,thiscanbedetectedasablockageanda10 correspondingoperatingparametersignal(33)canbegeneratedand / oranotification canbepresentedtotheoperator. Themonitoringsystem(34)caninsomeembodimentsbemodularlyprovided.Itis e.g.possibletoprovideafirstmoduleformonitoringand / orsettingofthehedgehog,15 asecondmoduleformonitoringand / orsettingofasaidsievingconveyor,athird moduleformonitoringand / orsettingofthehaulmroller,afourthmodulefor monitoringthereadingtable,etc.Eachofthesemodulescanthenbeseparately purchasable.Eachofthesemodulescanthenbesupplemental.Itisthene.g.also possibleforeachofthemodulestosavedeterminedharvestparametersinadatabank,20 consultablebytheothermodules,sothattheseharvestparameters(29,30,31,32)can alsobeusedbytheothermodules. Thei.