MOBILE ANALYSIS AND PROCESSING DEVICE

Description

[0001] The invention relates to a mobile analysis and processing device for agriculture for cultivating the soil and / or for manipulating flora and fauna according to claim 1 and to a method for real-time control of the cultivation of the soil and / or the manipulation of flora and fauna by the device according to claim 14.

[0002] Weed control in agriculture is a very labor-intensive task, especially in organic farming, which prohibits or restricts the use of chemicals. Depending on the crop, weed control is necessary in the immediate vicinity of the crop. This control generally takes place in the early growth stage. Both the crop and the weeds are still very small and close to each other. To avoid damaging the crop, it is advisable to use selective methods. In organic farming, for example for carrots, this is achieved through labor-intensive, physically damaging manual work with so-called "Weeding planes". Seasonal workers lie on their stomachs on a cot and remove the weeds.

[0003] For special crops with larger plant spacing, such as sugar beet or lettuce, tractor-mounted implements are known that are capable of detecting individual crops and controlling the corresponding tools so that the crop area remains untreated. No selectivity is necessary for this task. This means that these systems do not check the areas to be treated, but rather the tool. "on blind" based on the known crop position. Generally, the distance to the crop defines the accuracy requirements.

[0004] From DE 40 39 797 A1 a device for weed control is known, whereby an actuator for destroying the weeds runs continuously and is only interrupted briefly when a cultivated plant is detected by a sensor.

[0005] DE 10 2015 209 879 A1 discloses a device for damaging weeds, which comprises a processing tool. This processing tool serves to damage the weeds. A classification unit is also provided, which either has the position data of the weeds or recognizes the weeds and determines the position data. A localization unit determines a relative position between the processing tool and the weeds. A manipulator unit in the form of a carriage positions the processing tool accordingly depending on the determined relative positions.

[0006] A corresponding device with a pressure conveying unit and a liquid dispensing unit is known from DE 10 2015 209 891 A1. In this embodiment, the weeds are destroyed by spraying liquid under pressure.

[0007] From DE 10 2015 209 888 A1 it is known to apply the liquid to the weeds in pulsed manner and thus damage them.

[0008] DE 10 2013 222 776 A1 discloses a stamp mounted in a carriage, which is arranged in a guide device for guiding the stamp. The stamp is placed on the weeds and pressure is applied. The pressure destroys the weeds.

[0009] WO 2018 / 033925 A1 discloses a system and method for managing a drone fleet for harvesting and fertilization. The technical application of this subject matter lies in the field of harvesting and pruning fruit, vine, and tall vegetable plants. Its use in the technical field for field plants / outdoor plants and their cultivation is expressly considered unsuitable and will not be pursued.

[0010] WO 2017 / 181127 A1 describes robotic plant care systems and methods that deliver a clearly detectable signal, namely a biomarker, to a target plant for plant care. This biomarker is particularly biodegradable. Genetic modifications with fluorescent proteins, dyes, or labels / markers are suitable for this purpose, allowing differentiation from other vegetation through machine scanning. This is intended to represent plant areas that are not recognized as target plants as weeds or adjacent foliage and can be sprayed or cut with hydraulic hoes.

[0011] New approaches are currently being explored with automated agricultural robots and harvesting machines equipped with telematics to provide technical support for agriculture. Technical principles and insights from space travel, remote sensing, and robotics can often be applied to agricultural applications, but these must be specifically adapted to the specific tasks facing agriculture and require new devices and processes.

[0012] For example, the existing automated agricultural robots mentioned above are designed to travel only one row of plants at a time. Interventions are made only in the flora, and only in a serial fashion. Inspections are usually carried out afterward by a walkthrough, for example, by a qualified human.

[0013] Another disadvantage of the known devices is that they are special designs of trolleys which only travel through the crops row by row and are relatively inflexible in their use.

[0014] The invention is based on the object of providing a mobile analysis and processing device for agriculture for cultivating the soil and / or manipulating flora and fauna, as well as a method for the device, which enables real-time, controlled, qualified removal of the detected flora and / or fauna, as well as parallel analysis of the flora and fauna. Preferably, the device should be connectable to different supports that move the device to and across the site of use.

[0015] This object is achieved for the mobile analysis and processing device by the features of claim 1 and for the method by the features of claim 14.

[0016] The subclaims form advantageous developments of the invention.

[0017] The invention is based on the finding that by creating a mobile device independent of a carrier, which has all the units for analysis and processing, the flexibility of use and the resulting possibilities are considerably increased.

[0018] The invention therefore relates to a mobile analysis and processing device for agriculture for soil cultivation and / or for manipulating flora and fauna. The device comprises the features of claim 1.

[0019] This device creates mobility and flexibility, forming a single unit that processes all data in real time, generates control signals for the sensor, tool unit, and / or actuator, and can be immediately processed according to the control signals. This opens up possibilities for combination, for example, with various carriers that move the device across the field as needed.

[0020] To easily replace individual components and thus reduce setup times, the device is divided into two parts. The first and second units can be connected via the data exchange interface. Furthermore, the two units can be spatially separated from one another. This is advantageous, for example, when the weight of the device's moving parts needs to be kept as low as possible. In this case, the second unit could be fixed centrally, while the first unit could be moved around the field.

[0021] The housings protect the components in the units from external influences.

[0022] The first and second housings are detachably connected via a plug-in connector. This allows for modular assembly of the two units, as well as easy replacement in the event of one unit failing.

[0023] Preferably, the visual detection unit comprises a segmentation and data reduction device. The visual detection unit may comprise a classifier that evaluates the compared recording data against existing defined data sets in the database, particularly with the support of artificial intelligence.

[0024] According to one embodiment of the invention, the sensor is a visual detection unit with a camera. The data to be processed is thus image data, which can be easily compared with a database.

[0025] In order to be able to connect the device, if necessary, to a carrier, in particular one designed as a flying drone, which moves the device, corresponding means for connecting to the carrier are provided.

[0026] According to one embodiment of the invention, the first and second housings comprise receptacles as means for connecting to the carrier as needed, which are associated with corresponding holding means of the carrier, via which the device can be grasped and moved by the carrier. Alternatively or additionally, the first and second housings can comprise coupling means as means for connecting to the carrier as needed, which are associated with corresponding coupling means of the carrier, via which the device can be connected to the carrier and moved. This enables simple and quick connection to a carrier for transporting the device.

[0027] The tool unit preferably has at least one feed unit and one rotation unit that interacts with the motor. This allows the tool's application range to be easily expanded without requiring the device to be moved.

[0028] Preferably, the rotation unit is provided at a distal end with at least one tool, in particular a milling cutter or a blade unit. Rotation of the blade unit, for example, can selectively destroy small insects or weeds.

[0029] To further reduce the weight of the device, a voltage connection for an external power supply is provided. The voltage connection can be provided on the first unit. When the first unit and second unit are assembled, the second unit can supply voltage to both the first unit and the second unit via this voltage connection. Preferably, the voltage source of a carrier is used for the voltage supply.

[0030] In order to enable data exchange between a wearer and the device, a further communication interface is provided on the device for the wearer.

[0031] The additional communication interface can be arranged in either the first or the second unit. Preferably, it is provided in the second unit.

[0032] The above-mentioned object is also achieved by a method for real-time control of the cultivation of the soil and / or the manipulation of flora and fauna by the device according to the type just mentioned, the method comprising the steps of claim 14.

[0033] Preferably, once the control and / or regulation data are available, the motor, the actuator, the tool unit and / or an associated carrier are started up to work the soil and / or to manipulate flora and fauna.

[0034] According to a preferred method according to the invention, the evaluation is performed in a second computer interacting with the classifier, and the processing and conversion of the evaluation into control and / or regulation data is performed in the first computer, for which purpose the evaluation is transmitted from the second computer to the first computer. This reduces computing time because computers can operate in parallel. Furthermore, it is possible for the two computers not to be located next to each other. For example, the second computer with the second unit can be located remotely from the first unit with the first computer.

[0035] The storage, qualitative comparison of the recorded data with data stored in the database, and / or the evaluation by the classifier are preferably supported by artificial intelligence. This allows the creation of a virtually autonomously operating system. The device, in particular the first unit and the second unit, can be modular in design, allowing them to be connected to each other but also to other units of an overall system.

[0036] Real time refers to the ability to perform analysis and processing operations in situ in one operation.

[0037] In the context of the invention, a voxel is understood to be a spatial data set that is generated discretely or continuously over time via the sensor or a visual detection unit in an imaging process.

[0038] The actuator preferably has a mechanism, in particular a rotary unit, which is located in a holder in the housing of the first unit.

[0039] Further advantages, features and possible applications of the present invention will become apparent from the following description in conjunction with the embodiments shown in the drawings.

[0040] In the description, claims, and drawings, the terms and associated reference symbols used in the list of reference symbols below are used. In the drawings: Fig. 1 is a schematic representation of a support system with spatially separated housings of a mobile device according to a first embodiment of the invention; Fig. 2 is a schematic representation of a support system with spatially separated housings of a mobile device, which are connected to one another via a plug connection, according to a second embodiment of the invention; Fig. 3 is a side view of the support system according to the first embodiment of the invention with the mobile device connected to an aerial drone; Fig. 4 is a flowchart illustrating the steps of a method with the support system; Fig. 5 is a flowchart illustrating the steps of the method for determining the necessary measures; Fig. 6 is an image recorded by the visual detection unit; Fig. 7 is a schematic structure of a convolution neural network based on the image of Fig. 6; Fig. 8 a flowchart showing a method of the segmentation and data reduction unit; Fig. 9 an intermediate image created from the segmentation and data reduction unit; Fig. 10 a partial section of the intermediate image with three different cases for the classifier; Fig. 11 two basic representations of further pixel fields for evaluation by the classifier; Fig. 12 two basic representations of further pixel fields for evaluation by the classifier; Fig. 13 an image created and evaluated by the classifier, and Fig. 14 the basic representations of the operation of the classifier.

[0041] In Fig. 1A carrier system 10 is schematically shown, which consists of a carrier in the form of a drone 12 and a mobile device 14 for cultivating the soil and manipulating flora and fauna in agriculture. The drone 12 comprises a drive 16, which includes four electric motors 18 and propellers 20 driven by them, see Figure 3 . In addition, the flying drone 12 is provided with four feet 22 below the electric motors 18.

[0042] According to the first embodiment of the invention, the drone 12 comprises an energy source in the form of batteries 24, which supplies the energy for the drive 16 as well as for the other components of the drone 12 and the mobile device 14. For this purpose, a voltage interface 26a is provided on the drone 12 and a voltage interface 26b corresponding to this voltage interface 26a is provided on the mobile device 14, which are connected to one another via a detachable plug connection 28. In addition, a communication unit 30 with an antenna 32 and a GPS unit 34 is provided, which continuously determines the location of the drone 12 and transmits this location data of the drone 12 to the mobile device 14 for association with the data acquired by the mobile device 14, as well as to a remote central processing unit, which is not shown in detail here.Telemetry can be performed using the GPS unit 34, the communication unit 30, and the mobile device 14. A control device 12b is also provided, which controls the drive 16.

[0043] The communication unit 30 of the drone 12 comprises, in addition to the antenna 32, a further interface 36a, which is assigned to an associated interface 36b of the mobile device 14 and is connected to one another via a detachable plug connection 38 for data exchange.

[0044] The mobile device 14 consists of two units 14a, 14b, namely a first unit 14a with a first housing 40 and a second unit 14b with a second housing 42. The first housing 40 and the second housing 42 are detachably connected to one another via a plug connection 44 to form a unit that forms the mobile device 14. There is a set of different first units 14a on one side and a set of different second units 14b on the other side, which can be individually configured and adapted to requirements by simply connecting them together.

[0045] The first housing 40 contains a first computer 46, an actuator in the form of a movable, motor-driven arm 48, a motor 50 interacting with the arm 48, and a tool unit 52 arranged on the arm 48, which tool unit has a feed unit 54 and a rotation unit 56. A milling cutter 58 is provided as a tool at the distal end of the rotation unit 56. The motor 50 drives both the arm 48 and the feed unit 54, the rotation unit 56, and thus also the milling cutter 58. The arm 48 can be constructed in several parts and have various joints, which are not shown in detail, since such motor-driven kinematics are known.The tool unit 52 is moved relative to the drone 12 to its area of use via the arm 48, so that the tool unit 52 with the feed unit 54 and the rotation unit 56 can use the milling cutter 58 to work the plants, for example removing weeds, and / or to work the soil.

[0046] Furthermore, the first unit 14a includes a communication unit 60 and a visual detection unit 62. The visual detection unit 62 comprises a camera 64 for capturing images, a segmentation and data reduction device 66, and a classifier 68 that classifies multiple pixel fields consisting of pixels based on an intermediate image or intermediate data generated by the segmentation and data reduction device 66, as described in more detail below. The visual detection unit 62 is connected to the communication unit 60.

[0047] The first unit 14a has an interface 70a, which is assigned to an interface 70b of the second unit 14b. Via a communication connection 72, the communication unit 60 is connected via the interface 70a to the interface 70b and, via the interface 70a, to a communication unit 74 in the second unit 14b. The communication unit 74 of the second unit 14b is connected via the interface 36b via the plug connection 38 to the interface 36a and the communication unit 30 of the drone 12.

[0048] In the second unit 14b, a second computer 76 and a database 78 are also provided.

[0049] In the Fig. 2A further embodiment of the carrier system 10 is shown, wherein the flying drone 12 is identical to the first embodiment. Only the mobile device 14 differs by a plug connection 80 between the first unit 14a and the second unit 14b, which also detachably connects the communication unit 60 of the first unit 14a to the communication unit 74. By simply plugging them together, different first units 14a can be combined with different second units 14b and assembled into a mobile unit 14.

[0050] In Fig. 3The flying drone 12 is shown in a side view, whereby only two of four electric motors 18 with associated propellers 20 are visible. The feet 22 are arranged below the electric motors 18. Between the feet 22 are two gripping arms 82a, 82b, which grasp and lift the mobile device 14 as needed, and then lower and set it down again. The mobile device 14 consists of the two units 14a and 14b, which are detachably connected to one another via the plug connection 80. In the first unit 14a, the camera 64 can be seen as part of the visual detection unit 62, as well as the milling cutter 58 at the distal end of the rotation unit 56.

[0051] The mobile device 14 can also be equipped with several different tool units 52, each of which is provided with a common arm 48 and, for example, a tool turret that moves the required tool unit 52 into the activation position. However, it is also conceivable that the different tool units each have their own actuator.

[0052] In Fig. 4 A flow chart shows the steps that are carried out one after the other in order to carry out soil cultivation and the manipulation of flora and fauna in agriculture using the carrier system 10.

[0053] In a first step 84, the necessary measures on the assigned agricultural area are first determined using the carrier system 10. For example, the carrier system 10 is brought to an agricultural area to be cultivated, for example an agricultural field, or flown there directly from a central location. The drone 12 with the mobile device 14 takes off and flies over the agricultural field. The carrier system 10 receives the necessary data about the agricultural field to be inspected via a stationary central processing unit. The central processing unit can also be a smartphone. The agricultural field is captured in images via the visual detection unit 62 with the camera 64 of the mobile device 14. The images are evaluated, and the necessary measures for this agricultural field are ultimately determined by comparing them in the database 78.

[0054] In a next step 86, on the basis of the measures determined for the agricultural field or for sub-areas of the agricultural field, the mobile unit 14 suitable for the necessary measure is assembled from a set of first units 14a and a set of different second units 14b, and the two units 14a, 14b are connected to one another.

[0055] In a next step 88, the mobile device 14 with the drone 12 is gripped laterally by the gripper arm 82a and the gripper arm 82b, respectively, and moved upwards toward the drone 12 into a receptacle 12a of the drone 12. The voltage interfaces 26a, 26b are connected to one another via the plug connection 28, and the interfaces 36a, 36b are connected to one another via the plug connection 38. This supplies the mobile device 14 with voltage from the battery 24 of the drone 12, enabling data exchange via the antenna 32 of the communication unit 30 of the drone 12 with the communication units 60 and 74 of the mobile device 14 on the one hand and a central processing unit on the other hand. As explained above, the central processing unit, which is independent of the carrier system 10, can also be a smartphone.

[0056] In a next step 90, the determined measures are carried out on the agricultural field using the carrier system 10. For example, the drone 12 flies to the area of the agricultural field to be worked. The arm 48 with the tool unit 52 moves to the weed to be removed. The feed unit 54 moves the tiller 58 to the weed so that it is milled away by activating the rotation unit 56.

[0057] In a fifth step 92, the drone 12 then flies back and exchanges the mobile device 14 for another mobile device 14 which is optimized for a different measure, for example with a spreading device for pesticides or fertilizer.

[0058] Alternatively, steps 86 and 88 may be omitted if the drone 12 is already fully equipped for the measure to be carried out.

[0059] Based on the Figure 5The determination of the necessary measures by the carrier system 10, in particular by the mobile device 14, is now explained in detail.

[0060] In a first step 94, the visual detection unit 62 of the mobile device 14 continuously records data of technically defined voxels and / or pixels and / or images. The voxels, pixels, and images form recording data, which are continuously transmitted to the database 78 - second step 96.

[0061] In a third step 98 the recording data is saved.

[0062] In a fourth step 100, a qualitative comparison of the recording data with the data stored in the database 78 is performed. In this case, the recording data is segmented and reduced by the segmentation and data reduction device 66. In particular, the recording data can also be verified by the second computer 76.

[0063] In a fifth step 102, the evaluation is carried out by the classifier 68 in conjunction with the second computer 76 with the support of artificial intelligence, as will be explained in more detail below.

[0064] Finally, in a sixth step 104, the evaluation is processed and converted by the first computer 46 into control and regulation data for the motor 50, the arm 48, the tool unit 52 and the drone 12.

[0065] Finally, in a seventh step 106, the motor 50, the arm 48 and the tool unit 52 are started up to work the soil or to manipulate flora and fauna.

[0066] Whenever artificial intelligence is mentioned in this application, it refers, among other things, to the use of a classic convolutional neural network (CNN) consisting of one or more convolutional layers followed by a pooling layer. This sequence of convolutional layers and pooling layers can, in principle, be repeated any number of times. Typically, the input is a two- or three-dimensional matrix, e.g., the pixels of a grayscale or color image. The neurons in the convolutional layer are arranged accordingly.

[0067] The activity of each neuron is calculated using a discrete convolution (convolutional). Intuitively, a relatively small convolution matrix (filter kernel) is moved step by step over the input. The input of a neuron in the convolutional layer is calculated as the inner product of the filter kernel and the current underlying image section. Neighboring neurons in the convolutional layer react accordingly to overlapping areas.

[0068] A neuron in this layer only responds to stimuli in a local environment of the previous layer. This follows the biological model of the receptive field. Furthermore, the weights for all neurons in a convolutional layer are identical (shared weights). This means that, for example, each neuron in the first convolutional layer encodes the intensity of an edge in a specific local area of the input. Edge detection, as the first step in image recognition, has high biological plausibility. It follows directly from the shared weights that translation invariance is an inherent property of CNNs.

[0069] The input of each neuron, determined by discrete convolution, is then transformed by an activation function, in CNNs usually a Rectified Linear Unit, or ReLu for short (f(x) = max(0, x), into the output, which is intended to model the relative firing frequency of a real neuron. Since backpropagation requires the calculation of gradients, a differentiable approximation of ReLu is used in practice: f(x) = In(1 + ex< ). Analogous to the visual cortex, both the size of the receptive fields and the complexity of the recognized features increase in deeper convolutional layers.

[0070] In the next step, pooling, superfluous information is discarded. For object detection in images, for example, the exact position of an edge in the image is of negligible interest – the approximate location of a feature is sufficient. There are various types of pooling. By far the most common is max pooling, in which only the activity of the most active neuron (hence "max") from each 2 × 2 square of neurons in the convolutional layer is retained for the subsequent computation steps; the activity of the remaining neurons is discarded. Despite the data reduction (75% in the example), pooling generally does not reduce the network's performance.

[0071] The use of the convolutional neural network and the segmentation and data reduction device 66 is explained in more detail below using Figures 6 to 14.

[0072] There are various approaches to classifying all objects in an image using the classifier 68. Many approaches begin by first finding the individual objects in the image and then classifying them. However, this is not always possible. As an example, the classification of plants 108 in a field will be used here. An example image 108 shows Fig. 6 .

[0073] In Fig. 6 Various plants 108 are depicted, and they are to be classified by the classifier 68 in real time. In this case, the real time is the camera rate of 10 frames per second. Since, as in this example, it is not easy to distinguish exactly where a plant 110 ends, a different approach must be used, as the computing time is insufficient to first distinguish the plants 110 themselves and then classify them.

[0074] Picture 108 of Fig. 6consists of pixels, and each pixel can logically contain only one class. Therefore, a trivial approach would be to classify the entire image 108 pixels at a time. This means that each pixel is assigned to a class in turn.

[0075] However, since a single pixel does not contain the necessary information to make a statement about the class membership, a surrounding area must be used for classification. This area can then be classified using a convolution neural network (CNN), as described above. The network can generate a sequence such as Fig. 7 have.

[0076] The input image 110 is the image of Fig. 6. The elements of the CNN are now applied to this input image 110. In this example, this would be the convolution 112 with the features, a subsequent pooling 114, another convolution with additional features, another pooling, and a summary in the dense layer 116. The output of the network then outputs the class membership of the middle pixel of the input image 110, or of a pixel of the image 110 of Fig. 6 .

[0077] A new image section, usually shifted by one pixel, is then selected and classified again using the CNN. This procedure results in the calculations required by the convolutional neural network being repeated for the number of pixels to be classified. This is time-consuming. Image 110 of Fig. 6has a resolution of 2000 x 1000 pixels. The CNN would therefore have to be computed two million times. However, the initial problem is only the classification of the 108 plants themselves. On average, such an image contains about 5% plant pixels, which corresponds to only about 100,000 pixels.

[0078] By means of simple segmentation and data reduction by the segmentation and data reduction unit 66, it can be determined whether a pixel represents a part of a plant 108 or background 118. This segmentation is not as computationally complex as a CNN and is therefore faster. The segmentation and data reduction by the segmentation and data reduction unit 66 is carried out analogously to the Fig. 8 . In Fig. 8 the individual steps are shown.

[0079] In a first step 120, each image of several pixels transmitted to the database 78 is converted into the RGB (Red, Green, Blue) color model.

[0080] In a next step 122, each pixel of the transmitted image is transferred into an HSV (hue, saturation, value) color model based on the RGB color model.

[0081] In a next step 124, this HSV color model is evaluated.

[0082] Each pixel based on the HSV color model is evaluated for color saturation using a threshold value, where if the color saturation value exceeds a threshold, the pixel is assigned the binary value 1, and if the color saturation value falls below a threshold, the pixel is assigned the binary value 0.

[0083] In parallel, each pixel is evaluated for hue based on the HSV color model using a predetermined range, where if the hue is within the predetermined range, the pixel is assigned the binary value 1, and if the hue is outside the range, the pixel is assigned the binary value 0.

[0084] In a next step 126, an intermediate image is generated from the binary information of the color angle and the color saturation, which contains considerably less data than the image 108 generated by the camera.

[0085] From the segmentation Fig. 8 The formula given below is applied to each pixel. The segmented image S(x,y) is created by dividing the RGB image ψ ( x , y) into its three components red, green and blue. A pixel of the segmented image is then set to 1 if the minimum value of red, green or blue of a pixel divided by the green pixel is less than or equal to a threshold value ( THs ) . The threshold value in the 8-bit space of the image is determined by scaling it by 255. If the threshold value is not reached, the pixel of the segmented image is set to 0, as in equation 1. S x , y = 1 if MIN φ R ca ′ x , y . φ G acn x , y φ Blqc x , y φ G acn x , y ≤ 255 − THs 255 0 otherwise .

[0086] This results in the first optimization: Before the entire image 108 is divided into two million images, the segmentation is carried out according to Fig. 8 , is applied. This means that the entire image 108 is scanned and, using the formula given above, a decision is made as to whether it is a plant pixel or not. First, the image 108 is segmented, meaning that the background 118 is set to black (0), as shown in Fig. 9is shown. Second, if it's a plant pixel, its coordinates are written to a list. Subsequently, only the coordinates that are also in this list are fed into the CNN. The unnecessary pixels of the earth, i.e., the background 118, are eliminated. This reduces the number of CNN calls by about 20.

[0087] Through segmentation, the background 118 is set to the value 0. The image elements considered by the CNN now also have segmented images. Normally, the feature calculation would be applied to each pixel of the image element in a convolution layer. However, this results in three cases 128, 130, and 132 for the calculation, which are shown in Fig. 10 are shown, each for a feature 134 of size 5x5.

[0088] The red case 128 shows a feature calculation in which the feature lies entirely on the background 118. Here, each element is multiplied by 0, resulting in the entire calculation yielding 0, or the bias value. The result of this calculation is therefore already known before the calculation. Even if the background 118 were not zero, i.e., contained soil, this calculation would not contain any information about the plant 110, so the result can simply be a constant fictitious value.

[0089] In the yellow case of 130, the mean feature value is not on a plant of 110. This means that part of it is also a multiplication by zero. This case consumes the plant of 110 in the edge area and makes this size in the feature map.

[0090] In the blue case 132, at least the middle pixel of the feature lies on a plant.

[0091] After considering these three cases 128, 130, and 132, only the yellow and blue cases 130 and 132 need to be calculated. These are the cases 130 and 132 in which the feature has at least one input value non-zero. The results of all other feature calculations are known before the calculation; they are zero or only have the bias value. The coordinates at which the blue case 132 occurs are known. These are the coordinates that are saved during segmentation. The yellow case 130, in turn, requires a calculation to determine whether this has occurred. This requires checking each plant pixel found in the segmentation. Since this check is too complex and the yellow case 130 only occurs in the edge area of a plant 110, this case should be ignored.

[0092] Therefore, the calculation can be optimized so that the feature calculation and all other elements of the CNN are only applied to the found plant pixels.

[0093] In Fig. 11 A schematic representation of how two neighboring plant pixels 136, 138 differ is shown. On the left is a plant pixel 136, and on the right is the neighboring plant pixel 138. The blue / purple area 140, 142 could be different plants that need to be classified. The red / purple area 144, 142 represents the image element that the CNN considers to classify the orange pixel 146.

[0094] Upon closer inspection, it can be seen that the observed area (red / purple) 144, 142 overlaps significantly. This means that both image elements 136, 138 contain largely the same values. If the CNN now calculates the feature in the convolution layer, the same values would also be calculated from the feature calculation.

[0095] In Fig. 12A feature 148 with the size 5 x 5 pixels is sketched schematically in green. This feature 148 is located at the same coordinates within the entire image, but it is shifted within the image element (red / purple) 144, 142 to be viewed by the CNN. However, since the location is the same throughout the entire image, the calculation for the middle black box 150 would result in the same value in both the left image 136 and the right image 138. This finding can be applied to all elements of a CNN. This means that, provided the edge area is ignored, the individual feature calculation can first be applied to the entire image. Theoretically, the decomposition of the input image 108 only plays a decisive role with the dense layer 116. However, the dense layer 116 can be calculated in the same way as a convolution 112. The feature size results from the interaction of the input image size and the existing pooling layers in the network.This allows the classification to be further optimized; the CNN elements are now applied only to the detected plant pixels. The feature map calculated from the final convolution represents the classification result, as shown in . Fig. 13 Here, all carrot plants 152 are shown in green and all weeds 154 are shown in red, classified pixel by pixel.

[0096] However, these optimizations also result in changes to the classification results. The pooling layers have the greatest influence here. With each pooling, information is removed from the network. However, because the individual image elements are no longer viewed individually, the pooling loses its spatial reference. Fig. 14 illustrates the problem.

[0097] In the Fig. 14Each image element 156 is shown as a red frame 158. Before optimization, each image element would run individually through the CNN to classify its center pixel. The right image element 160 is shifted one pixel further to the right. The four colors: purple, blue, yellow, and green indicate the individual applications of pooling. As can be seen, they can produce different results because pooling always starts at the edge and moves on by one pooling element (here two fields). This results in two different pooling elements from two neighboring image elements 156, 160. If this is to be taken into account in the optimization, this would result in two new branches for further calculations with each pooling. This is because pooling would have to be applied once to the entire image with a starting point in the top left and another pooling with the starting point in the top left plus one pixel.In the subsequent calculation, both pooling results would then have to be processed separately. A second pooling would result in two new paths, meaning four separate results would have to be calculated. The result is then composed of the four results, rotating pixel by pixel through the results. If only one path is considered after pooling, the output image would be smaller after two poolings. The length and width of the output image would then each be only 1 / 4 the size of the input image. If all paths were considered, the resulting image would be approximately the size of the input image.

[0098] Another difference is the lack of edge regions of the plants. Since the features are not applied to all elements that have some overlap with the plant, calculation differences exist here. This can also change the classification result compared to the conventional calculation.

[0099] The lack of calculation of the feature values outside the plant can cause different values, as the result is shown as zero, which is in fact the bias value.

[0100] Although these three factors influence the results, it turns out that the CNN is very robust and therefore the results still achieve a very high accuracy level.

[0101] The next step would be to train the network directly with these modifications so that the network can adapt even better to its new calculation and thus compensate for any errors in the calculation directly.

[0102] The segmentation and data reduction device provides the pixels relating to the weed 154 with position coordinates. List of reference symbols

[0103] 10Carrier system 12Aerial drone 12aReceiver, receiving space of the aerial drone 12 12bControl device of the aerial drone 14Mobile device 14aFirst unit 14bSecond unit 14cReceiver for gripper arm on the mobile device 14 16Drive 18Electric motor 20Propeller 22Feet 24Battery 26aVoltage interface on the aerial drone 12 26bVoltage interface on the mobile device 14 28Plug connection 30Communication unit 32Antenna 34GPS unit 36aInterface on the aerial drone 12 36bInterface on the mobile device 14 38Plug connection 40First housing of the first unit 14a 42Second housing of the second unit 14b 44Plug connection 46First computer 48Arm as actuator 50Motor 52Tool unit 54Feed unit 56Rotation unit 58Milling cutter 60First communication unit of the first unit 14a 62Visual detection unit 64Camera 66Segmentation and data reduction device 68Classifier 70aInterface of the first unit 14a 70bInterface of the second unit 14b72 Communication connection 74 Second communication unit of the second unit 14b 76 Second computer 78 Database 80 Plug connection 82a Gripper arm, left 82b Gripper arm, right 84 First step: Determine the necessary measures 86 Second step: Select the mobile device 14 from the available mobile devices 14 88 Third step: Connect the device to the drone 12 90 Fourth step: Carry out the determined measures 92 Fifth step: Replace the mobile device 14 with another mobile device 14 and carry out another measure 94 First step: Continuous recording 96 Second step: Data transmission 98 Third step: Data storage 100 Fourth step: Data comparison 102 Fifth step: Evaluation by the classifier 68 104 Sixth step: Conversion into control and regulation data 106 Seventh step: Starting up the units 108Example image, input image 110Plant 112Convolution 114Pooling116Summary in a dense layer 118Background 120First step: Converting to an RGB color model 122Second step: Transferring to an HSV color model 124Third step: Evaluating the HSV image 126Fourth step: Creating an intermediate image 128First case, red 130Second case, yellow 132Third case, blue 134Feature 136Plant pixel, left 138Plant pixel, right 140Blue area 142Purple area 144Red area 146Orange area 148Feature, green 150Middle black box 152Carrot plant 154Weed, weeds 156Image element, left 158Red frame 160Image element, right

Claims

1. Mobile analysis and processing device (14) for use in agriculture for cultivating the soil and / or for manipulating flora and fauna, comprising at least one sensor (62), a tool unit (52) with at least one motor-driven tool (58), an actuator (48) for moving at least the tool (58) of the tool unit (52), a motor (50) for driving the tool unit (52) and / or the actuator (48), a database (78), a first communication unit (60) with an interface (70a) and a first computer (46) for controlling the sensor (62), the tool unit (52) and the actuator (48) on the basis of generated control commands, with the data collected by the sensor (62) being continuously compared with the data stored in the database (78) in order to generate appropriate control signals for the actuator (48), the tool unit (52) and / or the motor (50), characterized by a two-part design consisting of a first unit (14a) and a second unit (14b), with the sensor (62), the tool unit (52), the motor (50) for driving the tool (58), the tool unit (52) and / or the actuator (48), the actuator (48), a first computer (46) and a first communication unit (60) with an interface (70a) being provided in the first unit (14a), and the database (78), a second computer (76) and a second communication unit (74) with an interface (36b, 70b) being provided in the second unit (14b), wherein the first unit (14a) and the second unit (14b) can be connected to one another via the interfaces (70a, 70b) for data exchange, with the first unit (14a) comprising a first housing (40), and the second unit (14b) comprising a second housing (42), which first and second housings (40, 42) being detachably connected to one another via a plug connection (44, 80), and the data determined by the sensor (62) is compared with the data stored in the database (78) in real time with verification and classification of the data determined by the sensor (62) by the second computer (76).

2. Analysis and processing device according to claim 1, characterized in that the sensor is a visual detection unit (62) with a camera (64).

3. Analysis and processing device according to claim 2, characterized in that the visual detection unit (62) comprises a segmentation and data reduction device (66).

4. Analysis and processing device according to any one of claims 2 or 3 above, characterized in that the visual detection unit (62) comprises a classifier (68) which performs an evaluation in conjunction with the second computer (76) of the compared recording data with existing defined data sets in the database (78), in particular supported by artificial intelligence.

5. Analysis and processing device according to any one of the preceding claims, characterized in that means (82a, 82b) are provided for connecting it to a carrier (12) as required, in particular designed as an aerial drone, which moves the device (14).

6. Analysis and processing device according to any one of the preceding claims, characterized in that the first and second housings (40, 42) have receptacles (14c) as means for connection to the carrier (12) as required, which receptacles are assigned to corresponding holding means (82a, 82b) of the carrier (12), by means of which the device (14) can be gripped and moved by the carrier (12).

7. Analysis and processing device according to any one of the preceding claims, characterized in that the first and second housings (40, 42) have coupling means (26b, 28, 36b, 38) as means for connecting to the carrier (12) as required, which former coupling means are assigned to respective coupling means (26a, 28, 36a, 38) of the carrier (12), via which the device (14) can be connected to the carrier (12) and moved.

8. Analysis and processing device according to any one of the preceding claims, characterized in that the tool unit (52) has at least one feed unit (54) and a rotation unit (56) which interacts with the motor (50).

9. Analysis and processing device according to claim 8, characterized in that provided on a distal end of the rotation unit (56) there is at least the tool (58), in particular a milling cutter (58) or a blade unit.

10. Analysis and processing apparatus according to any one of the preceding claims, characterized by a voltage terminal (26b) for external voltage supply.

11. Analysis and processing device according to claim 10, characterized in that the voltage terminal (26b) is provided on the first unit (14a), and this terminal supplies voltage to the first unit (14a) and the second unit (14b).

12. Analysis and processing device according to any of the preceding claims, characterized by an additional communication interface (36b) for a carrier (12).

13. Analysis and processing device according to claim 12, characterized in that the additional communication interface (36b) is located in the second unit (14b).

14. Method for real-time control of the cultivation of the soil and / or manipulation of flora and fauna by the device according to any one of the preceding claims, comprising the following steps: a. continuous recording of data-defined voxels and / or pixels and / or images by the sensor; b. transmitting the recording data to the database (78); c. storing the recording data in the database (78); d. qualitative data comparison of the recording data with the data stored in the database (78), including implementing a segmentation and data reduction of the recording data by means of a segmentation and data reduction device (66) and / or implementing a verification of the recording data by the second computer (76); e. evaluating the compared recording data using existing defined data sets in the database (78) by a classifier (68) in conjunction with the second computer (76); f. processing and converting the evaluation by the first computer (46) into control and / or control-related data for the motor, the actuator, the tool unit and / or the associated carrier.

15. Method according to claim 14, characterized in that the motor (50), the actuator (48), the tool unit (52) and / or the carrier (12) for cultivating the soil and / or an associated carrier (12) for manipulating flora and fauna are started up once the regulation- and / or control-related data are available.

16. Method according to any one of claims 14 or 15 above, characterized in that the evaluation is carried out in a second computer (76) cooperating with the classifier (68), and the processing and conversion of the evaluation into regulation- and / or control-related data is carried out in the first computer (46), for which purpose the evaluation is transmitted from the second computer (46, 76) to the first computer (46).

17. Method according to any one of claims 14 or 15 above, characterized in that the storage, the qualitative data comparison of the recording data with data stored in the database (78) and / or the evaluation by the classifier (68) are all supported by artificial intelligence.