Inspection equipment and inspection method

Abstract

The present invention discloses a patrol device and a patrol method, which relate to the technical field of crop patrol inspection. The patrol device runs on a track in an agricultural land area, and includes a binocular camera module, an image processing module, and a wireless transmission module. The binocular camera module includes a first camera and a second camera; the image processing module is connected to the binocular camera module, and is used to process the first image and the second image; the wireless transmission module is connected to the image processing module, and is used to transmit the growth status information of the crop to the cloud. The present invention sets a patrol device that can patrol crops regularly and regularly, and sets a binocular camera module on the patrol device, so that the two cameras of the binocular camera module can take pictures of each crop while moving, and the pictures obtained by taking pictures are processed by the processing module and the crop growth information is transmitted to the remote end for management and analysis, so as to realize automatic regular and high-precision patrol inspection of crops.

Description

Technical Field

[0001] The present invention relates to the technical field of crop inspection, and in particular to inspection equipment and an inspection method. Background Art

[0002] With the development of science and technology, modern agriculture is moving towards a larger scale. In small-scale crop planting, crop inspections are usually conducted by managers who personally go to the crop planting area to observe crop growth information with their naked eyes and use their planting experience to determine the growth status of crops and quickly take corresponding effective measures.

[0003] However, for large-scale crop planting, inspection generally relies on manual inspection or drone inspection. When manual inspection is used, inspectors can only carry light equipment to sample crops and record relevant information. The disadvantages of this inspection method are low inspection coverage and long inspection time, especially when crops are planted in hilly areas. Due to the complex terrain of the planting area, the difficulty and workload of manual inspection are increased.

[0004] The use of drones for inspections specifically involves shooting from top to bottom through cameras, but this inspection method can only inspect crops from one dimension, making the inspection not precise enough and unable to conduct a full-scale inspection of crops.

[0005] It is difficult for the above two inspection methods of crops to fully and real-time grasp the growth status of crops. Summary of the invention

[0006] In order to solve the problems of the prior art, the present invention provides a patrol device, which runs on a track in an agricultural land area, and the track is a single track deployed along the crop planting position in the agricultural land area; comprising:

[0007] Binocular camera module, including:

[0008] A first camera, used to capture a first image of the crop;

[0009] a second camera, used to capture a second image of the crop;

[0010] an image processing module connected to the binocular camera module, and configured to process the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop; and determine the growth status information of the crop according to the actual size of the target feature of the crop and the growth information of the non-target feature of the crop;

[0011] The wireless transmission module is connected to the image processing module and is used to transmit the growth status information of the crops to the cloud.

[0012] Preferably, the image processing module further includes: a distance measurement module;

[0013] The ranging module comprises:

[0014] A target determination unit is used to determine target features of crops that appear in the first image and the second image respectively, and detect the imaging size of the target features:

[0015] A depth calculation unit is connected to the target determination unit and is used to calculate the depth of the target feature through the imaging size of the target feature; and determine the actual size of the target feature according to the depth of the target feature.

[0016] Preferably, the image processing module also includes: an image stitching module, used to stitch the first image information with the second image to obtain a stitched image; the image information of the stitched image is convolved through a convolutional neural network of the image processing module to extract non-target features of crops in the stitched image to obtain a non-target feature map, and then the network parameters of the fully connected layer, Softmax classifier, and bounding box regression are trained and adjusted through a transfer learning method and the detection results of the non-target features are output to obtain the growth information of the non-target features of the crops.

[0017] Preferably, the binocular camera module is also used to detect the running track information of the inspection equipment;

[0018] The inspection equipment also includes:

[0019] A control module is connected to the binocular camera module and is used for at least one of the following:

[0020] When it is determined according to the running track information that the running track information indicates that the inspection device is about to turn on the track, controlling the inspection device to turn;

[0021] When it is determined according to the running track trajectory information that the running track information indicates that the inspection device pre-climbs a slope on the track, the inspection device is controlled to climb the slope.

[0022] Preferably, the inspection equipment further includes:

[0023] An inspection mode selection module is connected to the control module;

[0024] The inspection mode selection module includes:

[0025] Daily inspection mode module, used to inspect crops regularly;

[0026] Temporary inspection mode module, used to inspect crops during bad weather.

[0027] The present invention also provides an inspection method using the above inspection device for inspection, the inspection method comprising:

[0028] The inspection equipment is driven to run on a single track laid in the area where the crops are planted;

[0029] Using the rotating first camera and second camera of the binocular camera module to respectively capture a first image and a second image of the crop;

[0030] Processing the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop; determining the growth status information of the crop according to the actual size of the target feature of the crop and the growth information of the non-target feature of the crop;

[0031] The wireless transmission module is used to transmit the growth status information of the crops to the cloud.

[0032] Preferably, the processing of the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop includes:

[0033] Determine the target features of the target crops that appear in the first image and the second image simultaneously, and detect the imaging size of the target features:

[0034] The depth of the target feature is calculated by the imaging size of the target feature; and the actual size of the target feature is determined according to the depth of the target feature.

[0035] Preferably, the processing of the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop includes:

[0036] The first image information and the second image are spliced ​​to obtain a spliced ​​image; the image information of the spliced ​​image is convolved through a convolutional neural network of an image processing module to extract non-target features of crops in the spliced ​​image to obtain a non-target feature map, and then the network parameters of the fully connected layer, the Softmax classifier, and the bounding box regression are trained and adjusted through a transfer learning method, and the detection results of the non-target features are output to obtain the growth information of the non-target features of the crops.

[0037] Preferably, the driving inspection equipment runs on a single track laid in the crop planting area, including one of the following:

[0038] In the daily inspection mode, in response to receiving the timing prompt information, the inspection equipment is driven to run on a single track laid in the crop planting location area;

[0039] In the temporary inspection mode, in response to detecting bad weather, the inspection equipment is driven to run on a single track laid in the area of ​​the crop planting location.

[0040] The present invention also provides an inspection device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor is used to implement the above-mentioned inspection method when running the computer program.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention arranges inspection equipment for inspecting crops and arranges a binocular camera module on the inspection equipment, so that the two cameras of the binocular camera module can take pictures of each crop while moving respectively. The pictures obtained by taking pictures are processed by the processing module to obtain the actual size of the target feature of each crop and the growth information of the non-target feature of each crop, and then the growth status information of each crop is determined, and finally the growth information of all the crops in the crop planting area is determined and the above crop growth information is transmitted to a remote end for management and analysis, so as to realize accurate inspection of crops, and the inspection coverage is comprehensive and the inspection accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the structure of an inspection device provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the structure of an image processing module provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of the structure of a distance measurement module provided by an embodiment of the present invention;

[0046] Figure 4 Schematic diagram of the principle of binocular ranging in one embodiment of the present invention;

[0047] Figure 5 A schematic diagram of a flow chart of an inspection method provided by a specific embodiment of the present invention;

[0048] Figure 6 A schematic diagram of the hardware structure of an inspection device provided in one embodiment of the present invention.

[0049] Figure labels: 1-memory; 2-processor. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0052] In the embodiment of the present invention, the application scenario of the inspection equipment can be to inspect the planted crops. It is understandable that in the prior art, for large-scale planted crops, the inspection thereof generally relies on manual inspection or drone inspection. The disadvantages of manual inspection are low inspection coverage and long inspection time, especially when crops are planted in hilly areas. Due to the complex terrain of the planting area, the difficulty and workload of manual inspection are increased. The use of drones for inspection is specifically to shoot from top to bottom through a camera, but this inspection method can only inspect crops from one dimension, which makes the inspection not detailed enough and cannot conduct a full-scale inspection of crops.

[0053] Based on this, how to automatically and efficiently inspect crops in agricultural land in an all-round way has become a technical problem that needs to be solved urgently.

[0054] The following takes the agricultural land as an orchard as an example, and the crops as fruit trees as an example. Of course, in other embodiments, the agricultural land may also be, for example, a vegetable garden, and the crops may also be vegetables, etc., without any limitation here.

[0055] Figure 1 A schematic diagram of the structure of an inspection device provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the inspection equipment runs on a track in the fruit tree planting area, which is a single track deployed along the fruit tree planting location in the agricultural land area; including:

[0056] Binocular camera module, including:

[0057] A first camera, used for collecting a first image of each fruit tree;

[0058] A second camera is used to collect a second image of each fruit tree;

[0059] The image processing module is connected to the binocular camera module and is used to process the first image and the second image to obtain the actual size of the target feature of the fruit tree and the growth information of the non-target feature of the fruit tree; and determine the growth status information of the fruit tree according to the actual size of the target feature of the fruit tree and the growth information of the non-target feature of the fruit tree;

[0060] The wireless transmission module is connected to the image processing module and is used to transmit the growth status information of the fruit trees to the cloud.

[0061] It should be noted that the agricultural areas here can be located in plains or hilly areas.

[0062] The planting of fruit trees is carried out in a row or strip.

[0063] The binocular camera module here is set on the top of the inspection equipment, and the first camera and the second camera of the binocular camera module can rotate clockwise or counterclockwise. When moving on the track, the first camera and the second camera can take pictures of the fruit trees no matter which side of the track the fruit trees are located on. And the first image of each fruit tree collected by the first camera and the second image of the fruit tree collected by the second camera have an overlapping area.

[0064] The first camera and the second camera here can be set as visible light cameras or infrared cameras. The visible light camera can be used to shoot fruit trees under good lighting conditions; while the infrared camera forms video images by receiving diffusely reflected infrared rays after irradiating the photographed object, so the infrared camera is suitable for shooting fruit trees under poor lighting conditions or no lighting conditions.

[0065] The target features of the fruit trees here include but are not limited to the fruits of the fruit trees and / or the trunks of the fruit trees, etc.; and the actual sizes of the target features here include but are not limited to the actual size of the fruits and / or the actual cross-sectional diameter of the trunks.

[0066] The non-target features of the fruit trees here include but are not limited to the leaves of the fruit trees and / or the flowers of the fruit trees. And the growth information of the non-target features of the fruit trees here includes but is not limited to the morphology of the leaves of the fruit trees at different periods in a growth period of the fruit trees and / or the morphology of the flowers of the fruit trees during the growth period.

[0067] The growth status information of the fruit trees here refers to the growth trend of the fruit trees within a growth cycle.

[0068] This embodiment uses a binocular camera assembly to take pictures of fruit trees, and extracts target features such as fruits and / or trunks from the pictures, as well as non-target features such as leaves and / or flowers of fruit trees. The actual size of the target features is calculated, and non-target features are extracted from the image and their shapes are obtained, thereby determining the growth status of the fruit trees in a growth cycle. The growth status data is transmitted to the remote end for management and analysis. This embodiment implements inspections of fruit trees, and the inspections cover the entire orchard, and the inspections are highly accurate, allowing planting personnel to more accurately understand the growth of each fruit tree in a growth cycle.

[0069] Figure 2 FIG. 1 is a structural diagram of an image processing module provided by an embodiment of the present invention, as shown in FIG. Figure 2 As shown, the image processing module includes a ranging module and an image stitching module, specifically:

[0070] Figure 3 A schematic diagram of the structure of a distance measurement module provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the image processing module in this embodiment is used to obtain the actual size of the fruit tree target feature, and specifically includes: a distance measurement module; the distance measurement module includes:

[0071] The target determination unit is used to determine the target features of the fruit trees that appear in the first image and the second image respectively, and detect the imaging size of the target features:

[0072] The depth calculation unit is connected to the target determination unit and is used to calculate the depth of the target feature through the imaging size of the target feature; and determine the actual size of the target feature according to the depth of the target feature.

[0073] The target determination unit here extracts the target features of the fruit trees in the first image and the second image and determines the imaging size of the target features by selecting a neural network model that has been trained on a large data set, and determines the depth of the target features by the imaging size of the target features.

[0074] There are many ways to calculate depth by imaging size. This depth can be relative depth or absolute depth. Relative depth refers to the change in the depth of the target object, such as the depth increases as the imaging size decreases. Absolute depth can correspond to the depth value. The depth of the target object can be determined by continuous measurement of the relative depth or absolute depth of the target's physical body.

[0075] The depth of the target feature of the fruit tree determined by the imaging size of the target feature can be determined by the ratio of the length or width or area of ​​the target feature imaging to the entire image, to determine the relative depth of the target feature; or, the relative depth of the target feature can be determined by the continuous imaging size of the target feature; or, the depth of the target feature can be determined by querying the pre-made comparison relationship between the imaging size and the depth through the imaging size of the target feature.

[0076] After determining the depth of the target feature, the actual size of the target feature is obtained by using proportional conversion.

[0077] The depth of the target feature can also be determined by ranging, such as Figure 4 As shown:

[0078] P is a point on the target feature, OR and OT are the optical centers of the first camera and the second camera respectively, the imaging points of point P on the two camera photoreceptors are P and P' respectively (the imaging plane of the camera is rotated and placed in front of the lens), f is the focal length of the camera, B is the center distance between the two cameras, Z is the depth of the target feature, and the distance from point P to point P' is dis, then:

[0079] ;

[0080] According to the principle of similar triangles:

[0081] ;

[0082] Then we get:

[0083]

[0084] The focal length f and the center distance B between the two cameras can be obtained through calibration measurement.

[0085] Therefore, the depth information can be obtained by simply obtaining the parallax between the first image and the second image.

[0086] The method for obtaining the parallax between the first image and the second image is:

[0087] A stereo region matching method is used to estimate a left-to-right disparity map sequence and a right-to-left disparity map sequence of the first image and the second image frame by frame respectively;

[0088] Specifically, the binocular stereo video image is over-segmented using the mean-shift algorithm;

[0089] Then construct the energy function as:

[0090] E energy = E data +λE smooth

[0091] in, E data is a data item, E smooth is the smoothing term, λ is the weight coefficient,

[0092] The corresponding grid map is constructed according to the data item and the smoothing item, and the disparity information corresponding to each pixel is calculated by using an algorithm to obtain a disparity map.

[0093] The data item of the energy function takes into account both intra-frame error and inter-frame error; the intra-frame error uses the mean absolute difference MAD criterion and the sum absolute difference of gradient SGRAD criterion, and the weight of each pixel is adaptively adjusted according to its importance in the window;

[0094] Disparity image consistency check: Check the consistency of the left-to-right disparity image sequence and the right-to-left disparity image sequence, and calculate the disparity after correcting the unreliable field of view matching information. Finally, the depth information is obtained through the calculated disparity.

[0095] In other embodiments, the function of the image processing module is to stitch the crop pictures taken by the first camera and the second camera to obtain a stitched image, and obtain images of all fruit trees in the planting area through the stitched image, so as to avoid the situation where the first camera or the second camera only captures a partial image of a fruit tree and cannot obtain the growth information of the fruit tree.

[0096] To achieve the above effect, the image processing module includes an image stitching module, which is used to stitch the first image information with the second image to obtain a stitched image; the specific method used for stitching is:

[0097] Taking a first image and a second image of the same fruit tree, selecting one of the images as a reference image and the other as an image to be stitched, wherein the reference image and the image to be stitched have an overlapping area;

[0098] Performing a first remapping on the reference image and the image to be stitched respectively;

[0099] Extracting coarse matching feature point pairs between the remapped reference image and the remapped image to be stitched;

[0100] Obtain fine matching feature point pairs from coarse matching feature point pairs using the second remapping;

[0101] estimating a rotation matrix and an offset matrix of the first camera and the second camera using the fine matching feature point pairs;

[0102] The reference image and the image to be stitched are stitched respectively according to the rotation matrix and the offset matrix to obtain a stitched image.

[0103] The method of performing the first remapping on the reference image and the image to be stitched includes:

[0104] The reference image and the image to be stitched are respectively first remapped using equidistant cylindrical projection to obtain corresponding equidistant cylindrical projection images, which are respectively used as the remapped reference image and the remapped image to be stitched.

[0105] Wherein, obtaining fine matching feature point pairs from coarse matching feature point pairs by using the second remapping includes:

[0106] Performing a second remapping of the coordinates of the rough matching feature point pairs using plane projection;

[0107] The feature point pairs that meet the homography in the coarse matching feature point pairs are obtained as the fine matching feature point pairs.

[0108] This embodiment adopts a static stitching method and utilizes the geometric properties of the projection of the acquired image to extract and match the feature points of the original image set, thereby obtaining accurate control point pairs and achieving high quality after image stitching.

[0109] The spliced ​​images in this embodiment are sent to a 101-layer convolutional neural network to output a non-target feature map of the fruit tree; and then the features of each part in the feature map are integrated;

[0110] The transfer learning method is used to train and adjust the network parameters of the final fully connected layer, Softmax classifier, and bounding box regression to obtain the recognition results of non-target features of fruit trees; the growth form of non-target features is determined by the recognition results.

[0111] By obtaining the growth morphology of the non-target features of the fruit trees and the actual size of the target features of the fruit trees, the growth status information of the fruit trees can be obtained.

[0112] In other embodiments, the binocular camera module can also be used to detect the running track information of the inspection equipment;

[0113] The inspection device also includes a control module connected to the binocular camera module, which is used for at least one of the following:

[0114] According to the running track track information, when it is determined that the running track information indicates that the inspection equipment is to pre-turn on the track, the inspection equipment is controlled to turn;

[0115] When it is determined according to the running track trajectory information that the running track information indicates that the inspection device is pre-climbing a slope on the track, the inspection device is controlled to climb the slope.

[0116] The running track information of this embodiment can be obtained through certain special position information points on the track. These special positions can be the turning points of the track or the climbing points of the track. A signal receiving and transmitting device can be set at the above-mentioned special positions. For example, photoelectric sensors can be set at all the turning points of the track, or proximity switches can be set at the climbing points of the track. At the same time, these photoelectric sensors and proximity switches are electrically connected to the control module. In this way, the inspection equipment can automatically realize turning and climbing when the track is running.

[0117] After the inspection equipment takes a picture of a fruit tree, it will automatically move along the track to take a picture of the next fruit tree.

[0118] The inspection device of this embodiment further includes an inspection mode selection module, which is connected to the control module;

[0119] Inspection mode selection modules include:

[0120] Daily inspection mode module, used to inspect fruit trees regularly;

[0121] Temporary inspection mode module, used to inspect crops during bad weather.

[0122] The regular inspection in the daily inspection mode can be set to inspect the fruit trees in the orchard every three days as an inspection cycle; and the scheduled inspection can be carried out at three time periods within the inspection day: 6 am, 1 pm, and 8 pm. Of course, the inspection cycle and time period can be determined according to actual needs, and the present invention does not make any limitation here.

[0123] Furthermore, the inspection mode can also be a temporary inspection under certain special weather conditions. The special weather conditions in this embodiment can be when the air temperature in the orchard is higher than 30°C, or the air temperature in the orchard is lower than 4°C, or the air humidity in the orchard is greater than 85%, and the control module drives the inspection equipment to perform a temporary inspection. The air temperature and humidity of the orchard can be measured by setting a temperature sensor and a humidity sensor on the inspection equipment, and the above sensors are also connected to the control module. Of course, the temperature and humidity of the temporary inspection can be determined according to the variety of fruit trees and actual needs, and the present invention does not make any limitation here.

[0124] It should be noted that the parameter settings for regular scheduled inspections of daily inspections and the temperature and humidity parameter settings for temporary inspections can be set through the control panel on the inspection equipment, or through the remote control terminal of the inspection equipment.

[0125] In the embodiment of the present invention, by setting a daily inspection mode and a temporary inspection mode, regular and scheduled routine inspections of fruit trees as well as temporary inspections of fruit trees in severe weather conditions such as high temperature, low temperature, and heavy rain can be achieved.

[0126] Figure 5 A schematic diagram of a process flow of an inspection method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the inspection method is applied to the above-mentioned inspection equipment to inspect fruit trees, and the inspection method includes:

[0127] The inspection equipment is driven to run on a single track laid in the area where the fruit trees are planted;

[0128] Using a first camera and a second camera of a rotating binocular camera module to respectively collect a first image and a second image of each fruit tree;

[0129] Processing the first image and the second image to obtain the actual size of the target feature of the fruit tree and the growth information of the non-target feature of the fruit tree; determining the growth status information of the fruit tree according to the actual size of the target feature of the fruit tree and the growth information of the non-target feature of the fruit tree;

[0130] The growth status information of fruit trees is transmitted to the cloud using a wireless transmission module.

[0131] In some optional embodiments, the first image and the second image are processed to obtain the actual size of the target feature of the fruit tree and the growth information of the non-target feature of the fruit tree, including:

[0132] Determine the target features of the fruit trees that appear in the first image and the second image simultaneously, and detect the imaging size of the target features:

[0133] The depth of the target feature is calculated through the imaging size of the target feature; and the actual size of the target feature is determined based on the depth of the target feature.

[0134] In some optional embodiments, the first image and the second image are processed to obtain the actual size of the target feature of the fruit tree and the growth information of the non-target feature of the fruit tree, including:

[0135] The first image information is spliced ​​with the second image to obtain a spliced ​​image; the image information of the spliced ​​image is convolved through the convolutional neural network of the image processing module to extract the non-target features of the crops in the spliced ​​image to obtain a non-target feature map, and then the network parameters of the fully connected layer, the Softmax classifier, and the bounding box regression are trained and adjusted through the transfer learning method, and the detection results of the non-target features are output to obtain the growth information of the non-target features of the fruit trees.

[0136] In some optional embodiments, the driving inspection device runs on a single track laid in the fruit tree planting area, including one of the following:

[0137] In the daily inspection mode, in response to receiving the timing prompt information, the inspection equipment is driven to run on a single track laid in the fruit tree planting location area;

[0138] In the temporary inspection mode, in response to detecting bad weather, the inspection equipment is driven to run on a single track laid in the area where the fruit trees are planted.

[0139] It should be noted that the description of the above inspection method is similar to the description of the above inspection equipment, and the beneficial effects of the same method are not described in detail. For technical details not disclosed in the embodiment of the inspection method of the present invention, please refer to the description of the embodiment of the inspection equipment of the present invention.

[0140] like Figure 6 As shown, an embodiment of the present invention further provides an inspection device, which includes a memory 1, a processor 2, and computer instructions stored in the memory 1 and executable on the processor 2; when the processor 2 executes the instructions, the steps applied to the vulnerability scanning method are implemented.

[0141] In some embodiments, the memory 1 in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus DRAM (DRRAM). The memory 1 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0142] The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 2. The above processor 2 may be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor may be a microprocessor or the processor may also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1, and the processor 2 reads the information in the memory 1 and completes the steps of the above method in combination with its hardware.

[0143] In some embodiments, the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0144] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0145] Another embodiment of the present invention provides a computer storage medium, the computer readable storage medium stores an executable program, and when the executable program is executed by the processor 2, the steps applied to the driving method can be implemented. Figure 1 One or more of the methods shown.

[0146] In some embodiments, the computer storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0147] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0148] Finally, it should be noted that the above only describes in detail the specific embodiments of the present invention. However, the present invention is not limited to the specific embodiments described above. Equivalent modifications and substitutions made by those skilled in the art to the present invention are also within the scope of the present invention. Therefore, equal changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of the present invention.

Claims

1. A patrol inspection device, characterized in that: The inspection equipment runs on a track in the agricultural land area, which is a single track deployed along the crop planting position in the agricultural land area; including: Binocular camera module, including: A first camera, used to capture a first image of the crop; a second camera, used to capture a second image of the crop; an image processing module connected to the binocular camera module, and configured to process the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop; and determine the growth status information of the crop according to the actual size of the target feature of the crop and the growth information of the non-target feature of the crop; A wireless transmission module, connected to the image processing module, for transmitting the growth status information of the crops to the cloud; The image processing module also includes: a distance measurement module; The ranging module comprises: A target determination unit is used to determine target features of crops that appear in the first image and the second image respectively, and detect the imaging size of the target features: a depth calculation unit connected to the target determination unit, and configured to calculate the depth of the target feature by the imaging size of the target feature; and determine the actual size of the target feature according to the depth of the target feature; the target feature includes the fruit of a fruit tree and / or the trunk of a fruit tree; The image processing module further includes: an image stitching module, which is used to stitch the first image information with the second image to obtain a stitched image; the image information of the stitched image is convolved by a convolutional neural network of the image processing module to extract non-target features of crops in the stitched image to obtain a non-target feature map, and then the network parameters of the fully connected layer, the Softmax classifier, and the bounding box regression are trained and adjusted by a transfer learning method and the detection results of the non-target features are output to obtain the growth information of the non-target features of the crops; the non-target features include leaves of fruit trees and / or flowers of fruit trees; The binocular camera module is also used to detect the running track information of the inspection equipment; The inspection equipment also includes: A control module is connected to the binocular camera module and is used for at least one of the following: When it is determined according to the running track trajectory information that the running track trajectory information indicates that the inspection device is about to turn on the track, controlling the inspection device to turn; When it is determined according to the running track trajectory information that the running track trajectory information indicates that the inspection device pre-climbs a slope on the track, the inspection device is controlled to climb the slope.

2. The inspection device according to claim 1, characterized in that: The inspection equipment also includes: An inspection mode selection module is connected to the control module; The inspection mode selection module includes: Daily inspection mode module, used to inspect crops regularly; Temporary inspection mode module, used to inspect crops during bad weather.

3. An inspection method applied to the inspection device according to claim 1, characterized in that: The inspection method comprises: The inspection equipment is driven to run on a single track laid in the area where the crops are planted; Using the rotating first camera and second camera of the binocular camera module to respectively capture a first image and a second image of the crop; Processing the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop; determining the growth status information of the crop according to the actual size of the target feature of the crop and the growth information of the non-target feature of the crop; Transmitting the growth status information of the crops to the cloud using the wireless transmission module; The processing of the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop includes: Determine the target features of the target crops that appear in the first image and the second image simultaneously, and detect the imaging size of the target features: The depth of the target feature is calculated by the imaging size of the target feature; the actual size of the target feature is determined according to the depth of the target feature; the target feature includes the fruit of the fruit tree and / or the trunk of the fruit tree; The processing of the first image and the second image to obtain the actual size of the target feature of the crop and the growth information of the non-target feature of the crop includes: The first image information is spliced ​​with the second image to obtain a spliced ​​image; the image information of the spliced ​​image is convolved by a convolutional neural network of an image processing module to extract non-target features of crops in the spliced ​​image to obtain a non-target feature map, and then the network parameters of the fully connected layer, the Softmax classifier, and the bounding box regression are trained and adjusted by a transfer learning method and the detection results of the non-target features are output to obtain the growth information of the non-target features of the crops; the non-target features include leaves of fruit trees and / or flowers of fruit trees; The binocular camera module is also used to detect the running track information of the inspection equipment; The inspection equipment also includes: A control module is connected to the binocular camera module and is used for at least one of the following: When it is determined according to the running track trajectory information that the running track trajectory information indicates that the inspection device is about to turn on the track, controlling the inspection device to turn; When it is determined according to the running track trajectory information that the running track trajectory information indicates that the inspection device pre-climbs a slope on the track, the inspection device is controlled to climb the slope.

4. The method according to claim 3, characterized in that The driving inspection equipment runs on a single track laid in the crop planting area, including one of the following: In the daily inspection mode, in response to receiving the timing prompt information, the inspection equipment is driven to run on a single track laid in the crop planting location area; In the temporary inspection mode, in response to detecting bad weather, the inspection equipment is driven to run on a single track laid in the area of ​​the crop planting location.

5. A patrol inspection device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein the processor is used to implement the inspection method according to any one of claims 3 to 4 when running the computer program.

Images