Method and device for identifying mulch recovery trajectory based on deep video analysis

Abstract

The present invention discloses a method and device for identifying a ground film recovery trajectory based on deep video analysis, and relates to the field of video recognition technology. The method includes: collecting target area cruise images through drone cruising, and matching the ground film area colors according to the ground film reference color characteristics and ground film transparency to obtain a quasi-ground film calibration area; using a drone fan device to blow at the ground film calibration area and perform video acquisition; performing flexible analysis on the collected quasi-ground film calibration area monitoring video, determining the ground film calibration area according to the analysis results, constructing a ground film recovery trajectory, and sending it to the ground film recovery control terminal. The present invention solves the technical problem that the existing ground film recovery method in the prior art determines the ground film drifting position through manual inspections for recovery, and the recovery efficiency is low, and achieves the technical effect of efficiently and accurately identifying farmland ground films and planning recovery trajectories based on deep video analysis, thereby improving the efficiency of ground film recovery.

Description

Technical Field

[0001] The present invention relates to the field of video recognition technology, and in particular to a method and device for identifying ground film recovery tracks based on deep video analysis. Background Art

[0002] Ground film is widely used in agricultural production, which can effectively increase crop yields, reduce water evaporation and inhibit weed growth. However, if the used ground film is not recycled in time, it will easily break and disperse in the soil, forming "white pollution" and causing long-term impacts on the soil and the environment. Ground film recycling refers to the process of recycling waste agricultural ground film (such as polyethylene film) to reduce its pollution to the environment and reuse it. The existing recycling method determines the location of the scattered ground film through manual inspection and recycling, and the recycling efficiency is low. Summary of the invention

[0003] The present application provides a method and device for identifying ground film recovery trajectories based on deep video analysis, which is used to solve the technical problem that the existing ground film recovery methods in the prior art determine the scattered ground film locations for recovery through manual inspections, resulting in low recovery efficiency.

[0004] The first aspect of the present application provides a method for identifying ground film recovery trajectories based on deep video analysis, the method comprising: collecting ground film recovery images by cruising a drone; obtaining ground film reference color features and ground film transparency; performing ground film area color matching on the ground film recovery images of the target area according to the ground film reference color features and the ground film transparency, to obtain a quasi-ground film calibration area; blowing the quasi-ground film calibration area by a drone fan device, and performing video capture of the quasi-ground film calibration area along a position perpendicular to the blowing direction by a drone image acquisition device to obtain a monitoring video of the quasi-ground film calibration area; performing flexibility analysis on the monitoring video of the quasi-ground film calibration area to obtain a flexibility coefficient calibration value; when the flexibility coefficient calibration value is greater than or equal to a flexibility coefficient threshold, adding the quasi-ground film calibration area to the ground film calibration area; constructing a ground film recovery trajectory according to the ground film calibration area, and sending it to a ground film recovery control terminal.

[0005] The second aspect of the present application provides a device for identifying ground film recovery tracks based on deep video analysis, the device comprising: a cruise image acquisition module, the cruise image acquisition module is used to collect cruise images of a target area through a drone cruise; a ground film benchmark feature acquisition module, the ground film benchmark feature acquisition module is used to obtain ground film benchmark color features and ground film transparency; a ground film area color matching module, the ground film area color matching module is used to perform ground film area color matching on the target area cruise image according to the ground film benchmark color features and the ground film transparency, and obtain a quasi-ground film calibration area; a calibration area video acquisition module, the calibration area video acquisition module is used to use a drone fan device to The quasi-ground film calibration area is blown, and a video of the quasi-ground film calibration area is collected along a position perpendicular to the blowing direction through an unmanned aerial vehicle image acquisition device to obtain a monitoring video of the quasi-ground film calibration area; a flexibility analysis module, the flexibility analysis module is used to perform flexibility analysis on the monitoring video of the quasi-ground film calibration area to obtain a flexibility coefficient calibration value; a ground film calibration area division module, the ground film calibration area division module is used to add the quasi-ground film calibration area into the ground film calibration area when the flexibility coefficient calibration value is greater than or equal to the flexibility coefficient threshold; a ground film recovery control module, the ground film recovery control module is used to construct a ground film recovery trajectory according to the ground film calibration area, and send it to the ground film recovery control terminal.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] The present application provides a method and device for identifying ground film recovery trajectory based on deep video analysis, which relate to the field of video recognition technology. The method and device are used to cruise the target area by unmanned aerial vehicle (UAV), and color matching of the ground film area is performed according to the reference color characteristics and transparency of the ground film to obtain a quasi-ground film calibration area. The fan device of the UAV is used to blow the ground film calibration area, and the monitoring video of the quasi-ground film calibration area is collected for flexible analysis. The ground film calibration area is determined according to the analysis results, and a ground film recovery trajectory is constructed to control the ground film recovery. The method solves the technical problem that the existing ground film recovery method in the prior art determines the scattered position of the ground film through manual inspection for recovery, and the recovery efficiency is low. The method realizes the technical effect of efficient and accurate identification of farmland ground film and recovery trajectory planning based on deep video analysis, thereby improving the efficiency of ground film recovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A schematic diagram of a method for identifying a ground film recovery trajectory based on deep video analysis provided in an embodiment of the present application;

[0010] Figure 2 A schematic diagram of a process for obtaining a quasi-ground film calibration area in a ground film recovery trajectory identification method based on deep video analysis provided in an embodiment of the present application;

[0011] Figure 3 A schematic diagram of the structure of a ground film recovery trajectory identification device based on deep video analysis provided in an embodiment of the present application.

[0012] Explanation of the reference numerals: cruise image acquisition module 11 , ground film reference feature acquisition module 12 , ground film area color matching module 13 , calibration area video acquisition module 14 , flexibility analysis module 15 , ground film calibration area division module 16 , ground film recovery control module 17 . DETAILED DESCRIPTION

[0013] The present application provides a method and device for identifying ground film recovery trajectories based on deep video analysis, which is used to solve the technical problem that the existing ground film recovery methods in the prior art determine the scattered ground film locations through manual inspections for recovery, resulting in low recovery efficiency.

[0014] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0015] It should be noted that the terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices.

[0016] Embodiment 1

[0017] like Figure 1 As shown, the present application provides a method for identifying a ground film recovery trajectory based on deep video analysis, the method comprising:

[0018] P10: Collect cruise images of the target area through drone cruising.

[0019] Specifically, drones are used to patrol and collect patrol images of the target area for subsequent identification and recovery of mulch. First of all, the drone is an unmanned aerial vehicle (UAV), which has highly flexible maneuverability and efficient coverage capabilities. It can quickly fly over large areas of farmland and obtain high-resolution image data in real time. The drone is equipped with a high-definition camera or multi-spectral sensor, and ensures that every corner of the target area is covered by setting a predetermined flight path or using GPS for automatic navigation.

[0020] During the cruise process, the image data collected by the drone will be transmitted to the ground control center in real time at a high frame rate to ensure timely processing. The cruise images mentioned here refer to the continuous image sequence taken by the camera during the flight of the drone. These images cover the entire target area and are continuous and sequential. In the acquired images, the clarity and color accuracy of the image need to be guaranteed to facilitate the subsequent ground film identification and color feature analysis.

[0021] Secondly, in order to improve the recognition accuracy of ground film, the flight altitude and speed of the drone need to be optimized according to the specific environment. Flying too high may cause the details of the ground film to be lost, while flying too low will increase the cruising time. The flight parameters of the drone are usually adjusted according to the size of the farmland, the complexity of the terrain and the ambient light conditions. During the acquisition of the cruising image of the target area, the camera of the drone can adjust the focal length and angle in real time to ensure that the captured ground film image has good resolution and color contrast.

[0022] In addition, thanks to the flexibility of the drone, obstacles can be avoided and multiple repeated scans can be performed to ensure the integrity of the ground film image data. The acquired cruise images will be transmitted to the image processing system for preprocessing and feature extraction to identify the baseline color characteristics and transparency of the ground film (this will be described in detail in the subsequent steps). Through accurate cruise image acquisition, the subsequent ground film matching and recovery path construction will be more accurate and efficient.

[0023] P20: Obtain the base color characteristics and transparency of the ground film.

[0024] Optionally, the base color characteristics and transparency of the mulch commonly used in the area can be determined through preliminary research or experiments. The base color characteristics of the mulch refer to its reflective spectrum distribution in the visible light range, and these characteristics determine the color performance of the mulch under human eye observation. For example, the reflectivity of black mulch in the visible light range is extremely low, while transparent mulch has a higher transmittance. The transparency of the mulch refers to the difficulty of light penetrating the mulch, which is usually expressed by transmittance. The higher the transmittance, the more transparent the mulch is, allowing more light to penetrate; conversely, the mulch is less transparent and has a stronger light blocking effect. These characteristics will serve as the basis for subsequent color matching and identification.

[0025] For example, first, collect representative samples of mulch films from farmland or production lines. These samples should cover different colors (such as transparent, black, white, silver, green, etc.) and possible production batches to ensure the wide applicability of the results. As an important material in agricultural production, the color and transparency of mulch films have a significant impact on soil temperature, water retention, weed control, etc.

[0026] Furthermore, spectral analysis technology (such as a spectrophotometer) is used to measure the reflectance spectrum of the ground film sample. Spectral analysis technology infers the chemical composition and physical properties of a substance by measuring its light absorption, reflection or transmission characteristics at different wavelengths. For ground films, specific wavelength regions in the reflection spectrum can reflect its color characteristics. At the same time, a transmittance tester is used to measure the transmittance of the ground film. The transmittance tester calculates the transmittance of the ground film by emitting light of a specific wavelength and measuring its intensity after penetrating the ground film. Transmittance is an important indicator to measure the transparency of the ground film, which directly affects the effect of the ground film on regulating soil temperature and light.

[0027] Finally, by comparing the reflectance spectra and transmittance of different color films, the benchmark color characteristics and transparency range of each color film can be summarized as the basis for subsequent color matching and identification.

[0028] P30: According to the ground film reference color feature and the ground film transparency, the ground film area color matching is performed on the cruise image of the target area to obtain a quasi-ground film calibration area.

[0029] Further, such as Figure 2 As shown, step P30 of the embodiment of the present application also includes:

[0030] P31: When the transparency of the ground film is greater than or equal to the transparency threshold, the ground film sample color is identified based on the ground film reference color feature and the background color feature to obtain the ground film sample color feature; P32: When the transparency of the ground film is less than the transparency threshold, the ground film reference color feature is set as the ground film sample color feature; P33: The ground film area color is matched on the cruise image of the target area according to the ground film sample color feature to obtain the quasi-ground film calibration area.

[0031] It should be understood that based on the previously determined film benchmark color characteristics and film transparency, the cruise image of the target area is accurately matched with the film area color to identify and demarcate the quasi-film area. This process combines the physical characteristics of the film and image processing technology to improve the automation and accuracy of farmland monitoring.

[0032] Specifically, when dealing with the key factor of ground film transparency, two different strategies can be adopted to define the color characteristics of ground film samples to adapt to ground films with different transparencies. If the transparency of the ground film is greater than or equal to the preset transparency threshold (this threshold is set based on experience and needs in practical applications), it means that the ground film is relatively transparent and its color characteristics may be affected by the background soil or other crops. Therefore, based on the comprehensive analysis of the ground film benchmark color characteristics and the background color characteristics, the actual color characteristics of the ground film samples are simulated and identified through algorithm models (such as mixed Gaussian models, color space conversion, etc.). This process takes into account the interaction between the ground film and the background, making the color matching more accurate.

[0033] On the contrary, if the transparency of the ground film is less than the transparency threshold, it means that the ground film is relatively opaque, and its color characteristics are mainly affected by the material of the ground film itself, and have little to do with the background. At this time, the predetermined ground film reference color characteristics can be directly set as the ground film sample color characteristics, which simplifies the matching process and ensures the accuracy of the matching.

[0034] Finally, the color features of the mulch samples obtained above are used to perform color matching on the cruise image of the target area. This step usually involves image processing techniques, such as color space conversion (such as RGB to HSV or Lab color space), threshold segmentation, morphological operations (such as corrosion, expansion, opening and closing operations), etc., to separate the mulch area from the complex image background. By comparing the color features of each pixel or pixel block in the image with the color features of the mulch sample, the quasi-mulch area is identified and calibrated, providing an important basis for subsequent farmland management and crop growth monitoring.

[0035] Furthermore, step P31 of the embodiment of the present application further includes:

[0036] P31-1: Obtain the acquisition coordinate identifier, acquisition angle identifier and acquisition time identifier of the cruise image of the target area; P31-2: Match the background color feature through the target area background color database according to the acquisition coordinate identifier, the acquisition angle identifier and the acquisition time identifier, wherein the target area background color database is a database that stores user-defined background colors at any position in the target area; P31-3: Configure the network according to the sample color feature, fit the ground film baseline color feature, the background color feature and the ground film transparency, and obtain the ground film sample color feature.

[0037] Optionally, a complex color matching method based on time, space and angle dimensions is used to accurately identify the sample color characteristics of the ground film. First, collect and record the acquisition coordinate identification, acquisition angle identification and acquisition time identification of the cruise image of the target area. These identification information reflects the specific environment and conditions when the image was acquired, which is crucial for the subsequent background color feature matching. The acquisition coordinate identification indicates the position of the image in the target area, the acquisition angle identification describes the shooting angle of the camera, and the acquisition time identification reveals the time point when the image was acquired. These factors may affect the performance of the background color in the image.

[0038] Next, the acquisition coordinate identifier, the acquisition angle identifier, and the acquisition time identifier are used to access the target area background color database to match the background color features of the corresponding position, angle, and time point. Exemplarily, for coordinate matching, a neighbor search algorithm, such as a k-nearest neighbor (KNN) algorithm or a quadtree structure, can be used to spatially match the acquisition coordinate identifier with the coordinates stored in the database to find the closest record. Angle matching is to find the entry in the database whose shooting angle is closest to the acquisition angle identifier by calculating the angle difference, and the matching standard can be the minimum value of the angle difference. For time matching, a time period division method is used. The time field in the database can be stored according to the segmentation of time (for example, one segment per hour), and the system matches the acquisition time identifier with the corresponding time period in the database to find the closest record. Among them, the target area background color database is a database that stores the user-defined background color at any position in the target area, and contains the background color labels of each place in the entire space of the target area that change over time. The construction of the database needs to combine historical data with dynamically updated monitoring data to ensure accurate grasp of the background color changes in the target area. By matching the acquisition identifier with the records in the database, the background color features corresponding to the current cruise image can be accurately obtained, thus providing basic data for subsequent color fitting.

[0039] Finally, a special network (which may be a deep learning network, a machine learning model, or other types of color processing algorithms) is configured based on the sample color features to fit the base color features, background color features, and transparency of the ground film. This network can comprehensively consider the color properties of the ground film itself, the influence of the background color, and the transparency characteristics of the ground film. Through complex calculations and learning processes, it ultimately outputs a more accurate and realistic ground film sample color feature. This feature not only reflects the color properties of the ground film, but also includes the interaction with the background color and the influence of transparency on color performance, thereby greatly improving the accuracy and reliability of color matching.

[0040] Furthermore, step P31-3 of the embodiment of the present application also includes:

[0041] P31-31: Collecting a ground film benchmark color record dataset, a background color record dataset, a ground film transparency record dataset, and a color record dataset of images shot through the ground film; P31-32: Convolutional neural network topology architecture, configured with 3 input nodes and 1 output node; P31-33: Using the ground film benchmark color record dataset, the background color record dataset, and the ground film transparency record dataset as input data of the 3 input nodes, and using the color record dataset of images shot through the ground film as output supervision data of 1 output node, to train the sample color feature configuration network.

[0042] Specifically, the fitting accuracy of the color features of ground film samples is improved by training convolutional neural networks (CNN). First, four key data sets are systematically collected: ground film benchmark color record data set, background color record data set, ground film transparency record data set, and ground film color record data set. These data sets are the basis for subsequent training of convolutional neural networks, representing the color characteristics of the ground film itself, the color changes of the background environment, the transparency properties of the ground film, and the actual image color captured under the combined effect of these factors. The construction of these data sets requires precise measurement and recording techniques to ensure the accuracy and representativeness of the data.

[0043] Next, a specific topological architecture is designed for the convolutional neural network, which contains 3 input nodes and 1 output node. This design is based on the task requirements, that is, to predict the color of the image taken through the ground film by comprehensively considering the ground film reference color, background color and ground film transparency. Each input node corresponds to the ground film reference color record data set, the background color record data set, and the ground film transparency record data set, while the output node is responsible for outputting the predicted image color, which should match the real color in the color record data set of the image taken through the ground film. This design enables the network to learn the complex relationship between different input factors and make accurate predictions based on this.

[0044] Finally, the sample color feature configuration network is trained using the collected data set. Specifically, the ground film benchmark color record data set, the background color record data set, and the ground film transparency record data set are used as the input data of three input nodes, and the color record data set of images shot through the ground film is used as the output supervision data of one output node. Through the continuous iteration of the forward propagation and back-propagation algorithms, the network gradually adjusts its weights and bias parameters to minimize the difference between the predicted color and the true color, and obtains the sample color feature configuration network. This model provides strong technical support and guarantee for the subsequent ground film area color matching.

[0045] P40: The quasi-ground film calibration area is blown by a drone fan device, and a video of the quasi-ground film calibration area is captured along a position perpendicular to the blowing direction by a drone image acquisition device to obtain a monitoring video of the quasi-ground film calibration area.

[0046] Optionally, a drone fan device is used to physically intervene in the marked area of ​​the ground film, and then a video acquisition device is used to capture the behavior of the ground film after blowing, so as to further confirm the process of the position and status of the ground film.

[0047] First, the drone is not only equipped with an image acquisition device, but also a fan device. The function of this fan device is to apply airflow to the quasi-film calibration areas after identifying them, so that the film undergoes observable physical changes, such as shaking, movement or deformation. These changes will help the system further confirm the physical properties of the film (such as flexibility and elasticity) and verify the previous recognition results based on color and transparency.

[0048] Furthermore, to ensure the accuracy of video acquisition, the fan device will blow air towards the calibrated area of ​​the ground film according to a pre-set direction. The blowing direction is usually aligned with the geometric center of the calibrated area of ​​the ground film to ensure that the airflow is evenly applied to the entire ground film area. Secondly, the image acquisition device of the drone will capture video in a position perpendicular to the blowing direction. This means that the drone will adjust its flight path so that the camera can capture the movement of the ground film after blowing from multiple angles. This ensures that the captured video not only clearly shows the physical changes of the ground film, but also minimizes image occlusion or distortion caused by fan blowing.

[0049] Technically, the resolution and frame rate of the video acquisition device need to be high enough to ensure that the subtle shaking and deformation of the ground film can be captured. High frame rate can record fast movements, ensuring that the system can accurately analyze the flexibility coefficient of the ground film. In addition, the image stability and wind resistance of the video acquisition device are also important, because the drone may shake during flight and blowing, which will affect the image quality.

[0050] Finally, a set of monitoring videos of the quasi-membrane calibration area after wind intervention were obtained. These videos provide basic data for the subsequent flexibility coefficient analysis, help further confirm the actual physical state of the membrane, and accurately calibrate the location of the membrane, providing a reliable basis for the construction of the recovery path.

[0051] P50: Perform flexibility analysis on the monitoring video of the quasi-membrane calibration area to obtain a flexibility coefficient calibration value.

[0052] Furthermore, step P50 of the embodiment of the present application also includes:

[0053] P51: Perform frame division on the monitoring video of the quasi-film calibration area to obtain a monitoring video frame sequence of the quasi-film calibration area; P52: Perform video frame extraction on the monitoring video frame sequence of the quasi-film calibration area according to the first step length to obtain a first monitoring video frame sequence of the quasi-film calibration area, wherein the first step length is 16 frames to 18 frames; P53: Perform video frame extraction on the monitoring video frame sequence of the quasi-film calibration area according to the second step length to obtain a second monitoring video frame sequence of the quasi-film calibration area, wherein the second step length is equal to one eighth of the first step length; P54: Extract the first monitoring video frame sequence of the quasi-film calibration area The frame sequence is input into the SLOW static feature analysis channel of the SLOWFAST neural network to obtain the ground film positioning feature information; P55: the second quasi-ground film calibration area monitoring video frame sequence is input into the FAST dynamic feature analysis channel of the SLOWFAST neural network to obtain the calibration area motion feature information; P56: through the first feature fusion channel, based on the ground film positioning feature information, the edge coordinate time series information of the quasi-ground film calibration area monitoring video is extracted from the calibration area motion feature information; P57: through the second feature fusion channel, the edge coordinate time series information is fused to output the flexibility coefficient calibration value.

[0054] It should be understood that the SLOWFAST neural network is used to perform flexibility analysis on the surveillance video of the ground film calibration area to extract the calibration value of the flexibility coefficient of the ground film.

[0055] First, frame segmentation is performed, that is, the monitoring video of the quasi-film calibration area is segmented into a series of separate frames in time sequence to form a monitoring video frame sequence of the quasi-film calibration area. This step is the basis of video analysis and provides a basic data unit for subsequent processing.

[0056] Subsequently, in order to balance computational efficiency and accuracy, we adopted a two-step video frame extraction strategy. In the first step, video frames are extracted from the frame sequence according to the larger first step length to obtain the first quasi-ground film calibration area monitoring video frame sequence. These frames are mainly used to capture the static characteristics of the ground film, such as the overall coverage. Among them, the value range of the first step length is 16 frames to 18 frames, which can be adaptively adjusted according to the actual situation of the target area. In the second step, a second step length much smaller than the first step length (set to one eighth of the first step length) is used for extraction to obtain the second quasi-ground film calibration area monitoring video frame sequence. These frames focus on reflecting the dynamic changes of the ground film, such as fluctuations under wind.

[0057] Next, the two video frame sequences are processed in parallel using the advanced deep learning model, the SLOWFAST neural network. The network consists of two main channels: the SLOW static feature analysis channel and the FAST dynamic feature analysis channel. The first quasi-film calibration area monitoring video frame sequence is input into the SLOW channel, which uses its slower update speed and higher calculation accuracy to extract the static features of the film, such as color, texture, etc., to obtain the film positioning feature information. The second quasi-film calibration area monitoring video frame sequence is input into the FAST channel, which captures the dynamic changes of the film, such as motion trajectory, speed, etc., at a faster update speed to obtain the motion feature information of the calibration area.

[0058] After feature extraction, the edge coordinate time series information is extracted from the motion feature information of the calibration area based on the ground film positioning feature information through the first feature fusion channel. Specifically, first, the initial edge position of the ground film in the video is locked using the ground film positioning feature information, that is, the static edge data. These static edge data lay the foundation for the extraction of motion feature information. After locking the initial edge position of the ground film, the changes in the edge of the ground film are captured in the motion feature information through image processing and feature tracking algorithms, such as optical flow method, edge detection and key point tracking, etc., with the initial edge position as the starting point, and the changes are converted into time series data to obtain the edge coordinate time series information. The edge coordinate time series information describes the position change of the edge of the ground film on the time axis. It not only includes the specific coordinates of the edge of the ground film in each frame, but also describes the change trajectory of these coordinates over time. These data can help the system capture the flexible performance of the ground film, such as how the edge is displaced and deformed under the action of wind.

[0059] Finally, the second feature fusion channel deeply fuses the edge coordinate time series information with other static and dynamic features, and outputs the flexibility coefficient calibration value by combining the results of static and dynamic analysis. The flexibility coefficient calibration value reflects the flexibility characteristics of the ground film under the action of wind and is a key parameter for measuring the physical properties of the ground film. This value can help subsequent recycling equipment adjust its strategy to adapt to ground films with different flexibility.

[0060] Furthermore, step P57 of the embodiment of the present application also includes:

[0061] P57-1: The edge coordinate timing information includes an edge coordinate sequence at the first moment, wherein the edge coordinate sequence at the first moment has an edge windless position identification sequence; P57-2: The edge coordinate timing information includes an edge coordinate sequence at the Nth moment, wherein the edge coordinate sequence at the Nth moment has an edge windless position identification sequence; P57-3: Based on the edge windless position identification sequence, calculate the same-position variance of the edge coordinate sequence from the first moment to the edge coordinate sequence at the Nth moment, and obtain a number of edge coordinate variance eigenvalues; P57-4: Count the proportion of the several edge coordinate variance eigenvalues ​​that are greater than or equal to the variance eigenvalue threshold, and set it as the flexibility coefficient calibration value.

[0062] Specifically, the specific process of fusing the edge coordinate time series information and outputting the flexibility coefficient calibration value may be, first, clarifying the composition of the edge coordinate time series information. In the video frame sequence, each frame contains the edge information of the ground film, which is represented in the form of a coordinate sequence and forms time series information over time. In particular, the edge position at the first moment (i.e., the initial moment or reference moment) is selected as the edge windless position identification sequence. This setting is based on the assumption that in a windless or stationary state, the edge of the ground film will remain in a relatively stable position. Subsequently, for the Nth moment in the video (N is an arbitrary positive integer, representing a specific point in the time series), its edge coordinate sequence is also extracted, and the edge windless position identification corresponding to the first moment is retained.

[0063] Next, the edge position information is used to calculate the edge coordinate variance eigenvalue. Specifically, for each corresponding position (i.e., the edge coordinates of the same position at the first moment and the Nth moment), its variance is calculated. Variance is a statistic that measures the degree of data dispersion, and here it reflects the fluctuation of the edge coordinates of the position over time. The larger the variance, the more significant the change of the edge coordinates of the position in the time series, that is, the greater the fluctuation state of the ground film, thus showing stronger flexibility. Through this process, several edge coordinate variance eigenvalues ​​are obtained, each of which corresponds to the flexibility characteristics of a certain position on the ground film.

[0064] Finally, in order to comprehensively evaluate the flexibility of the entire mulch, statistical operations are performed. A variance eigenvalue threshold is set, which is determined according to the actual application requirements and the previous experimental data to distinguish significant fluctuations from normal minor changes. Then, the proportion of all edge coordinate variance eigenvalues ​​greater than or equal to the threshold is counted, and this proportion is set as the flexibility coefficient calibration value. This calibration value intuitively reflects the overall degree of flexibility of the mulch during the monitoring period, providing a quantitative evaluation basis for agricultural managers. In this way, not only can the specific flexibility characteristics of the mulch be identified, but also the flexibility state of the entire mulch can be comprehensively evaluated.

[0065] P60: When the flexibility coefficient calibration value is greater than or equal to the flexibility coefficient threshold, the quasi-ground film calibration area is added into the ground film calibration area.

[0066] It should be understood that, based on the previously calculated flexibility coefficient calibration value, further evaluation and decision-making are made on the ground film covering state of the ground film calibration area. First, a flexibility coefficient threshold is set, which is determined based on the analysis of a large amount of experimental data and actual application scenarios, and is used to distinguish whether the flexibility in the ground film covering state has reached a level that requires special attention. The setting of the flexibility coefficient threshold takes into account a variety of factors, including the material, thickness, covering method of the ground film, and local climatic conditions.

[0067] Next, the calculated flexibility coefficient calibration value is compared with the set flexibility coefficient threshold. If the flexibility coefficient calibration value is greater than or equal to the flexibility coefficient threshold, it means that during the monitoring period, the ground film in the quasi-ground film calibration area has shown significant flexibility, that is, the ground film has undergone large fluctuations or deformations under the action of wind or other external forces. This means that the ground film in this area has shown sufficient flexibility under the action of wind, indicating that there is indeed ground film in this area and its physical state meets the recycling standards.

[0068] Once it is determined that the area meets the conditions, the area will be upgraded from a quasi-film calibration area to a formal film calibration area. This calibration area will serve as a clear target area for film recycling. The system will summarize the information of these areas and provide accurate positioning data for subsequent recycling equipment. Through flexible analysis, the system not only relies on visual features (such as color and transparency), but also combines physical properties to ensure the reliability and accuracy of the calibration area. It improves the efficiency of film recycling, reduces resource waste, and lays a solid foundation for subsequent path planning and recycling execution.

[0069] Furthermore, the embodiment of the present application further includes step P60a, and step P60a further includes:

[0070] P61a: When the flexibility coefficient calibration value is less than the flexibility coefficient threshold, collect the historical color features of the quasi-ground film calibration area, wherein the historical color features are the color features when there is no ground film coverage; P62a: Analyze the repetition frequency of the historical color features and the detected color features of the cruise image of the target area is less than or equal to the color deviation threshold, wherein the color deviation threshold is a color difference value of 1.2; P63a: When the repetition frequency is less than or equal to the repetition frequency threshold, add the quasi-ground film calibration area into the ground film calibration area, wherein the repetition frequency threshold is 20% of the number of historical color feature collection samples.

[0071] In a possible embodiment of the present application, when the flexibility coefficient calibration value is lower than the preset flexibility coefficient threshold, the system cannot confirm whether there is a ground film in the area based on the flexibility characteristics alone. Therefore, the historical color characteristics of the area when there is no ground film coverage can be retrieved from the database. These historical color characteristics represent the normal color state of the area, that is, the natural color information when it is not covered by the ground film, such as the color of bare soil or vegetation.

[0072] Next, the color features detected in the cruise image of the target area are compared with the historical color features of the area. A color deviation threshold is preset to measure the difference between the current color and the historical color. The color deviation threshold can be set according to the ground film characteristics of the current area and recognition experience. For example, the color deviation threshold is set to a color difference value of 1.2. If the deviation between the detected color and the historical color is less than or equal to the color deviation threshold, it is considered that the color of the current area is highly similar to the historical color. The system will further calculate the repetition frequency of this color similarity, that is, the frequency of multiple detections of the color feature.

[0073] Finally, the calculated repetition frequency is compared with the preset repetition frequency threshold. The repetition frequency threshold is 20% of the number of samples collected from historical color features, which can be set based on the number of historical identification samples and historical identification experience. If the repetition frequency of similar color features is less than or equal to the repetition frequency threshold, it indicates that the color of the area is significantly different from the historical color, indicating that the area may be covered with ground film. In this case, although the calibration value of the flexibility coefficient is low, the area is still added to the ground film calibration area so that agricultural managers can notice this problem and take corresponding management measures. This technology greatly reduces the possibility of missed judgments due to insufficient sensitivity of flexibility analysis, improves the comprehensiveness and accuracy of ground film identification, and provides more accurate data support for subsequent ground film recovery path planning.

[0074] P70: According to the ground film calibration area, a ground film recovery trajectory is constructed and sent to the ground film recovery control end.

[0075] Specifically, by using the marked ground film area, an accurate ground film recovery trajectory is constructed, and the trajectory is sent to the ground film recovery control end to perform the actual recovery operation. For example, first, the coordinate information and geometric features of the ground film calibration area are combined with the operating capabilities of the drone or other recovery equipment to calculate an optimal path covering all ground film calibration areas. This path takes into account the distribution density of the ground film area, the operating radius of the recovery equipment, and the path optimization algorithm (such as the shortest path algorithm or the coverage path planning algorithm) to ensure that the equipment can efficiently cover all ground film calibration areas.

[0076] Furthermore, after the film recycling trajectory is constructed, it is converted into a format that is easy to understand and operate (such as GPS coordinate sequence, map annotation, etc.), and sent to the film recycling control end through a wireless network or a dedicated communication channel. The film recycling control end is a central control system, mobile application or remote monitoring platform, which is responsible for receiving, processing and displaying the recycling trajectory information, and sending instructions to recycling personnel or equipment to guide them to carry out film recycling work according to the predetermined trajectory.

[0077] In addition, in order to ensure the smooth progress of recycling work, functional modules such as real-time monitoring, scheduling management, and data analysis can also be integrated into the mulch film recycling control end. These modules can monitor the recycling progress, adjust the recycling plan, and analyze the recycling efficiency in real time, providing comprehensive decision-making support for agricultural managers.

[0078] In summary, the embodiments of the present application have at least the following technical effects:

[0079] The present application collects cruise images of the target area through drone patrol, and performs color matching of the film area according to the film baseline color characteristics and film transparency to obtain a quasi-film calibration area. The drone fan device is used to blow towards the film calibration area, and surveillance video of the quasi-film calibration area is collected for flexible analysis. The film calibration area is determined according to the analysis results, and a film recovery trajectory is constructed to conduct film recovery management and control.

[0080] The technical effect of efficiently and accurately identifying farmland mulch and planning its recovery trajectory based on deep video analysis has been achieved, thereby improving the efficiency of mulch recovery.

[0081] Embodiment 2

[0082] Based on the same inventive concept as the method for identifying ground film recovery tracks based on deep video analysis in the aforementioned embodiment, Figure 3 As shown, the present application provides a device for identifying the trajectory of ground film recovery based on deep video analysis. The device and method embodiments in the present application are based on the same inventive concept. The device includes:

[0083] The cruise image acquisition module 11 is used to collect cruise images of the target area by cruising the unmanned aerial vehicle.

[0084] The ground film reference feature acquisition module 12 is used to obtain the ground film reference color feature and ground film transparency.

[0085] The ground film area color matching module 13 is used to perform ground film area color matching on the target area cruise image according to the ground film reference color characteristics and the ground film transparency to obtain a quasi-ground film calibration area.

[0086] The calibration area video acquisition module 14 is used to blow the quasi-membrane calibration area through the drone fan device, and to acquire the video of the quasi-membrane calibration area along the position perpendicular to the blowing direction through the drone image acquisition device to obtain the monitoring video of the quasi-membrane calibration area.

[0087] The flexibility analysis module 15 is used to perform flexibility analysis on the monitoring video of the quasi-membrane calibration area to obtain a flexibility coefficient calibration value.

[0088] The ground film calibration area division module 16 is used for adding the quasi-ground film calibration area into the ground film calibration area when the flexibility coefficient calibration value is greater than or equal to the flexibility coefficient threshold.

[0089] The ground film recovery control module 17 is used to construct a ground film recovery trajectory according to the ground film calibration area and send it to the ground film recovery control terminal.

[0090] Furthermore, the film area color matching module 13 is also used to perform the following steps:

[0091] When the transparency of the ground film is greater than or equal to a transparency threshold, the ground film sample color is identified based on the ground film reference color feature and the background color feature to obtain the ground film sample color feature; when the ground film transparency is less than the transparency threshold, the ground film reference color feature is set as the ground film sample color feature; and the ground film area color is matched on the cruise image of the target area according to the ground film sample color feature to obtain the quasi-ground film calibration area.

[0092] Furthermore, the film area color matching module 13 is also used to perform the following steps:

[0093] Obtain the acquisition coordinate identifier, acquisition angle identifier and acquisition time identifier of the cruise image of the target area; match the background color feature through the target area background color database according to the acquisition coordinate identifier, the acquisition angle identifier and the acquisition time identifier; configure the network according to the sample color feature, fit the ground film reference color feature, the background color feature and the ground film transparency, and obtain the ground film sample color feature.

[0094] Furthermore, the film area color matching module 13 is also used to perform the following steps:

[0095] Collect a ground film benchmark color record dataset, a background color record dataset, a ground film transparency record dataset, and a color record dataset of images shot through the ground film; configure three input nodes and one output node for a convolutional neural network topology architecture; use the ground film benchmark color record dataset, the background color record dataset, and the ground film transparency record dataset as input data of the three input nodes, and use the color record dataset of images shot through the ground film as output supervision data of one output node to train the sample color feature configuration network.

[0096] Furthermore, the flexibility analysis module 15 is also used to perform the following steps:

[0097] Perform frame segmentation on the monitoring video of the quasi-ground film calibration area to obtain a monitoring video frame sequence of the quasi-ground film calibration area; perform video frame extraction on the monitoring video frame sequence of the quasi-ground film calibration area according to the first step length to obtain a first monitoring video frame sequence of the quasi-ground film calibration area; perform video frame extraction on the monitoring video frame sequence of the quasi-ground film calibration area according to the second step length to obtain a second monitoring video frame sequence of the quasi-ground film calibration area, wherein the second step length is equal to one eighth of the first step length; input the first monitoring video frame sequence of the quasi-ground film calibration area into the SLOW static feature analysis channel of the SLOWFAST neural network; input the second monitoring video frame sequence of the quasi-ground film calibration area into the FAST dynamic feature analysis channel of the SLOWFAST neural network; extract the edge coordinate time series information of the monitoring video of the quasi-ground film calibration area through the first feature fusion channel; fuse the edge coordinate time series information through the second feature fusion channel, and output the flexibility coefficient calibration value.

[0098] Furthermore, the flexibility analysis module 15 is also used to perform the following steps:

[0099] The edge coordinate timing information includes an edge coordinate sequence at a first moment, wherein the edge coordinate sequence at the first moment has an edge windless position identification sequence; the edge coordinate timing information includes an edge coordinate sequence at an Nth moment, wherein the edge coordinate sequence at the Nth moment has an edge windless position identification sequence; based on the edge windless position identification sequence, the same-position variance of the edge coordinate sequence from the first moment to the edge coordinate sequence at the Nth moment is calculated to obtain a number of edge coordinate variance eigenvalues; and the proportion of the several edge coordinate variance eigenvalues ​​that is greater than or equal to the variance eigenvalue threshold is counted and set as the flexibility coefficient calibration value.

[0100] Furthermore, the ground film calibration area division module 16 is also used to perform the following steps:

[0101] When the flexibility coefficient calibration value is less than the flexibility coefficient threshold, the historical color features of the quasi-ground film calibration area are collected, wherein the historical color features are the color features when there is no ground film coverage; the repetition frequency of the historical color features and the detected color features of the cruise image of the target area is analyzed to be less than or equal to the color deviation threshold; when the repetition frequency is less than or equal to the repetition frequency threshold, the quasi-ground film calibration area is added to the ground film calibration area.

[0102] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0104] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A method for identifying the trajectory of ground film recovery based on deep video analysis, characterized in that: include: Collect cruise images of the target area through drone cruise; Obtaining the base color characteristics and transparency of the ground film; According to the ground film reference color characteristics and the ground film transparency, color matching of the ground film area is performed on the cruise image of the target area to obtain a quasi-ground film calibration area; The quasi-film calibration area is blown by a drone fan device, and a video of the quasi-film calibration area is captured by a drone image acquisition device along a position perpendicular to the blowing direction to obtain a monitoring video of the quasi-film calibration area; Performing flexibility analysis on the monitoring video of the quasi-membrane calibration area to obtain a flexibility coefficient calibration value; When the flexibility coefficient calibration value is greater than or equal to the flexibility coefficient threshold, the quasi-ground film calibration area is added into the ground film calibration area; According to the ground film calibration area, a ground film recovery trajectory is constructed and sent to the ground film recovery control terminal; The flexibility analysis is performed on the monitoring video of the quasi-membrane calibration area to obtain the flexibility coefficient calibration value, including: Performing frame segmentation on the monitoring video of the quasi-film calibration area to obtain a monitoring video frame sequence of the quasi-film calibration area; Extracting video frames from the monitoring video frame sequence of the quasi-film calibration area according to the first step length to obtain a first monitoring video frame sequence of the quasi-film calibration area, wherein the first step length is 16 frames to 18 frames; Extracting video frames from the quasi-film calibration area monitoring video frame sequence according to a second step length to obtain a second quasi-film calibration area monitoring video frame sequence, wherein the second step length is equal to one eighth of the first step length; Inputting the first quasi-ground film calibration area monitoring video frame sequence into the SLOW static feature analysis channel of the SLOWFAST neural network to obtain ground film positioning feature information; Inputting the second quasi-film calibration area monitoring video frame sequence into the FAST dynamic feature analysis channel of the SLOWFAST neural network to obtain the calibration area motion feature information; Extracting edge coordinate time series information of the monitoring video of the quasi-film calibration area from the calibration area motion feature information based on the film positioning feature information through a first feature fusion channel; Through a second feature fusion channel, the edge coordinate time series information is fused to output the flexibility coefficient calibration value; The edge coordinate time series information is fused through the second feature fusion channel to output the flexibility coefficient calibration value, including: The edge coordinate time series information includes a first moment edge coordinate sequence, wherein the first moment edge coordinate sequence has an edge windless position identification sequence; The edge coordinate time series information includes an edge coordinate sequence at the Nth moment, wherein the edge coordinate sequence at the Nth moment has an edge windless position identification sequence; According to the edge calm position identification sequence, calculating the same-position variance of the edge coordinate sequence from the first moment to the edge coordinate sequence at the Nth moment, and obtaining a plurality of edge coordinate variance eigenvalues; The proportion of the plurality of edge coordinate variance eigenvalues ​​that is greater than or equal to the variance eigenvalue threshold is counted and set as the flexibility coefficient calibration value.

2. The method according to claim 1, characterized in that According to the ground film reference color feature and the ground film transparency, the ground film area color matching is performed on the target area cruise image to obtain a quasi-ground film calibration area, including: When the transparency of the ground film is greater than or equal to a transparency threshold, the ground film sample color is identified based on the ground film reference color feature and the background color feature to obtain the ground film sample color feature; When the transparency of the ground film is less than the transparency threshold, setting the ground film reference color feature as the ground film sample color feature; The film area color is matched on the target area cruise image according to the color characteristics of the film sample to obtain the quasi-film calibration area.

3. The method according to claim 2, characterized in that When the transparency of the ground film is greater than or equal to the transparency threshold, the ground film sample color is identified based on the ground film reference color feature and the background color feature to obtain the ground film sample color feature, including: Obtaining the acquisition coordinate identification, acquisition angle identification and acquisition time identification of the cruise image of the target area; According to the acquisition coordinate identifier, the acquisition angle identifier and the acquisition time identifier, the background color feature is matched through a target area background color database, wherein the target area background color database is a database storing user-defined background colors at any position of the target area; A network is configured according to the sample color features, and the base color features of the ground film, the background color features and the transparency of the ground film are fitted to obtain the sample color features of the ground film.

4. The method according to claim 3, characterized in that According to the sample color feature, a network is configured, and the base color feature of the ground film, the background color feature and the ground film transparency are fitted to obtain the ground film sample color feature, which includes: Collecting ground film base color record data set, background color record data set, ground film transparency record data set and ground film shot image color record data set; For the convolutional neural network topology, configure 3 input nodes and 1 output node; The sample color feature configuration network is trained by using the ground film reference color record dataset, the background color record dataset, and the ground film transparency record dataset as input data of three input nodes, and using the ground film shot image color record dataset as output supervision data of one output node.

5. The method according to claim 1, characterized in that Also includes: When the flexibility coefficient calibration value is less than the flexibility coefficient threshold, collecting the historical color features of the quasi-ground film calibration area, wherein the historical color features are color features when there is no ground film coverage; Analyzing the repetition frequency of the historical color features and the detected color features of the target area cruise image being less than or equal to a color deviation threshold, wherein the color deviation threshold is a color difference value of 1.2; When the repetition frequency is less than or equal to a repetition frequency threshold, the quasi-ground film calibration area is added into the ground film calibration area, wherein the repetition frequency threshold is 20% of the number of historical color feature collection samples.

6. A device for identifying the trajectory of ground film recovery based on deep video analysis, characterized in that: The device comprises: A cruise image acquisition module, wherein the cruise image acquisition module is used to collect cruise images of a target area by cruising the drone; A ground film reference feature acquisition module, wherein the ground film reference feature acquisition module is used to obtain ground film reference color features and ground film transparency; A ground film area color matching module, the ground film area color matching module is used to perform ground film area color matching on the target area cruise image according to the ground film reference color characteristics and the ground film transparency, so as to obtain a quasi-ground film calibration area; A calibration area video acquisition module, wherein the calibration area video acquisition module is used to blow the quasi-film calibration area through a drone fan device, and to acquire a video of the quasi-film calibration area along a position perpendicular to the blowing direction through a drone image acquisition device, so as to obtain a monitoring video of the quasi-film calibration area; A flexibility analysis module, the flexibility analysis module is used to perform flexibility analysis on the monitoring video of the quasi-membrane calibration area to obtain a flexibility coefficient calibration value; A ground film calibration area division module, wherein the ground film calibration area division module is used to add the quasi-ground film calibration area into the ground film calibration area when the flexibility coefficient calibration value is greater than or equal to the flexibility coefficient threshold; A ground film recovery control module, which is used to construct a ground film recovery trajectory according to the ground film calibration area and send it to the ground film recovery control terminal; The flexibility analysis module is further used to perform the following steps: Perform frame segmentation on the monitoring video of the quasi-ground film calibration area to obtain a monitoring video frame sequence of the quasi-ground film calibration area; perform video frame extraction on the monitoring video frame sequence of the quasi-ground film calibration area according to the first step length to obtain a first monitoring video frame sequence of the quasi-ground film calibration area, wherein the first step length is 16 frames to 18 frames; perform video frame extraction on the monitoring video frame sequence of the quasi-ground film calibration area according to the second step length to obtain a second monitoring video frame sequence of the quasi-ground film calibration area, wherein the second step length is equal to one eighth of the first step length; input the first monitoring video frame sequence of the quasi-ground film calibration area into the SLOW static feature analysis channel of the SLOWFAST neural network to obtain ground film positioning feature information; input the second monitoring video frame sequence of the quasi-ground film calibration area into the FAST dynamic feature analysis channel of the SLOWFAST neural network to obtain calibration area motion feature information; A feature fusion channel is used to extract the edge coordinate time series information of the monitoring video of the quasi-ground film calibration area from the motion feature information of the calibration area based on the ground film positioning feature information; through a second feature fusion channel, the edge coordinate time series information is fused to output the flexibility coefficient calibration value; the edge coordinate time series information includes an edge coordinate sequence at a first moment, wherein the edge coordinate sequence at the first moment has an edge windless position identification sequence; the edge coordinate time series information includes an edge coordinate sequence at an Nth moment, wherein the edge coordinate sequence at the Nth moment has an edge windless position identification sequence; according to the edge windless position identification sequence, the same-position variance of the edge coordinate sequence from the first moment to the edge coordinate sequence at the Nth moment is calculated to obtain a number of edge coordinate variance eigenvalues; the proportion of the several edge coordinate variance eigenvalues ​​that are greater than or equal to the variance eigenvalue threshold is counted and set as the flexibility coefficient calibration value.

Images