Residual film recovery tool real-time operation supervision method, device, equipment, medium and product

Abstract

The application discloses a residual film recycling tool real-time operation monitoring method, device, equipment, medium and product, relates to the field of intelligent agricultural equipment, and comprises the following steps: acquiring scene images, position coordinates, angles between pull rods and horizontal ground, and main shaft rotating speeds at each moment in the working process of the residual film recycling tool; inputting the scene images at each moment into a classification and recognition model to determine whether the images are valid; inputting the valid images into a semantic segmentation model and calculating the picking rate at each moment based on the ratio of the total number of residual film pixel points in the images to the total number of image pixels; updating triangles according to the position coordinates at each moment by using a triangle subdivision algorithm; obtaining the working area at each moment by subtracting the overlapping area from the total area of the triangles at each moment; and calculating the film tooth depth at each moment according to the angles at each moment and the length of the pull rod. The application can monitor the picking rate, working area and working state (main shaft rotating speed and film tooth depth) of the residual film recycling tool in real time.

Description

Technical Field

[0001] The present application relates to the field of smart agricultural equipment, and in particular to a method, device, equipment, medium and product for real-time operation supervision of residual film recovery equipment. Background Art

[0002] Agricultural mulch film is mainly made of polyethylene, but the current residual film recycling equipment often malfunctions during operation, has low operational reliability and a low residual film pickup rate. The lack of effective residual film recycling supervision technology has resulted in insignificant effects on the control of residual film pollution in farmland, causing serious non-point source pollution of the soil.

[0003] Currently, there is no method that can monitor the pickup rate, operating area, and operating status (spindle speed and film-removing tooth depth) of residual film recovery equipment in real time. Summary of the Invention

[0004] The purpose of this application is to provide a real-time operation supervision method, device, equipment, medium and product for residual film recovery equipment, which can monitor the picking rate, operating area and operating status (spindle speed and film tooth depth) of the residual film recovery equipment in real time.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for real-time operation supervision of a residual film recovery machine, comprising:

[0007] At any moment during the operation of the residual film recovery machine, obtain the scene image of the residual film recovery machine at the current moment, the position coordinates of the residual film recovery machine, the main shaft speed of the residual film recovery machine, and the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground;

[0008] Inputting the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid;

[0009] If the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; the category is residual film or not residual film;

[0010] Determine the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment as the pickup rate of the residual film recycling machine at the current moment;

[0011] Using the triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recycling machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained based on the vertices of the residual film recycling machine's operating area;

[0012] The operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment;

[0013] The film tooth depth at the current moment is calculated according to the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool at the current moment.

[0014] Optionally, the classification recognition model is obtained by replacing the first convolution module in the DenseNet network model with the first multi-scale serial hole convolution module, and embedding each nonlinear combination function in each dense block in the DenseNet network model into the SE-Block module before and after.

[0015] Optionally, the semantic segmentation model includes an encoder and a decoder;

[0016] The encoder includes a second multi-scale cascaded hole convolution module, N maximum pooling downsampling modules and N+1 SE-Dense Block convolution modules;

[0017] The SE-Dense Block convolution module includes a SE-Dense Block module and a first convolution module connected in sequence; the output end of the second multi-scale serial hole convolution module is connected to the input end of the SE-Dense Block module in the first SE-Dense Block convolution module; the output end of the first convolution module in the i-th SE-Dense Block convolution module is connected to the input end of the i-th maximum pooling downsampling module, and the output end of the i-th maximum pooling downsampling module is connected to the input end of the first convolution module in the i+1-th SE-Dense Block convolution module, 1≤i≤N; the SE-Dense Block module is obtained by embedding each nonlinear combination function in any dense block in the DenseNet network model into the SE-Block module front and back;

[0018] The decoder includes N+1 sequentially connected upsampling convolution modules; the upsampling convolution module includes sequentially connected upsampling modules and convolution layers; the convolution layer includes multiple sequentially connected second convolution modules;

[0019] The first convolution module in the encoder is connected to the upsampling module in the first upsampling convolution module in the decoder;

[0020] The nth SE-Dense Block module in the encoder is jump-connected to the first second convolution module in the N+2-nth upsampling convolution module in the decoder, 1≤n≤N+1.

[0021] In a second aspect, the present application provides a real-time operation monitoring device for a residual film recovery machine, which is used to implement the above-mentioned real-time operation monitoring method for a residual film recovery machine. The real-time operation monitoring device for a residual film recovery machine includes:

[0022] A cloud server and a camera, a positioning device, a tilt sensor, and a Hall sensor respectively connected to the cloud server;

[0023] At any moment during the operation of the residual film recovery machine, the camera is used to obtain a scene image of the residual film recovery machine at the current moment, the positioning device is used to obtain the position coordinates of the residual film recovery machine at the current moment, the inclination sensor is used to measure the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment; the Hall sensor is used to measure the pulse signal of the main shaft speed of the residual film recovery machine at the current moment;

[0024] The cloud server is used to input the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; determine that the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is the picking rate of the residual film recycling machine at the current moment; the category is residual film or not residual film;

[0025] The cloud server is further configured to use a triangulation algorithm to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area; and the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment;

[0026] The cloud server is also used to calculate the film tooth depth at the current moment based on the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool;

[0027] The cloud server is also used to calculate the spindle speed of the residual film recovery machine at the current moment based on the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor.

[0028] Optionally, the residual film recovery tool real-time operation monitoring device further comprises:

[0029] an intelligent display control terminal and a remote communication module, the intelligent display control terminal being connected with the remote communication module, a camera, a positioning device, an inclination sensor and a Hall sensor respectively;

[0030] The intelligent display control terminal is configured to send, through the remote communication module, a scene image of the residual film recovery tool at the current time acquired by the camera, a position coordinate of the residual film recovery tool at the current time acquired by the positioning device, an angle between a pull rod on a traction frame on the residual film recovery tool and a horizontal ground at the current time measured by the inclination sensor, and a pulse signal of a main shaft rotating speed of the residual film recovery tool at the current time measured by the Hall sensor to a cloud server.

[0031] The cloud server is further configured to send, through the remote communication module, a pickup rate of the residual film recovery tool at the current time, an operation area of the residual film recovery tool at the current time, a film tooth depth at the current time, and the main shaft rotating speed of the residual film recovery tool at the current time to the intelligent display control terminal for display.

[0032] Optionally, the residual film recovery tool real-time operation monitoring device further comprises:

[0033] an alarm lamp connected with the intelligent display control terminal, the intelligent display control terminal being configured to control the alarm lamp to alarm according to the film tooth depth at the current time and the main shaft rotating speed of the residual film recovery tool at the current time.

[0034] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the residual film recovery tool real-time operation monitoring method according to any one of the above.

[0035] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the residual film recovery tool real-time operation monitoring method according to any one of the above.

[0036] In a fifth aspect, the present application provides a computer program product comprising a computer program executable by a processor to implement the residual film recovery tool real-time operation monitoring method according to any one of the above.

[0037] According to the embodiments provided in the present application, the present application has the following technical effects:

[0038] The present application provides a method, device, equipment, medium and product for real-time operation supervision of residual film recycling machines. The method inputs the scene image of the residual film recycling machine at the current moment into a classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into a semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is determined as the picking rate of the residual film recycling machine at the current moment; the picking rate of the residual film recycling machine is monitored in real time , using a triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment, and the operating area of ​​the residual film recovery machine is monitored in real time; the film tooth depth at the current moment is calculated according to the angle between the pull rod on the traction frame of the residual film recovery machine at the current moment and the horizontal ground and the length of the pull rod on the traction frame of the residual film recovery machine, and the film tooth depth of the residual film recovery machine is monitored in real time, and the spindle speed of the residual film recovery machine can be directly obtained, and the spindle speed of the residual film recovery machine can be monitored in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A flow chart of a real-time operation supervision method for a residual film recovery machine provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the structure of a real-time operation monitoring device for a residual film recovery machine provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the workflow for calculating the pickup rate in the real-time operation monitoring device for residual film recycling equipment provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the workflow for calculating the operating area in the real-time operation monitoring device for residual film recovery equipment provided in an embodiment of the present application;

[0044] Figure 5A schematic diagram of the workflow for monitoring the spindle speed in the real-time operation monitoring device for the residual film recovery machine provided in an embodiment of the present application;

[0045] Figure 6 This is the data collection flow chart for the tilt sensor;

[0046] Figure 7 A schematic diagram of the cloud server workflow provided in an embodiment of the present application;

[0047] Figure 8 This is a diagram of the SE-DenseNet-DC model structure provided in the embodiments of the present application;

[0048] Figure 9 This is a structural diagram of the SE-DenseNet-DC segmentation model provided in an embodiment of the present application;

[0049] Figure 10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0050] Reference numerals:

[0051] 1—Intelligent display and control terminal, 2—Camera, 3—Positioning device, 4—Warning light, 5—Tilt sensor, 6—Hall sensor, 7—Moulding tooth. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0054] In an exemplary embodiment, a method for real-time operation supervision of a residual film recovery machine is provided, such as Figure 1 As shown, the following steps are included, wherein:

[0055] Step 201: At any moment during the operation of the residual film recovery machine, obtain the scene image of the residual film recovery machine at the current moment, the position coordinates of the residual film recovery machine, the spindle speed of the residual film recovery machine, and the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground.

[0056] Step 202: input the scene image of the residual film recycling machine at the current time into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current time is valid. If the scene image of the residual film recycling machine at the current time is a post-collection valid image, i.e., an image of the residual film recycling machine after processing, the scene image of the residual film recycling machine at the current time is valid.

[0057] Step 203: if the scene image of the residual film recycling machine at the current time is valid, input the scene image of the residual film recycling machine at the current time into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current time; the category is residual film or not residual film.

[0058] Step 204: determine the ratio of the total number of pixel points of the residual film in the scene image of the residual film recycling machine at the current time to the total number of pixels of the scene image of the residual film recycling machine at the current time as the pickup rate of the residual film recycling machine at the current time.

[0059] Step 205: update the triangle at the previous time according to the position coordinates of the residual film recycling machine at the current time to obtain the triangle at the current time by using a triangle subdivision algorithm; the triangle at the initial time is obtained from the vertices of the working area of the residual film recycling machine. Specifically, the triangle is obtained by processing the vertices of the working area by using the triangle subdivision algorithm.

[0060] Step 206: obtain the working area of the residual film recycling machine at the current time by subtracting the overlapping area between the triangles at the current time from the total area of the triangles at the current time.

[0061] Step 207: calculate the film lifting tooth depth at the current time according to the angle between the pull rod on the towing frame of the residual film recycling machine and the horizontal ground and the length of the pull rod on the towing frame of the residual film recycling machine.

[0062] By implementing the above steps 201 to 207, the pickup rate, working area, and working state (main shaft speed and film lifting tooth depth) of the residual film recycling machine can be monitored in real time.

[0063] In an exemplary embodiment, the triangle at the previous time is updated according to the position coordinates of the residual film recycling machine at the current time to obtain the triangle at the current time by using a triangle subdivision algorithm, specifically including:

[0064] The position coordinates of the residual film recycling machine at the current time are converted into plane coordinate system coordinates by using a Gaussian projection algorithm.

[0065] The triangle at the previous time is updated according to the plane coordinate system coordinates to obtain the triangle at the current time by using a triangle subdivision algorithm.

[0066] Specifically, after converting the coordinates of the trajectory points to a plane coordinate system, the trajectory points on the plane can be used to calculate the work area. The coordinates of these trajectory points are defined as a finite point set P, which is constructed into a larger initial triangle to ensure that all points in the set are included.

[0067] Each point in the point set is added to the triangle in turn. Each time a new point is inserted, a line is constructed with the vertices of the original triangle, thereby generating three new triangles. Check whether there are other points on the circumcircles corresponding to these newly formed triangles besides the vertices that constitute the triangle.

[0068] When two adjacent triangles form a quadrilateral, check whether the fourth vertex is within the circumcircle. If so, it means that the current segmentation does not meet the conditions, so the diagonals of the quadrilateral need to be swapped. If the fourth vertex is not within any circumcircle, the current segmentation remains unchanged. Finally, a triangulated network that meets the conditions is constructed. The specific steps are as follows: Figure 4 shown.

[0069] In an exemplary embodiment, the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment, specifically including:

[0070] Calculate the sum of the areas of all triangles at the current moment to get the rough working area.

[0071] The sum of the overlapping areas between the triangles at the current moment is calculated to obtain the repeated operation area.

[0072] The precise operating area is obtained by subtracting the repeated operating area from the rough operating area, that is, the operating area of ​​the residual film recovery machine at the current moment.

[0073] Specifically, the boundary of the work plot is set as a closed polygon composed of n vertices, and the coordinates of each vertex are expressed as A1(x1, y1), A2(x2, y2), ..., ..., A i (x i ,y i )(where i = 1, 2, 3, ..., n). Therefore, the formula for calculating the rough working area S by calculating the sum of the areas of each triangle at the current moment is as follows:

[0074]

[0075]

[0076]

[0077] In the above formula, x n+1 =x1,y n+1 =y1.

[0078] Calculate the sum of the overlapping areas of the triangles at the current moment to get the repeated operation area. The specific steps are:

[0079] Define Grid: Divide the entire work area into grid cells of equal size. For example, set the grid size to 1×1×1 units.

[0080] Mark grid cells: Traverse each triangle and mark the grid cells covered by the triangle. If multiple operation paths cover the same grid cell, it will be marked as a repeated operation area.

[0081] The sum of the areas of all repeated operation areas is calculated to obtain the repeated operation area.

[0082] In an exemplary embodiment, the film tooth depth at the current moment is calculated based on the angle α between the upper pull rod of the traction frame of the residual film recovery tool and the horizontal ground and the length L of the upper pull rod of the traction frame of the residual film recovery tool at the current moment, specifically including:

[0083] Based on geometric relationships, the value corresponding to L×sinα reflects the vertical displacement difference of the beam relative to the rotation axis. When the lifting teeth touch the ground at zero position, the inclination angle of the pull rod on the traction frame is set to α0. The vertical displacement difference in this state is recorded as Δh0 = L×sinα0.

[0084] When the residual film recycling machine enters the operating state, the traction frame rotates downward around the connecting shaft, and the residual film recycling machine is kept level under the action of the upper and lower pull rods. Therefore, the depth change value of the film-lifting tooth relative to the zero state position is the film-lifting tooth depth, which is also the height change value of the connection point between the rotating shaft, the connecting rod and the tillage implement.

[0085] From the geometric relationship, we can know that the calculation formula of the depth H of the membrane tooth into the soil is: H = Δh-Δh0 = L×sinα-L×sinα0

[0086] Where: H——depth of film tooth, unit: mm;

[0087] L——upper pull rod length, unit: mm;

[0088] α——the inclination angle between the traction frame and the horizontal ground, unit: °;

[0089] α0——The initial tilt angle of the traction frame to the horizontal ground, unit: °.

[0090] Specifically, in the complex environment of actual operation of residual film recycling equipment, there are many similar non-target scene interferences in the collected image data, and the non-target scenes have similar image features to the target scene. When the original DenseNet121 model is used to extract the target scene image, the non-target scene is often misextracted. DenseNet121 promotes feature reuse through dense connections and can fully capture the details and structure of complex images, but does not consider the correlation between different channels. The SE-Net module adaptively enhances the role of important features by modeling the dependency between channels and the global loss function. Therefore, the channel attention mechanism SE-Net is introduced in DenseNet, and the basic module SE-Block of SE-Net is embedded in the nonlinear combination function in each dense block (Dense Block) to construct the SE-Dense Block before and after, thereby constructing a feature channel weighted SE-DenseNet to improve the model's recognition ability for similar features.

[0091] In order to alleviate the problem of increased computational complexity of the model when the channel attention mechanism is added, while retaining more detailed information. The hole convolution is introduced to replace the first convolution layer of the original model. A hole convolution that is too large cannot extract fine-grained information, and a hole convolution that is too small cannot extract large-scale information. In addition, the exponential increase in the receptive field can more effectively capture information of different scales, forming a more obvious hierarchical feature extraction structure. Therefore, multi-scale serial hole convolutions with different parameter combinations of hole rates of 1, 2, and 4 are introduced to increase the receptive field while maintaining sensitivity to details, so as to further enhance the feature extraction ability of the model. The network model that has been optimized by SE-Net and introduced with multi-scale serial hole convolution is called SE-DenseNet-DC model, so the classification recognition model is obtained by replacing the first convolution module in the DenseNet network model with the first multi-scale serial hole convolution module, and embedding each nonlinear combination function in each dense block in the DenseNet network model into the SE-Block module. The specific structure is as follows: Figure 8 As shown, it has the characteristics of lightweight and high precision.

[0092] Based on the SE-DenseNet-DC model, a densely connected U-shaped segmentation network was constructed to accurately identify residual film in the target scene. The model adopts an encoder-decoder structure, which aims to simultaneously extract multi-scale features and positioning information of the image, thereby improving the segmentation effect. The encoder part is built based on four Dense Blocks embedded with a channel attention mechanism, ensuring efficient extraction of target features and enhancing the perception of image input through multi-scale serial dilated convolution. The decoder part mainly consists of four upsampling layers. Skip connections are used between the encoder and decoder to combine semantic information of different resolutions and positioning information of different resolutions, thereby maximizing the retention of detail information. The final model is called the SE-DenseNet-DC segmentation model, so the semantic segmentation model includes an encoder and a decoder.

[0093] The encoder includes a second multi-scale serial hole convolution module, N maximum pooling downsampling modules and N+1 SE-Dense Block convolution modules.

[0094] The SE-Dense Block convolution module includes a SE-Dense Block module and a first convolution module connected in sequence; the output end of the second multi-scale serial hole convolution module is connected to the input end of the SE-Dense Block module in the first SE-Dense Block convolution module; the output end of the first convolution module in the i-th SE-Dense Block convolution module is connected to the input end of the i-th maximum pooling downsampling module, and the output end of the i-th maximum pooling downsampling module is connected to the input end of the first convolution module in the i+1-th SE-Dense Block convolution module, 1≤i≤N; the SE-Dense Block module is obtained by embedding each nonlinear combination function in any dense block in the DenseNet network model into the SE-Block module front and back.

[0095] The decoder includes N+1 upsampling convolution modules connected in sequence; the upsampling convolution module includes an upsampling module and a convolution layer connected in sequence; and the convolution layer includes a plurality of second convolution modules connected in sequence.

[0096] The first convolution module in the encoder is connected to the upsampling module in the first upsampling convolution module in the decoder.

[0097] The nth SE-Dense Block module in the encoder is jump-connected to the first second convolution module in the N+2-nth upsampling convolution module in the decoder, 1≤n≤N+1, and the overall structure is as follows: Figure 9 As shown, it has the characteristics of high precision, Figure 8 and Figure 9In it, Conv is the convolution module, Dilated Conv is the multi-scale serial hole convolution module, Upsamping is the upsampling module, Pooling is the maximum pooling downsampling module, BN is the batch normalization layer, ReLu is the activation function, SkipConnection is the skip connection, encoder is the encoder part, decoder is the decoder part, SE-Dense Block is the dense block embedded in SE-Block, Linear function is the linear function, and r is the hole rate of hole convolution.

[0098] This application combines deep learning, image processing, and Internet of Things technologies to achieve real-time, efficient, and accurate monitoring and evaluation of the recovery effect, operating area, and operating status of residual film recovery equipment. Because this method has the characteristics of high accuracy, low cost, and can be checked at any time, it is very suitable for real-time supervision of on-site operations of residual film recovery equipment. It is of great significance for guiding subsidies for residual film recovery operations and guiding the efficient control of residual film pollution.

[0099] Based on the same inventive concept, embodiments of the present application also provide a device for real-time monitoring of residual film recovery equipment, which is used to implement the aforementioned method for real-time monitoring of residual film recovery equipment. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for real-time monitoring of residual film recovery equipment provided below can be found in the limitations of the method for real-time monitoring of residual film recovery equipment described above and will not be repeated here.

[0100] In an exemplary embodiment, a real-time operation monitoring device for a residual film recovery machine is provided, which is used to implement the above-mentioned real-time operation monitoring method for the residual film recovery machine. The real-time operation monitoring device for the residual film recovery machine includes:

[0101] A cloud server and a camera 2, a positioning device 3, a tilt sensor 5 and a Hall sensor 6 respectively connected to the cloud server.

[0102] At any moment during the operation of the residual film recovery machine, the camera 2 is used to obtain the scene image of the residual film recovery machine at the current moment, the positioning device 3 is used to obtain the position coordinates of the residual film recovery machine at the current moment, and the inclination sensor 5 is used to measure the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment; the Hall sensor 6 is used to measure the pulse signal of the main shaft speed of the residual film recovery machine at the current moment.

[0103] The cloud server is used to input the scene image of the residual film recycling machine at the current moment into a classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into a semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; determine that the ratio of the total number of pixels of residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is the picking rate of the residual film recycling machine at the current moment; the category is residual film or not residual film.

[0104] The cloud server is also used to use a triangulation algorithm to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the operating area of ​​the residual film recovery machine; the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment.

[0105] The cloud server is also used to calculate the depth of the film teeth 7 at the current moment based on the angle between the upper pull rod of the traction frame on the residual film recovery machine and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery machine.

[0106] The cloud server is also used to calculate the spindle speed of the residual film recovery machine at the current moment based on the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor 6.

[0107] This application uses residual film recovery equipment as a carrier to develop a corresponding real-time operation monitoring device for residual film recovery equipment, which is an effective way to achieve real-time monitoring of the picking rate, operating area and operating status of residual film recovery equipment.

[0108] In an exemplary embodiment, the real-time operation monitoring device for the residual film recovery machine further includes:

[0109] The intelligent display and control terminal 1 and the remote communication module, the intelligent display and control terminal 1 is connected to the remote communication module, the camera 2, the positioning device 3, the tilt sensor 5 and the Hall sensor 6 respectively.

[0110] The intelligent display and control terminal 1 is used to send the scene image of the residual film recovery machine at the current moment obtained by the camera 2, the position coordinates of the residual film recovery machine at the current moment obtained by the positioning device 3, the angle between the pull rod on the traction frame of the residual film recovery machine at the current moment and the horizontal ground measured by the inclination sensor 5, and the pulse signal of the main shaft speed of the residual film recovery machine at the current moment measured by the Hall sensor 6 to the cloud server through the remote communication module.

[0111] The cloud server is further configured to send the pickup rate of the residue film recycling machine at the current time, the working area of the residue film recycling machine at the current time, the depth of the film tooth 7 at the current time, and the main shaft rotating speed of the residue film recycling machine at the current time to the intelligent display and control terminal 1 through the remote communication module for display.

[0112] In an exemplary embodiment, the residue film recycling machine real-time working supervision device further comprises:

[0113] The alarm lamp 4 is connected with the intelligent display and control terminal 1, and the intelligent display and control terminal 1 controls the alarm lamp 4 to alarm according to the depth of the film tooth 7 at the current time and the main shaft rotating speed of the residue film recycling machine at the current time.

[0114] In an exemplary embodiment, as shown in Figure 2 The residue film recycling machine is a spring-tooth type residue film recycling machine pulled by a tractor, wherein the camera 2 is a distortion-free camera mounted on the right windshield of the tractor cab and fixed by a suction cup type support to obtain a complete image of the ground 5m×3m worked by the residue film recycling machine.

[0115] The positioning device 3 is a positioning antenna mounted above the tractor cab to obtain the position coordinates and working track of the residue film recycling machine, and further predict the working area.

[0116] The inclination sensor 5 is mounted on the upper pull rod of the residue film recycling machine traction frame to obtain the inclination angle formed by the upper pull rod and the horizontal ground.

[0117] The Hall sensor 6 is mounted on the main shaft inside the residue film recycling machine responsible for transmitting power to obtain the pulse signal of the main shaft rotating speed.

[0118] The intelligent display and control terminal 1 is installed in the tractor cab.

[0119] The remote communication module is built-in in the intelligent display and control terminal 1.

[0120] The warning lamp 4 is installed in the tractor cab to regulate the driver's illegal behavior, prevent the machine failure and poor recycling quality caused by the excessive main shaft rotating speed or the excessive depth of the film tooth 7, and the warning lamp gives a warning when the main shaft rotating speed and the depth of the film tooth 7 reach the alarm value.

[0121] The camera 2, the positioning device 3, the inclination sensor 5, the Hall sensor 6, and the warning lamp 4 are connected with the intelligent display and control terminal 1 through a circuit, the engine is started to power on the intelligent display and control terminal 1, and the system automatically runs. The image after the residue film recycling machine is collected, the latitude and longitude information, and the main shaft rotating speed are quickly obtained, the information is fused, and the real-time uploading is realized to the cloud server.

[0122] In an exemplary embodiment, a smart display and control terminal 1 fixing bracket and a smart display and control terminal 1 fixing plate are provided on the tractor cab frame, and the smart display and control terminal 1 is connected to the smart display and control terminal 1 fixing bracket, and the smart display and control terminal 1 fixing bracket is connected to the smart display and control terminal 1 fixing plate, and the smart display and control terminal 1 fixing plate is a "C"-shaped fixing plate with a bend, which is fixedly connected to the frame in the tractor cab.

[0123] like Figure 3 As shown, the working process of calculating the pickup rate in the real-time operation monitoring device for the residual film recycling machine provided in the embodiment of the present application is as follows:

[0124] (1) After the residual film recycling machine is powered on, it receives the 5m×3m plot image of the ground surface after work taken by camera 2 (the camera is set to take one picture every 6 seconds through a Python program). The residual film recycling machine is required to maintain a steady speed while driving and collect the surface images after collection in sequence.

[0125] (2) The collected surface images are transmitted to the cloud server in real time through the remote communication module, and the data set is divided into a training set and a test set to train the classification model to obtain a classification recognition model.

[0126] (3) The cloud server uses a classification and recognition model to screen effective images that can be used to evaluate the pickup rate under complex operating conditions of residual film recycling equipment. If the classification and recognition model outputs a result of 1, it is valid.

[0127] (4) The cloud server uses the embedded semantic segmentation model to perform residual film recognition and segmentation on the filtered valid images, obtaining the category of each pixel on the image. Python is then used to calculate the total proportion of residual film pixels in each image relative to the total number of pixels in the entire image, obtaining the corresponding pickup rate for each image. Finally, the cloud server calculates the average value based on the pickup rate of each image to obtain the final pickup rate used for evaluation.

[0128] (5) The cloud server transmits the real-time updated collection rate of the residual film recycling equipment to the intelligent display and control terminal 1 for display.

[0129] The working process of calculating the operating area in the real-time operation monitoring device for the residual film recovery machine provided in the embodiment of the present application is as follows:

[0130] (1) After the residual film recovery machine is powered on and started, the positioning device 3 receives the position coordinate information in real time.

[0131] (2) The collected coordinate information is transmitted to the cloud server in real time through the remote communication module.

[0132] (3) The cloud server uses Gaussian projection to convert the coordinate information into plane coordinate system coordinates.

[0133] (4) The cloud server uses a triangulation algorithm to divide the initial job vertex data (the coordinate points obtained in the first three moments) into triangles.

[0134] (5) During the operation, the coordinate information updated in real time is inserted, and the original triangle is updated using the triangulation algorithm. The vertices of the original triangle are connected to generate new triangles. The area of ​​each triangle is calculated, and these areas are accumulated to obtain a rough operation area.

[0135] (6) Use the grid method to mark whether each triangle is covered by one or more operation areas and obtain the repeated operation area.

[0136] (7) Finally, the rough operation area is subtracted from the repeated operation area to obtain the operation area.

[0137] (8) The cloud server transmits the real-time updated operation area to the intelligent display and control terminal 1 for display.

[0138] The working process of status monitoring in the real-time operation monitoring device of the residual film recovery machine provided in the embodiment of the present application is as follows:

[0139] (1) After the residual film recovery machine is powered on, the Hall sensor 6 is used to obtain the pulse signal of the spindle speed. Then the cloud server starts the timer to count the pulses and calculates the spindle speed using the period measurement method to ensure that the measured speed value is accurate enough. The built-in program of the intelligent display and control terminal 1 encodes the data according to the communication protocol, displays the spindle speed and triggers an alarm signal when it exceeds the preset alarm threshold. The specific steps are as follows: Figure 5 shown.

[0140] (2) Use the inclination sensor 5 installed on the pull rod of the traction frame to obtain the inclination angle formed by the traction frame and the horizontal base surface ( Figure 6 This is the data acquisition flow chart of the inclination sensor 5. When the system starts, it issues a data acquisition instruction, triggers the buffer area of ​​the serial communication interface and prepares to receive data. After successfully receiving the data receipt returned by the sensor, the data integrity check is immediately performed. After verification, the 3-byte data content of the receipt is decoded and the data value is converted to floating point type for subsequent processing and calculation). The cloud server combines the radial length of the upper pull rod to obtain the zero-state vertical displacement difference when not working, and calculates the depth change value of the film tooth 7 relative to the zero-state position after the residual film recovery machine is working, which is the depth of the film tooth 7. The depth of the film tooth 7 is displayed and an alarm signal is triggered when it exceeds the preset alarm threshold.

[0141] The real-time operation monitoring device provided by this application is installed on the residual film recycling machine, and the cloud server workflow is as follows Figure 7As shown in the figure, after the system is running, the cloud server receives the positioning coordinate data, image data, and operation sensor data transmitted by the intelligent display and control terminal in real time. After calculation and analysis, the cloud server transmits the operation area, residual film recovery rate, spindle speed and other data information through the remote communication module to the residual film recovery information monitoring cloud platform, and finally issues the residual film recovery machine operation quality assessment report through the cloud platform.

[0142] The real-time operation monitoring device provided by this application, which is installed on the residual film recovery machine, is easy to install, economical and practical. It has a reasonable structure, efficient and accurate operation, and a high degree of automation. It can provide a hardware foundation for real-time monitoring and evaluation of the pickup rate, operating area and operating status of the residual film recovery machine, and supervision of the violations of operators. The monitoring of the pickup rate of the residual film recovery machine is achieved through the combination of on-board visual imaging and deep learning methods; the monitoring of the operating area is achieved through the positioning antenna receiving position coordinate information combined with Gaussian projection and triangulation algorithms; the monitoring of the spindle speed and film-lifting tooth depth of the residual film recovery machine is achieved by receiving sensor data and analyzing and calculating.

[0143] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store real-time operation supervision data of residual film recovery equipment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a real-time operation supervision method for residual film recovery equipment is implemented.

[0144] Those skilled in the art will understand that Figure 10 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0145] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above-mentioned method embodiments when executed by a processor.

[0146] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above method embodiments are implemented.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0148] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0149] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0150] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0151] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A real-time operation supervision method for a residual film recovery machine, characterized in that: The real-time operation supervision method of the residual film recovery machine includes: At any moment during the operation of the residual film recovery machine, obtain the scene image of the residual film recovery machine at the current moment, the position coordinates of the residual film recovery machine, the main shaft speed of the residual film recovery machine, and the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground; Inputting the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; If the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; the category is residual film or not residual film; Determine the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment as the pickup rate of the residual film recycling machine at the current moment; Using the triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recycling machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained based on the vertices of the residual film recycling machine's operating area; The operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment; The film tooth depth at the current moment is calculated according to the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool at the current moment.

2. The real-time operation supervision method of residual film recycling equipment according to claim 1 is characterized in that: The classification recognition model is obtained by replacing the first convolution module in the DenseNet network model with the first multi-scale serial hole convolution module, and embedding each nonlinear combination function in each dense block in the DenseNet network model into the SE-Block module before and after.

3. The real-time operation supervision method of residual film recycling equipment according to claim 1 is characterized in that: The semantic segmentation model includes an encoder and a decoder; The encoder includes a second multi-scale cascaded hole convolution module, N maximum pooling downsampling modules and N+1 SE-Dense Block convolution modules; The SE-Dense Block convolution module includes a SE-Dense Block module and a first convolution module connected in sequence; the output end of the second multi-scale serial hole convolution module is connected to the input end of the SE-DenseBlock module in the first SE-Dense Block convolution module; the output end of the first convolution module in the i-th SE-Dense Block convolution module is connected to the input end of the i-th maximum pooling downsampling module, and the output end of the i-th maximum pooling downsampling module is connected to the input end of the first convolution module in the i+1-th SE-Dense Block convolution module, 1≤i≤N; the SE-Dense Block module is obtained by embedding each nonlinear combination function in any dense block in the DenseNet network model into the SE-Block module front and back; The decoder includes N+1 sequentially connected upsampling convolution modules; the upsampling convolution module includes sequentially connected upsampling modules and convolution layers; the convolution layer includes multiple sequentially connected second convolution modules; The first convolution module in the encoder is connected to the upsampling module in the first upsampling convolution module in the decoder; The nth SE-Dense Block module in the encoder is jump-connected to the first second convolution module in the N+2-nth upsampling convolution module in the decoder, 1≤n≤N+1.

4. A device for real-time operation supervision of a residual film recovery machine, used to implement the real-time operation supervision method of a residual film recovery machine according to any one of claims 1 to 3, characterized in that: The real-time operation monitoring device of the residual film recovery machine includes: A cloud server and a camera, a positioning device, a tilt sensor, and a Hall sensor respectively connected to the cloud server; At any moment during the operation of the residual film recovery machine, the camera is used to obtain a scene image of the residual film recovery machine at the current moment, the positioning device is used to obtain the position coordinates of the residual film recovery machine at the current moment, the inclination sensor is used to measure the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment; the Hall sensor is used to measure the pulse signal of the main shaft speed of the residual film recovery machine at the current moment; The cloud server is used to input the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; determine that the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is the picking rate of the residual film recycling machine at the current moment; the category is residual film or not residual film; The cloud server is further configured to use a triangulation algorithm to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area; and the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment; The cloud server is also used to calculate the film tooth depth at the current moment based on the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool; The cloud server is also used to calculate the spindle speed of the residual film recovery machine at the current moment based on the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor.

5. The real-time operation monitoring device for residual film recycling equipment according to claim 4 is characterized in that: The residual film recovery machine real-time operation monitoring device also includes: An intelligent display and control terminal and a remote communication module, wherein the intelligent display and control terminal is connected to the remote communication module, the camera, the positioning device, the tilt sensor, and the Hall sensor respectively; The intelligent display and control terminal is used to send a pulse signal of the residual film recovery machine's current scene image acquired by the camera, the position coordinates of the residual film recovery machine acquired by the positioning device, the angle between the upper pull rod of the traction frame of the residual film recovery machine and the horizontal ground measured by the inclination sensor, and the main shaft speed of the residual film recovery machine measured by the Hall sensor to the cloud server through the remote communication module; The cloud server is also used to send the current picking rate of the residual film recovery machine, the current operating area of ​​the residual film recovery machine, the current film tooth depth and the current spindle speed of the residual film recovery machine to the intelligent display and control terminal through the remote communication module for display.

6. The real-time operation monitoring device for residual film recycling equipment according to claim 5 is characterized in that: The residual film recovery machine real-time operation monitoring device also includes: An alarm light connected to the intelligent display and control terminal controls the alarm light to sound an alarm based on the current depth of the film teeth and the current spindle speed of the residual film recovery machine.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the real-time operation supervision method for the residual film recovery equipment according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the real-time operation supervision method of the residual film recovery equipment described in any one of claims 1 to 3 is implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the real-time operation supervision method of the residual film recovery equipment described in any one of claims 1 to 3 is implemented.

Images