Residual film recovery machine real-time operation supervision method, device, equipment, medium and product

Abstract

The invention discloses a method, a device, equipment, a medium and a product for supervising real-time operation of a residual film recovery machine tool, and relates to the field of intelligent agricultural equipment.The method comprises the steps that scene images, position coordinates, the angle between an upper pull rod and the horizontal ground and the rotating speed of a main shaft at all moments in the working process of the residual film recovery machine tool are acquired; inputting the scene image at each moment into a classification identification model to judge whether the image is valid; inputting the effective image into a semantic segmentation model, and calculating the pickup rate at each moment based on the ratio of the total number of pixel points of a residual film in the image to the total number of pixels of the image; a triangle subdivision algorithm is adopted, and triangles are updated according to the position coordinates at all moments; subtracting the overlapping area from the sum of the areas of the triangles at each moment to obtain the operation area at each moment; and calculating the film lifting tooth depth at each moment according to the angle at each moment and the length of the upper pull rod. According to the application, the picking rate, the operation area and the operation state (the rotating speed of the main shaft and the film lifting tooth depth) of the residual film recovery machine can be monitored in real time.

Description

Technical Field

[0001] The present application relates to the field of smart agricultural equipment, and in particular to a method, device, equipment, medium and product for real-time operation supervision of residual film recovery equipment. Background Art

[0002] Agricultural mulch film is mainly made of polyethylene, but current residual film recycling equipment often malfunctions during operation, has low operational reliability and a low residual film pickup rate. The lack of effective residual film recycling supervision technology has resulted in insignificant effects on the control of farmland residual film pollution, which has caused serious soil non-point source pollution.

[0003] Currently, there is no method that can monitor the pick-up rate, operating area and operating status (spindle speed and film-removing tooth depth) of residual film recovery equipment in real time. Summary of the invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for real-time operation supervision of a residual film recovery machine, which can monitor the picking rate, operating area and operating status (spindle speed and film tooth depth) of the residual film recovery machine in real time.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for real-time operation supervision of a residual film recovery machine, comprising:

[0007] At any moment during the operation of the residual film recovery machine, the scene image of the residual film recovery machine at the current moment, the position coordinates of the residual film recovery machine, the spindle speed of the residual film recovery machine, and the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground are obtained;

[0008] Inputting the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid;

[0009] If the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; the category is residual film or not residual film;

[0010] Determine the ratio of the total number of pixel points of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment as the picking rate of the residual film recycling machine at the current moment;

[0011] Using the triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area;

[0012] The operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment;

[0013] The film tooth depth at the current moment is calculated according to the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool at the current moment.

[0014] Optionally, the classification recognition model is obtained by replacing the first convolution module in the DenseNet network model with the first multi-scale serial hole convolution module, and embedding each nonlinear combination function in each dense block in the DenseNet network model into the SE-Block module before and after.

[0015] Optionally, the semantic segmentation model includes an encoder and a decoder;

[0016] The encoder includes a second multi-scale cascaded hole convolution module, N maximum pooling downsampling modules and N+1 SE-Dense Block convolution modules;

[0017] The SE-Dense Block convolution module includes a SE-Dense Block module and a first convolution module connected in sequence; the output end of the second multi-scale serial hole convolution module is connected to the input end of the SE-Dense Block module in the first SE-Dense Block convolution module; the output end of the first convolution module in the i-th SE-Dense Block convolution module is connected to the input end of the i-th maximum pooling downsampling module, and the output end of the i-th maximum pooling downsampling module is connected to the input end of the first convolution module in the i+1-th SE-Dense Block convolution module, 1≤i≤N; the SE-Dense Block module is obtained by embedding each nonlinear combination function in any dense block in the DenseNet network model into the SE-Block module front and back;

[0018] The decoder includes N+1 upsampling convolution modules connected in sequence; the upsampling convolution module includes upsampling modules and convolution layers connected in sequence; the convolution layer includes a plurality of second convolution modules connected in sequence;

[0019] The first convolution module in the encoder is connected to the upsampling module in the first upsampling convolution module in the decoder;

[0020] The nth SE-Dense Block module in the encoder is jump-connected to the first second convolution module in the N+2-nth upsampling convolution module in the decoder, 1≤n≤N+1.

[0021] In a second aspect, the present application provides a real-time operation monitoring device for a residual film recovery machine, which is used to implement the real-time operation monitoring method for the residual film recovery machine described above. The real-time operation monitoring device for the residual film recovery machine comprises:

[0022] A cloud server and a camera, a positioning device, a tilt sensor, and a Hall sensor respectively connected to the cloud server;

[0023] At any moment during the operation of the residual film recovery machine, the camera is used to obtain the scene image of the residual film recovery machine at the current moment, the positioning device is used to obtain the position coordinates of the residual film recovery machine at the current moment, and the inclination sensor is used to measure the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment; the Hall sensor is used to measure the pulse signal of the main shaft speed of the residual film recovery machine at the current moment;

[0024] The cloud server is used to input the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; determine that the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is the picking rate of the residual film recycling machine at the current moment; the category is residual film or not residual film;

[0025] The cloud server is also used to use a triangulation algorithm to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area; the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment;

[0026] The cloud server is also used to calculate the film tooth depth at the current moment according to the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool at the current moment;

[0027] The cloud server is also used to calculate the spindle speed of the residual film recovery machine at the current moment based on the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor.

[0028] Optionally, the real-time operation monitoring device for the residual film recovery machine further includes:

[0029] An intelligent display and control terminal and a remote communication module, wherein the intelligent display and control terminal is respectively connected to the remote communication module, the camera, the positioning device, the tilt sensor and the Hall sensor;

[0030] The intelligent display and control terminal is used to send a pulse signal of the current residual film recovery machine scene image acquired by the camera, the current position coordinates of the residual film recovery machine acquired by the positioning device, the current angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground measured by the inclination sensor, and the current spindle speed of the residual film recovery machine measured by the Hall sensor to the cloud server through the remote communication module;

[0031] The cloud server is also used to send the current picking rate of the residual film recovery machine, the current operating area of ​​the residual film recovery machine, the current film tooth depth and the current spindle speed of the residual film recovery machine to the intelligent display and control terminal through the remote communication module for display.

[0032] Optionally, the real-time operation monitoring device for the residual film recovery machine further includes:

[0033] An alarm light connected to the intelligent display and control terminal controls the alarm light to sound an alarm according to the film tooth depth at the current moment and the spindle speed of the residual film recovery machine at the current moment.

[0034] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned methods for real-time operation supervision of residual film recovery equipment.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for real-time operation supervision of residual film recovery equipment.

[0036] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for real-time operation supervision of residual film recovery equipment.

[0037] According to the specific embodiments provided in this application, this application has the following technical effects:

[0038] The present application provides a method, device, equipment, medium and product for real-time operation supervision of a residual film recovery machine. By inputting the scene image of the residual film recovery machine at the current moment into a classification recognition model, it is determined whether the scene image of the residual film recovery machine at the current moment is valid; if the scene image of the residual film recovery machine at the current moment is valid, the scene image of the residual film recovery machine at the current moment is input into a semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recovery machine at the current moment; the ratio of the total number of pixels of the residual film in the scene image of the residual film recovery machine at the current moment to the total number of pixels of the scene image of the residual film recovery machine at the current moment is determined as the picking rate of the residual film recovery machine at the current moment; the picking rate of the residual film recovery machine is monitored in real time. , using a triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment, and the operating area of ​​the residual film recovery machine is monitored in real time; the film tooth depth at the current moment is calculated according to the angle between the pull rod on the traction frame of the residual film recovery machine at the current moment and the horizontal ground and the length of the pull rod on the traction frame of the residual film recovery machine, and the film tooth depth of the residual film recovery machine is monitored in real time, and the spindle speed of the residual film recovery machine can be directly obtained, and the spindle speed of the residual film recovery machine can be monitored in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flow chart of a real-time operation supervision method for a residual film recovery machine provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the structure of a real-time operation monitoring device for a residual film recovery machine provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the workflow for calculating the pick-up rate in the real-time operation monitoring device for the residual film recovery machine provided in the embodiment of the present application;

[0043] Figure 4 A schematic diagram of the workflow for calculating the operating area in the real-time operation monitoring device for the residual film recovery machine provided in the embodiment of the present application;

[0044] Figure 5A schematic diagram of the working process for monitoring the spindle speed in the real-time operation monitoring device for the residual film recovery machine provided in the embodiment of the present application;

[0045] Figure 6 This is the data collection flow chart of the tilt sensor;

[0046] Figure 7 A schematic diagram of the cloud server workflow provided in the embodiment of the present application;

[0047] Figure 8 A structural diagram of the SE-DenseNet-DC model provided in an embodiment of the present application;

[0048] Fig. 9 A structural diagram of the SE-DenseNet-DC segmentation model provided in an embodiment of the present application;

[0049] Fig.10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0050] Reference numerals:

[0051] 1—intelligent display and control terminal, 2—camera, 3—positioning device, 4—warning light, 5—tilt sensor, 6—Hall sensor, 7—film-lifting tooth. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0054] In an exemplary embodiment, a method for real-time operation supervision of a residual film recovery machine is provided, such as Figure 1 As shown, the following steps are included, wherein:

[0055] Step 201: At any time during the operation of the residual film recovery machine, obtain the scene image of the residual film recovery machine at the current time, the position coordinates of the residual film recovery machine, the spindle speed of the residual film recovery machine, and the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground.

[0056] Step 202: Input the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid. If the scene image of the residual film recycling machine at the current moment is a valid image after collection, that is, an image processed by the residual film recycling machine, then the scene image of the residual film recycling machine at the current moment is valid.

[0057] Step 203: If the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; the category is residual film or not residual film.

[0058] Step 204: Determine the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment as the picking rate of the residual film recycling machine at the current moment.

[0059] Step 205: Using a triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area. Specifically, the triangle is obtained by processing the vertices of the operating area using a triangulation algorithm.

[0060] Step 206: The operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment.

[0061] Step 207: Calculate the film tooth depth at the current moment according to the angle between the upper pull rod of the traction frame on the residual film recovery machine and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery machine.

[0062] By implementing the above-mentioned steps 201 to 207, the present application can monitor the pick-up rate, operating area and operating status (spindle speed and film-removing tooth depth) of the residual film recovery machine in real time.

[0063] In an exemplary embodiment, a triangulation algorithm is used to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment, specifically including:

[0064] The Gaussian projection algorithm is used to convert the position coordinates of the residual film recovery machine at the current moment into the coordinates of the plane coordinate system.

[0065] Using the triangulation algorithm, the triangle at the previous moment is updated according to the coordinates of the plane coordinate system to obtain the triangle at the current moment.

[0066] Specifically: After the coordinates of the trajectory points are converted to the plane coordinate system, the trajectory points on these planes can be used to calculate the operating area. The coordinates of these trajectory points are defined as a finite point set P, which is constructed into a larger initial triangle to ensure that all point sets are included.

[0067] Add each point in the point set to the triangle in turn. Each time a new point is inserted, a line is built with the vertices of the original triangle, thereby generating three new triangles. Check whether there are other points on the circumscribed circles corresponding to these newly formed triangles except for the vertices that constitute the triangle.

[0068] When two adjacent triangles form a quadrilateral, check whether the fourth vertex is within the circumscribed circle. If so, it means that the current segmentation does not meet the conditions, so the diagonals of the quadrilateral need to be swapped. If the fourth vertex is not within any circumscribed circle, keep the current segmentation unchanged. Finally, a triangulated network that meets the conditions is constructed. The specific steps are as follows: Figure 4 shown.

[0069] In an exemplary embodiment, the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment, which specifically includes:

[0070] Calculate the sum of the areas of all triangles at the current moment to get the rough working area.

[0071] The sum of the overlapping areas between the triangles at the current moment is calculated to obtain the repeated operation area.

[0072] The precise operating area is obtained by subtracting the repeated operating area from the rough operating area, that is, the operating area of ​​the residual film recovery machine at the current moment.

[0073] Specifically, the boundary of the work plot is set to be a closed polygon composed of n vertices, and the coordinates of each vertex are represented as A1(x1, y1), A2(x2, y2), ..., ..., A i (x i ,y i )(where i = 1, 2, 3, ..., n). Therefore, the calculation formula for calculating the sum of the areas of the triangles at the current moment to obtain the rough working area S is as follows:

[0074]

[0075]

[0076]

[0077] In the above formula, x n+1 =x1,y n+1 =y1.

[0078] Calculate the sum of the overlapping areas between the triangles at the current moment to get the repeated operation area. The specific steps are:

[0079] Define Grid: Divide the entire work area into grid units of equal size. For example, set the grid size to 1×1×1 unit.

[0080] Mark grid cells: Traverse each triangle and mark the grid cells covered by the triangle. If multiple operation paths cover the same grid cell, it is marked as a repeated operation area.

[0081] The sum of the areas of all repeated operation areas is calculated to obtain the repeated operation area.

[0082] In an exemplary embodiment, the film tooth depth at the current moment is calculated according to the angle α between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length L of the upper pull rod of the traction frame on the residual film recovery tool at the current moment, specifically including:

[0083] According to the geometric relationship, the value corresponding to L×sinα reflects the vertical displacement difference of the beam compared to the rotation axis. When the film teeth touch the ground and are in the zero position, the inclination angle of the pull rod on the traction frame is set to α0, and the vertical displacement difference in this state is recorded as Δh0=L×sinα0.

[0084] When the residual film recycling machine enters the operating state, the traction frame rotates downward around the connecting shaft, and the residual film recycling machine is kept level under the action of the upper and lower pull rods. Therefore, the depth change value of the film-lifting tooth relative to the zero state position is the depth of the film-lifting tooth, that is, the height change value of the connection point between the rotating shaft, the connecting rod and the tillage implement.

[0085] From the geometric relationship, we can know that the formula for calculating the depth H of the membrane tooth into the soil is: H = Δh-Δh0 = L×sinα-L×sinα0

[0086] Where: H——depth of film teeth, unit: mm;

[0087] L——length of upper pull rod, in mm;

[0088] α——the inclination angle between the traction frame and the horizontal ground, unit: °;

[0089] α0——The initial inclination angle of the traction frame to the horizontal ground, unit: °.

[0090] Specifically, in the complex environment of the actual operation of the residual film recycling machine, there are many similar non-target scene interferences in the collected image data, and the non-target scenes have similar image features to the target scene. When the original DenseNet121 model is used to extract the target scene image, the non-target scene is often mis-extracted. DenseNet121 promotes feature reuse through dense connections, and can fully capture the details and structure of complex images, but does not consider the correlation between different channels. The SE-Net module adaptively enhances the role of important features by modeling the dependencies between channels and the global loss function. Therefore, the channel attention mechanism SE-Net is introduced in DenseNet, and the SE-Dense Block is constructed before and after the SE-Block, the basic module of SE-Net, is embedded in each dense block (Dense Block), so as to construct a feature channel weighted SE-DenseNet to improve the model's recognition ability for similar features.

[0091] In order to alleviate the problem of increased computational complexity of the model by adding the channel attention mechanism, while retaining more detailed information. The hole convolution is introduced to replace the first convolution layer of the original model. A hole convolution that is too large cannot extract fine-grained information, and a hole convolution that is too small cannot extract large-scale information. In addition, the exponential increase in the receptive field can more effectively capture information of different scales, forming a more obvious hierarchical feature extraction structure. Therefore, multi-scale serial hole convolutions with different parameter combinations of hole rates of 1, 2, and 4 are introduced to increase the receptive field while maintaining sensitivity to details, so as to further enhance the feature extraction ability of the model. The network model that is optimized by SE-Net and introduces multi-scale serial hole convolutions is called SE-DenseNet-DC model. Therefore, the classification recognition model is obtained by replacing the first convolution module in the DenseNet network model with the first multi-scale serial hole convolution module, and embedding each nonlinear combination function in each dense block in the DenseNet network model into the SE-Block module. The specific structure is as follows Figure 8 As shown, it has the characteristics of lightweight and high precision.

[0092] Based on the SE-DenseNet-DC model, a densely connected U-shaped segmentation network is constructed to accurately identify residual film in the target scene. The model adopts an encoder-decoder structure, which aims to simultaneously extract multi-scale features and positioning information of the image, thereby improving the segmentation effect. The encoder part is built based on four Dense Blocks embedded with a channel attention mechanism, which ensures the efficient extraction of target features and enhances the perception of image input through multi-scale serial hole convolution. The decoder part is mainly composed of 4 upsampling layers. Skip connections are used between the encoder and decoder to combine semantic information of different resolutions and positioning information of different resolutions, thereby maximizing the retention of detail information. The final model is called the SE-DenseNet-DC segmentation model, so the semantic segmentation model includes an encoder and a decoder.

[0093] The encoder includes a second multi-scale serial hole convolution module, N maximum pooling downsampling modules and N+1 SE-Dense Block convolution modules.

[0094] The SE-Dense Block convolution module includes a SE-Dense Block module and a first convolution module connected in sequence; the output end of the second multi-scale serial hole convolution module is connected to the input end of the SE-Dense Block module in the first SE-Dense Block convolution module; the output end of the first convolution module in the i-th SE-Dense Block convolution module is connected to the input end of the i-th maximum pooling downsampling module, and the output end of the i-th maximum pooling downsampling module is connected to the input end of the first convolution module in the i+1th SE-Dense Block convolution module, 1≤i≤N; the SE-Dense Block module is obtained by embedding each nonlinear combination function in any dense block in the DenseNet network model into the SE-Block module front and back.

[0095] The decoder includes N+1 upsampling convolution modules connected in sequence; the upsampling convolution module includes an upsampling module and a convolution layer connected in sequence; the convolution layer includes a plurality of second convolution modules connected in sequence.

[0096] The first convolution module in the encoder is connected to the upsampling module in the first upsampling convolution module in the decoder.

[0097] The nth SE-Dense Block module in the encoder is jump-connected to the first second convolution module in the N+2-nth upsampling convolution module in the decoder, 1≤n≤N+1, and the overall structure is as follows: Fig. 9 As shown, it has the characteristics of high precision. Figure 8 and Fig. 9In which Conv is a convolution module, Dilated Conv is a multi-scale serial hole convolution module, Upsamping is an upsampling module, Pooling is a maximum pooling downsampling module, BN is a batch normalization layer, ReLu is an activation function, SkipConnection is a skip connection, encoder is the encoder part, decoder is the decoder part, SE-Dense Block is a dense block embedded in SE-Block, Linear function is a linear function, and r is the hole rate of the hole convolution.

[0098] This application combines deep learning, image processing, and Internet of Things technology to achieve real-time, efficient, and accurate monitoring and evaluation of the recovery effect, operating area, and operating status of residual film recovery equipment. Since this method has the characteristics of high accuracy, low cost, and can be checked at any time, it is very suitable for real-time supervision of on-site operations of residual film recovery equipment. It is of great significance for guiding subsidies for residual film recovery operations and guiding the efficient control of residual film pollution.

[0099] Based on the same inventive concept, the embodiment of the present application also provides a residual film recovery machine real-time operation supervision device for implementing the residual film recovery machine real-time operation supervision method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the residual film recovery machine real-time operation supervision device provided below can refer to the limitations of the residual film recovery machine real-time operation supervision method above, and will not be repeated here.

[0100] In an exemplary embodiment, a real-time operation monitoring device for a residual film recovery machine is provided, which is used to implement the real-time operation monitoring method for the residual film recovery machine described above. The real-time operation monitoring device for the residual film recovery machine comprises:

[0101] A cloud server and a camera 2, a positioning device 3, a tilt sensor 5 and a Hall sensor 6 respectively connected to the cloud server.

[0102] At any moment during the operation of the residual film recovery machine, the camera 2 is used to obtain the scene image of the residual film recovery machine at the current moment, the positioning device 3 is used to obtain the position coordinates of the residual film recovery machine at the current moment, and the inclination sensor 5 is used to measure the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment; the Hall sensor 6 is used to measure the pulse signal of the spindle speed of the residual film recovery machine at the current moment.

[0103] The cloud server is used to input the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; determine that the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is determined as the picking rate of the residual film recycling machine at the current moment; the category is residual film or not residual film.

[0104] The cloud server is also used to use a triangulation algorithm to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the operating area of ​​the residual film recovery machine; the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment.

[0105] The cloud server is also used to calculate the depth of the film teeth 7 at the current moment according to the angle between the upper pull rod of the traction frame on the residual film recovery machine and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery machine.

[0106] The cloud server is also used to calculate the spindle speed of the residual film recovery machine at the current moment according to the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor 6.

[0107] This application uses residual film recovery equipment as a carrier to develop a corresponding real-time operation monitoring device for residual film recovery equipment, which is an effective way to achieve real-time monitoring of the picking rate, operating area and operating status of residual film recovery equipment.

[0108] In an exemplary embodiment, the real-time operation monitoring device for the residual film recovery machine further includes:

[0109] The intelligent display and control terminal 1 and the remote communication module, wherein the intelligent display and control terminal 1 is connected to the remote communication module, the camera 2, the positioning device 3, the tilt sensor 5 and the Hall sensor 6 respectively.

[0110] The intelligent display and control terminal 1 is used to send the scene image of the residual film recovery machine at the current moment obtained by the camera 2, the position coordinates of the residual film recovery machine at the current moment obtained by the positioning device 3, the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment measured by the inclination sensor 5, and the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor 6 to the cloud server through the remote communication module.

[0111] The cloud server is also used to send the current picking rate of the residual film recovery machine, the current working area of ​​the residual film recovery machine, the current depth of the film tooth 7 and the current spindle speed of the residual film recovery machine to the intelligent display and control terminal 1 through the remote communication module for display.

[0112] In an exemplary embodiment, the real-time operation monitoring device for the residual film recovery machine further includes:

[0113] The alarm light 4 connected to the intelligent display and control terminal 1 controls the alarm light 4 to sound an alarm according to the depth of the film teeth 7 at the current moment and the spindle speed of the residual film recovery machine at the current moment.

[0114] In an exemplary embodiment, Figure 2 As shown, the residual film recovery tool is a spring-tooth type residual film recovery tool towed by a tractor, wherein the camera 2 is a distortion-free camera: it is installed on the right windshield of the tractor cab and fixed by a suction cup bracket to obtain a complete image of the ground surface of 5m×3m after the residual film recovery tool is working.

[0115] The positioning device 3 is a positioning antenna installed above the tractor cab to obtain the position coordinates and operation trajectory of the residual film recovery equipment, and then predict the operation area.

[0116] Inclination sensor 5: installed on the upper pull rod of the residual film recovery machine traction frame to obtain the inclination angle formed by the upper pull rod and the horizontal ground.

[0117] Hall sensor 6: installed on the main shaft responsible for transmitting power inside the residual film recovery machine to obtain the pulse signal of the main shaft speed.

[0118] Intelligent display and control terminal 1: installed in the tractor cab.

[0119] Remote communication module: built in the intelligent display and control terminal 1.

[0120] Warning light 4: It is installed in the tractor cab to regulate the driver's illegal behavior and prevent machine failure and poor recovery quality caused by excessive spindle speed or excessive depth of the film tooth 7. When the spindle speed and the depth of the film tooth 7 reach the alarm value, the warning light will warn.

[0121] The camera 2, positioning device 3, tilt sensor 5, Hall sensor 6 and warning light 4 are connected to the intelligent display and control terminal 1 through a line, and the engine is started to power on the intelligent display and control terminal 1, and the system automatically runs. The image, longitude and latitude information, and spindle speed of the residual film recovery machine are quickly acquired and information is integrated, and uploaded to the cloud server in real time.

[0122] In an exemplary embodiment, a smart display and control terminal 1 fixing bracket and a smart display and control terminal 1 fixing plate are provided on the tractor cab frame, the smart display and control terminal 1 is connected to the smart display and control terminal 1 fixing bracket, the smart display and control terminal 1 fixing bracket is connected to the smart display and control terminal 1 fixing plate, the smart display and control terminal 1 fixing plate is a "C"-shaped fixing plate with a bend, and is fixedly connected to the frame in the tractor cab.

[0123] like Figure 3 As shown, the working process of calculating the picking rate in the real-time operation monitoring device for the residual film recovery machine provided in the embodiment of the present application is as follows:

[0124] (1) After the residual film recycling machine is powered on and started, it receives the 5m×3m plot image of the ground surface after work taken by camera 2 (the camera is set to take one picture every 6s by writing a program in Python). The residual film recycling machine is required to maintain a steady speed while driving to sequentially collect the ground surface images after collection.

[0125] (2) The collected surface images are transmitted to the cloud server in real time through the remote communication module, and the data set is divided into a training set and a test set to train the classification model to obtain a classification recognition model.

[0126] (3) The cloud server uses a classification and recognition model to screen effective images that can be used to evaluate the pickup rate under complex operating conditions of the residual film recycling machine. If the output result of the classification and recognition model is 1, it is valid.

[0127] (4) The cloud server uses the embedded semantic segmentation model to perform residual film recognition and segmentation on the filtered valid images to obtain the category of each pixel on the image. Python is then used to calculate the total percentage of residual film pixels in each image relative to the total percentage of pixels in the entire image to obtain the corresponding pickup rate of each image. Finally, the cloud server calculates the average value based on the pickup rate of each image to obtain the final pickup rate for evaluation.

[0128] (5) The cloud server transmits the real-time updated collection rate of the residual film recycling machine to the intelligent display and control terminal 1 for display.

[0129] The working process of calculating the operating area in the real-time operation monitoring device for the residual film recovery machine provided in the embodiment of the present application is as follows:

[0130] (1) After the residual film recovery machine is powered on and started, the positioning device 3 receives the position coordinate information in real time.

[0131] (2) The collected coordinate information is transmitted to the cloud server in real time through the remote communication module.

[0132] (3) The cloud server uses Gaussian projection to convert the coordinate information into plane coordinate system coordinates.

[0133] (4) The cloud server uses a triangulation algorithm to divide the initial job vertex data (the coordinate points obtained in the first three moments) into triangles.

[0134] (5) During the operation, real-time updated coordinate information is inserted, and the original triangle is updated using the triangulation algorithm. Connections are constructed with the vertices of the original triangle to generate new triangles. The area of ​​each triangle is calculated, and these areas are accumulated to obtain a rough operation area.

[0135] (6) Use the grid method to mark whether each triangle is covered by one or more work areas and obtain the repeated work area.

[0136] (7) Finally, the rough operation area is subtracted from the repeated operation area to obtain the operation area.

[0137] (8) The cloud server transmits the real-time updated operation area to the intelligent display and control terminal 1 for display.

[0138] The working process of status monitoring in the real-time operation monitoring device of the residual film recovery machine provided in the embodiment of the present application is as follows:

[0139] (1) After the residual film recovery machine is powered on, the Hall sensor 6 is used to obtain the pulse signal of the spindle speed. Then the cloud server starts the timer to count the pulses and calculates the spindle speed using the period measurement method to ensure that the measured speed value is accurate enough. The built-in program of the intelligent display and control terminal 1 encodes the data according to the communication protocol, displays the spindle speed and triggers an alarm signal when it exceeds the preset alarm threshold. The specific steps are as follows: Figure 5 shown.

[0140] (2) Use the inclination sensor 5 installed on the pull rod of the traction frame to obtain the inclination angle between the traction frame and the horizontal base surface ( Figure 6 This is the data collection flow chart of the inclination sensor 5. When the system starts, it issues a data collection instruction, triggers the buffer area of ​​the serial communication interface and prepares to receive data. After successfully receiving the data receipt returned by the sensor, the data integrity check is immediately performed. After verification, the 3-byte data content of the receipt is decoded and processed, and the data value is converted to a floating point type for subsequent processing and calculation). The cloud server combines the radial length of the upper pull rod to obtain the zero-state vertical displacement difference when not working, and calculates the depth change value of the film tooth 7 relative to the zero-state position after the residual film recovery machine is working, which is the depth of the film tooth 7. The depth of the film tooth 7 is displayed and an alarm signal is triggered when it exceeds the preset alarm threshold.

[0141] The real-time operation monitoring device provided in this application is installed on the residual film recycling machine, and the cloud server workflow is as follows Figure 7As shown, after the system is running, the cloud server receives the positioning coordinate data, image data, and operation sensor data transmitted by the intelligent display and control terminal in real time. After calculation and analysis, the operating area, residual film recovery rate, spindle speed and other data information are transmitted to the residual film recovery information monitoring cloud platform through the remote communication module, and finally a residual film recovery machine operation quality assessment report is issued through the cloud platform.

[0142] The real-time operation monitoring device installed on the residual film recovery machine provided by the present application is easy to install, economical and practical. It has a reasonable structure, efficient and accurate operation, and a high degree of automation. It can provide a hardware foundation for real-time monitoring and evaluation of the pickup rate, operating area and operating status of the residual film recovery machine, and supervision of the violations of operators. The monitoring of the pickup rate of the residual film recovery machine is achieved through the combination of on-board visual imaging and deep learning methods; the monitoring of the operating area is achieved through the reception of position coordinate information by the positioning antenna combined with Gaussian projection and triangulation algorithms; the monitoring of the spindle speed and film-lifting tooth depth of the residual film recovery machine is achieved by receiving sensor data and analyzing and calculating.

[0143] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store real-time operation supervision data of residual film recovery equipment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a real-time operation supervision method for residual film recovery equipment is implemented.

[0144] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0145] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0146] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned method embodiments are implemented.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0148] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0150] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0151] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for real-time operation supervision of a residual film recovery machine, characterized in that: The real-time operation supervision method of the residual film recovery machine comprises: At any moment during the operation of the residual film recovery machine, the scene image of the residual film recovery machine at the current moment, the position coordinates of the residual film recovery machine, the spindle speed of the residual film recovery machine, and the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground are obtained; Inputting the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; If the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; the category is residual film or not residual film; Determine the ratio of the total number of pixel points of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment as the picking rate of the residual film recycling machine at the current moment; Using the triangulation algorithm, the triangle at the previous moment is updated according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area; The operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment; The film tooth depth at the current moment is calculated according to the angle between the upper pull rod of the traction frame on the residual film recovery tool and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery tool at the current moment.

2. The real-time operation supervision method of residual film recovery equipment according to claim 1 is characterized in that: The classification recognition model is obtained by replacing the first convolution module in the DenseNet network model with the first multi-scale serial hole convolution module, and embedding each nonlinear combination function in each dense block in the DenseNet network model into the SE-Block module.

3. The real-time operation supervision method of residual film recovery equipment according to claim 1 is characterized in that: The semantic segmentation model includes an encoder and a decoder; The encoder includes a second multi-scale cascaded hole convolution module, N maximum pooling downsampling modules and N+1 SE-Dense Block convolution modules; The SE-Dense Block convolution module includes a SE-Dense Block module and a first convolution module connected in sequence; the output end of the second multi-scale serial hole convolution module is connected to the input end of the SE-DenseBlock module in the first SE-Dense Block convolution module; the output end of the first convolution module in the i-th SE-Dense Block convolution module is connected to the input end of the i-th maximum pooling downsampling module, and the output end of the i-th maximum pooling downsampling module is connected to the input end of the first convolution module in the i+1-th SE-Dense Block convolution module, 1≤i≤N; the SE-Dense Block module is obtained by embedding each nonlinear combination function in any dense block in the DenseNet network model into the SE-Block module front and back; The decoder includes N+1 upsampling convolution modules connected in sequence; the upsampling convolution module includes upsampling modules and convolution layers connected in sequence; the convolution layer includes a plurality of second convolution modules connected in sequence; The first convolution module in the encoder is connected to the upsampling module in the first upsampling convolution module in the decoder; The nth SE-Dense Block module in the encoder is jump-connected to the first second convolution module in the N+2-nth upsampling convolution module in the decoder, 1≤n≤N+1.

4. A real-time operation monitoring device for residual film recovery equipment, used to implement the real-time operation monitoring method for residual film recovery equipment described in any one of claims 1 to 3, characterized in that: The real-time operation monitoring device of the residual film recovery machine comprises: A cloud server and a camera, a positioning device, a tilt sensor, and a Hall sensor respectively connected to the cloud server; At any moment during the operation of the residual film recovery machine, the camera is used to obtain the scene image of the residual film recovery machine at the current moment, the positioning device is used to obtain the position coordinates of the residual film recovery machine at the current moment, and the inclination sensor is used to measure the angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground at the current moment; the Hall sensor is used to measure the pulse signal of the main shaft speed of the residual film recovery machine at the current moment; The cloud server is used to input the scene image of the residual film recycling machine at the current moment into the classification recognition model to determine whether the scene image of the residual film recycling machine at the current moment is valid; if the scene image of the residual film recycling machine at the current moment is valid, the scene image of the residual film recycling machine at the current moment is input into the semantic segmentation model to obtain the category of each pixel in the scene image of the residual film recycling machine at the current moment; determine that the ratio of the total number of pixels of the residual film in the scene image of the residual film recycling machine at the current moment to the total number of pixels of the scene image of the residual film recycling machine at the current moment is the picking rate of the residual film recycling machine at the current moment; the category is residual film or not residual film; The cloud server is also used to use a triangulation algorithm to update the triangle at the previous moment according to the position coordinates of the residual film recovery machine at the current moment to obtain the triangle at the current moment; the triangle at the initial moment is obtained according to the vertices of the residual film recovery machine's operating area; the operating area of ​​the residual film recovery machine at the current moment is obtained by subtracting the overlapping area between the triangles at the current moment from the sum of the areas of the triangles at the current moment; The cloud server is also used to calculate the film tooth depth at the current moment according to the angle between the upper pull rod of the traction frame on the residual film recovery machine and the horizontal ground and the length of the upper pull rod of the traction frame on the residual film recovery machine; The cloud server is also used to calculate the spindle speed of the residual film recovery machine at the current moment based on the pulse signal of the spindle speed of the residual film recovery machine at the current moment measured by the Hall sensor.

5. The real-time operation monitoring device for residual film recovery equipment according to claim 4 is characterized in that: The real-time operation monitoring device of the residual film recovery machine also includes: An intelligent display and control terminal and a remote communication module, wherein the intelligent display and control terminal is respectively connected to the remote communication module, the camera, the positioning device, the tilt sensor and the Hall sensor; The intelligent display and control terminal is used to send a pulse signal of the current residual film recovery machine scene image acquired by the camera, the current position coordinates of the residual film recovery machine acquired by the positioning device, the current angle between the pull rod on the traction frame of the residual film recovery machine and the horizontal ground measured by the inclination sensor, and the current spindle speed of the residual film recovery machine measured by the Hall sensor to the cloud server through the remote communication module; The cloud server is also used to send the current picking rate of the residual film recovery machine, the current operating area of ​​the residual film recovery machine, the current film tooth depth and the current spindle speed of the residual film recovery machine to the intelligent display and control terminal through the remote communication module for display.

6. The real-time operation monitoring device for residual film recovery equipment according to claim 5 is characterized in that: The real-time operation monitoring device of the residual film recovery machine also includes: An alarm light connected to the intelligent display and control terminal controls the alarm light to sound an alarm according to the film tooth depth at the current moment and the spindle speed of the residual film recovery machine at the current moment.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the real-time operation supervision method for the residual film recovery equipment according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the real-time operation supervision method of the residual film recovery equipment described in any one of claims 1 to 3 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the real-time operation supervision method of the residual film recovery equipment described in any one of claims 1 to 3 is implemented.

Images