Sugarcane impurity detection scheme based on key frame extraction

Abstract

The invention discloses a sugarcane impurity detection scheme based on key frame extraction, and the scheme comprises the following steps: obtaining video monitoring data after sugarcane is transported and unloaded into a sugarcane groove through a monitoring camera, carrying out the splitting of the video monitoring data, obtaining image frames, constructing a video signal, carrying out the extraction of a key frame of long monitoring based on a sparse representation algorithm, obtaining features, and carrying out the detection of the impurities of the sugarcane. And automatically identifying key frames with impurities in monitoring, and triggering an abnormal alarm. According to the method, monitoring videos of the sugarcanes in the transportation process and after the sugarcanes are unloaded into the sugarcane grooves are extracted through the key frames, possible abnormal conditions can be found in real-time monitoring, and high automation of sugarcane impurity detection under the scene of large transportation volume and long transportation time can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of sugarcane impurity detection, and particularly refers to a sugarcane impurity detection based on video key frame extraction. Background Art

[0002] Guangxi is the largest sugarcane planting and sucrose production base in China, and the sugarcane industry occupies an important position in the local economic development. However, during the processes of sugarcane harvesting, transportation, and processing, impurities (such as soil, leaves, withered leaves, stones, and other foreign objects) are mixed into the raw materials, seriously affecting the extraction efficiency of sucrose and the quality of the final products. Therefore, efficient and automated impurity detection and removal technologies are crucial for improving the intelligent level of the sugar manufacturing industry in Guangxi.

[0003] Traditional sugarcane impurity detection relies on manual screening, but these methods have many limitations such as high manual detection costs, low efficiency, and being easily affected by subjective factors. In recent years, with the development of computer vision and deep learning technologies, impurity detection solutions based on intelligent video analysis have good development prospects.

[0004] The key frame extraction technology plays an important role in video analysis. It can automatically select the most representative image frames from a continuous video stream, thereby reducing data redundancy and improving analysis efficiency. In the application of sugarcane impurity detection, the key frame extraction technology can be used to automatically screen the key pictures during transportation, extract the typical scenes containing impurities, and combine with a deep learning model for classification and recognition. By introducing the key frame extraction technology, the sugar manufacturing industry can automatically detect impurities before the sugarcane raw materials enter the processing link, thereby improving the raw material quality, reducing production losses, and promoting the development of the sugarcane industry towards intelligence and precision. Therefore, it is necessary to develop a sugarcane impurity detection solution based on key frame extraction. Summary of the Invention

[0005] The purpose of the present invention is to provide a sugarcane impurity detection solution based on key frame extraction to solve the problems existing in the above-mentioned prior art, improve the automation level of the sugar manufacturing industry, and reduce the detection cost.

[0006] To achieve the above purpose, the present invention provides the following solution: The present invention provides a sugarcane impurity detection solution based on key frame extraction, including the following steps:

[0007] Obtain video signal data or image data from sugarcane transportation monitoring, split the monitoring signal to obtain image frames, and based on the obtained image frames, construct an impurity detection video signal matrix;

[0008] Construct a model: Based on the MCP sparse constraint, construct a sparse representation model;

[0009] Input the video signal matrix into the sparse representation model, optimize the sparse representation model using DC coding, calculate the sparse coefficient matrix, and based on the sparse coefficient matrix, analyze the sparse representation result. Among them, frames with larger changes may be more representative. Based on this, detect the changes in the image content and obtain the key frame index;

[0010] Based on the key frame index, extract the key frames in the surveillance video.

[0011] Optionally, constructing the video signal matrix based on the image frames includes: extracting the image information of the image frames in the surveillance video, and using the image information of each frame as columns to construct the video signal matrix.

[0012] Optionally, the sparse representation model is:

[0013]

[0014] where S is the video signal matrix, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, where λ is the weight coefficient.

[0015] Optionally, the sparse coefficient matrix is:

[0016]

[0017] where X is the sparse coefficient matrix.

[0018] Optionally, optimizing the sparse representation model using DC coding and calculating the sparse coefficient matrix includes:

[0019] Use the DC decomposition method to decompose the sparse coefficient matrix into the difference of two convex functions, as shown in the following formula:

[0020]

[0021] where S is the video matrix signal, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, and ||·|| represents the norm function;

[0022] Use the DC algorithm to calculate the sparse coefficient matrix.

[0023] Optionally, the method for calculating the sparse coefficient matrix using the DC algorithm is:

[0024]

[0025] where S is the video matrix signal, X is the sparse coefficient matrix, and J MCP(X) is the MCP sparse constraint, λ is the weight coefficient, < > represents the inner product operation, represents the partial derivative operation, and ||·|| represents the norm operation function.

[0026] Optionally, based on the sparse coefficient matrix, obtaining the key frame index includes: extracting the non-zero rows in the sparse coefficient matrix, which are the key frame indexes.

[0027] Optionally, the extraction method further includes verifying the effect of extracting key frames using the key frame compression rate.

[0028] The present invention discloses the following technical effects:

[0029] A sugarcane impurity detection solution based on key frame extraction provided by the present invention extracts key frames in video monitoring, extracts a small number of key frames among them, judges abnormal situations existing in the transportation process through the above key frames, and approximately uses the frames to replace the original video for monitoring the transportation process, improving the automation degree and efficiency of the monitoring system; the extraction of key frames optimizes the non-convex MCP regularization key frame extraction problem by applying DC coding, provides a set of sub-problem optimization solution methods to obtain accurate key frames, and at the same time improves the calculation speed of the key frame extraction algorithm; this method is simple to operate, inputs the monitoring video signal, and through the steps of this sugarcane impurity detection solution based on key frame extraction in sequence, the video key frames can be obtained and a judgment can be made based on the impurity situation in the sugarcane. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1 It is a schematic diagram of a sugarcane impurity detection model based on key frame extraction of the present invention;

[0032] Figure 2 It is a schematic diagram of a sugarcane impurity detection method based on key frame extraction in an embodiment of the present invention. Specific Embodiments

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0034] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The present invention provides a sugarcane impurity detection solution based on key frame extraction, as Figure 1-2 shown. Specifically, it includes:

[0036] Obtain video monitoring data after sugarcane is transported and unloaded into the cane trough through a monitoring camera, perform preprocessing, split it into image frames, and represent the above image frames in the form of columns in a matrix. The matrix composed of each column is the desired video signal matrix S. By the above operations, a monitoring video signal matrix is constructed, and the original video is expressed in a matrix form that can be sparsely represented.

[0037] The video signals obtained from the monitoring are all composed of multiple consecutive video frames, and there may be a large number of repeated and similar video frames in the long-term monitoring video, with a lot of redundant information. Each video frame is an information-containing image extracted from the original video. Extract the pixel points in each video frame, arrange and combine them in sequence from left to right and from top to bottom to form a feature vector. Each video frame can correspondingly obtain a feature vector, and the feature vectors of each video frame are combined to form the video signal matrix S, that is

[0038] S = [s1, s2, …, s i , …, s N , s i (1 ≤ i ≤ N) is the feature vector of the i-th video frame.

[0039] Construct a sparse representation model based on Minimax Concave Penalty (MCP) sparse regularization, so that the effect of key frame extraction has a lower compression rate through this model. This model introduces MCP sparse regularization constraints to strongly sparsely constrain the generated sparse coefficient matrix. This constraint can make the sparse matrix have strong sparsity and can use convex optimization methods to solve problems.

[0040] Initial two matrices Y, D, where Y is the original signal matrix and D is the dictionary matrix. Substitute the matrices Y, D into the following formula (1) to calculate the sparse coefficient matrix X.

[0041]

[0042] where f(X) is the sparse representation model function, Y is the original signal matrix, X is the sparse coefficient matrix, D is the dictionary matrix, J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, and ||·|| represents the norm operation function.

[0043] Replace the original signal matrix Y and the dictionary matrix D in Equation (1) with the video signal matrix S, and the constructed sparse representation model is shown in Equation (2):

[0044]

[0045] where S is the video matrix signal, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, and generally takes a value greater than 0 and less than 1. Through the substitution of the video signal matrix, the constructed sparse representation model is used for key frame extraction.

[0046] Based on the DC coding to optimize the sparse representation model, the sparse coefficient matrix is calculated, as Figure 2 shown, where the non-zero rows of the sparse coefficient matrix represent the indices of the key frames. DC coding can solve the non-convex MCP regularization key frame extraction problem to obtain accurate key frames, and at the same time, using DC coding can improve the calculation speed of the key frame extraction algorithm.

[0047] Since the product of the dictionary (i.e., the video matrix) and the sparse coefficient matrix is the video signal matrix, the key frame solving problem can form an optimization sparse representation problem, that is, to find the sparse coefficient matrix that satisfies the MCP sparse constraint. In this embodiment, the DC coding is used to optimize the sparse representation model, and the sparse coefficient matrix X is calculated. The solution formula for calculating the sparse coefficient matrix X is shown in Equation (3):

[0048]

[0049] where S is the video matrix signal, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, generally takes a value greater than 0 and less than 1, and ||·|| represents the norm function.

[0050] Perform DC decomposition on Equation (3) and divide it into the form of the difference between two convex functions as shown in Equation (4):

[0051]

[0052] where S is the video matrix signal, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, generally takes a value greater than 0 and less than 1, and ||·|| represents the norm operation function.

[0053] Use the DC algorithm to calculate the value of X, and the solution formula is shown in Equation (5):

[0054]

[0055] where S is the video matrix signal, X is the sparse coefficient matrix, and J MCP(X) is the MCP sparse constraint, λ is the weight coefficient, generally taking values greater than 0 and less than 1, < > represents the inner product operation, represents the partial derivative operation, and ||·|| represents the norm operation function.

[0056] The calculated X is the sparse coefficient matrix. Most elements in this sparse matrix are zero, and the non-zero rows represent the key frame indices in the video. For example, if the i-th and j-th rows in X are non-zero rows, then the key frame indices of the video are {i, j}. The corresponding video key frames are selected according to the obtained key frame indices. For example, if the key frame indices of the video are {i, j}, then the key frames of the video are the i-th video frame image and the j-th video frame image. Associating the key frame indices with the key frames can quickly and accurately bind the key frames, improving the accuracy and efficiency of key frame extraction.

[0057] For the key frames extracted from the surveillance video, the video key frame gan compression rate and F-measure are used as evaluation metrics. The key frame compression rate is an objective criterion for evaluating video processing, representing the ratio of the number of extracted key frames to all frames of the video, with its unit being %, and the smaller the value, the stronger the video compression level. Given a video, its key frame compression rate (Summary length) is shown in Equation (6)

[0058]

[0059] where, N select is the number of extracted key frames, and N whole is the total number of frames of the video.

[0060] Meanwhile, in this embodiment, F-measure is used to measure the accurate effect of the extracted key frames. The definition of F-measure is shown in Equation (7):

[0061]

[0062] where, P and R are the precision rate and recall rate respectively. The higher the F-measure, the better the effect of the extracted key frames, and the more accurately it can reflect the content of the original video.

[0063] To improve the efficiency of the sugarcane impurity detection system, the present invention introduces a key frame extraction method based on MCP sparse representation. This key frame extraction method effectively extracts key frames from continuous and similar video frames, reducing the monitoring cost and complexity. The MCP sparse regularization constraint introduced by this model strongly sparsely constrains the generated sparse coefficient matrix, which can effectively reduce the storage cost and processing cost when dealing with tasks with large scales and complex transportation lines. Additionally, by inputting the original surveillance signal and through the above steps, accurate video key frames can be obtained, without the need for manual consumption of a large amount of time to view surveillance information, which is relatively convenient and simple.

[0064] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A sugarcane impurity detection scheme based on key frame extraction, characterized in that: It includes the following steps: Obtain video signal data or image data from sugarcane transportation monitoring, split the monitoring signal to obtain image frames, and construct an impurity detection video signal matrix based on the obtained image frames; Construct a sparse representation model for sugarcane impurity monitoring based on MCP sparse constraints; Input the video signal matrix of sugarcane transportation monitoring into the sparse representation model, optimize the sparse representation model using DC coding, calculate the sparse coefficient matrix, and obtain the key frame index for sugarcane impurity detection based on the sparse coefficient matrix; Extract the key frames in the video based on the above key frame index and judge the parts that may be abnormal during the freight process.

2. The sugarcane impurity detection solution based on key frame extraction according to claim 1, characterized in that, The construction of the freight video signal matrix based on the image frames obtained by processing the video signal of sugarcane unloading into the cane trough through a camera includes: extracting the image information of the image frames and constructing a video signal matrix for sugarcane impurity detection with the obtained image information of each frame as columns.

3. The sugarcane impurity detection solution based on key frame extraction according to claim 1, characterized in that The sparse representation model is: where S is a video signal matrix, X is a sparse coefficient matrix, and J MCP (X) is an MCP sparse constraint, where λ is a weight coefficient.

4. The sugarcane impurity detection solution based on key frame extraction according to claim 1, characterized in that, The sparse coefficient matrix is: Where X is the sparse coefficient matrix.

5. The sugarcane impurity detection solution based on key frame extraction according to claim 4, characterized in that, Optimizing the sparse representation model using DC coding and calculating the sparse coefficient matrix includes: Using the DC decomposition method to decompose the sparse coefficient matrix into the form of the difference between two convex functions as shown in the following formula: Among them, S is the video matrix signal, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, and ||·|| represents the norm function; Using the DC algorithm to calculate the sparse coefficient matrix.

6. The sugarcane impurity detection solution based on key frame extraction according to claim 5, characterized in that, The method of using the DC algorithm to calculate the sparse coefficient matrix is: Among them, S is the video matrix signal, X is the sparse coefficient matrix, and J MCP (X) is the MCP sparse constraint, λ is the weight coefficient, < > represents the inner product operation, represents the partial derivative operation, and ||·|| represents the norm operation function.

7. The sugarcane impurity detection solution based on key frame extraction according to claim 1, characterized in that, Based on the sparse coefficient matrix, obtaining the key frame index for impurity detection includes: extracting the non-zero rows in the sparse coefficient matrix, which are the key frame indexes for sugarcane impurity detection.

8. The sugarcane impurity detection solution based on key frame extraction according to claim 1, characterized in that The method for extracting the key frames for sugarcane impurity detection also includes verifying the effect of the extracted key frames using the key frame compression ratio.

9. The sugarcane impurity detection solution based on key frame extraction according to claim 1, characterized in that, Analyze the key frames and make decisions using the results of sparse representation analysis, such as dealing with abnormal impurities and analyzing the impurity content rate of sugarcane.

Images