A multi-sensor fusion-based harvester grain breakage detection and adjustment system

Abstract

This invention relates to the field of intelligent harvester detection technology, and discloses a harvester grain breakage detection and adjustment system based on multi-sensor fusion. The system includes: a multi-sensor acquisition module, which acquires image information characterizing surface breakage of the grain, laser reflection signals characterizing microcracks on the grain surface, and capacitance value change signals characterizing internal structural breakage of the grain; a data fusion processing module, which uses a weighted fusion algorithm to comprehensively analyze the image information, laser reflection signals, and capacitance value change signals to generate a comprehensive grain breakage rate; and an adjustment control module, which adjusts the harvester's performance parameters to reduce the grain breakage rate when the comprehensive grain breakage rate exceeds a preset threshold, and performs closed-loop control based on the adjusted comprehensive breakage rate. This invention can improve the efficiency and reliability of grain breakage detection in harvesters, thereby significantly enhancing the intelligence level of grain breakage detection and adjustment and the harvester's ability to adapt to complex working conditions.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for harvesters, and in particular to a harvester grain damage detection and adjustment system based on multi-sensor fusion. Background Technology

[0002] Currently, intelligent inspection in grain harvesters primarily relies on machine vision technology. This technology uses cameras to capture images of the grains and employs image processing algorithms to analyze surface damage, enabling rapid detection of grain breakage. However, it has significant limitations. Firstly, machine vision can only detect surface damage, failing to identify micro-cracks or internal structural damage, resulting in a limited scope and difficulty in comprehensively assessing grain breakage. Secondly, machine vision is highly dependent on environmental conditions. In complex environments such as insufficient light, impurities on the grain surface, vibration interference, or complex backgrounds, detection accuracy and reliability significantly decrease, leading to false positives or false negatives and ultimately lower accuracy. Furthermore, existing parameter adjustment mechanisms lack intelligence and real-time capabilities, failing to optimize performance parameters comprehensively based on inspection results, thus hindering the high-precision, complex-condition-critical intelligent inspection and adjustment capabilities required for optimal performance.

[0003] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a harvester grain breakage detection and adjustment system based on multi-sensor fusion.

[0005] In a first aspect, the present invention provides a harvester grain breakage detection and adjustment system based on multi-sensor fusion, the technical solution of which is as follows:

[0006] It includes: a multi-sensor acquisition module, a data fusion processing module, and an adjustment and control module;

[0007] The multi-sensor acquisition module is used to: acquire image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain, respectively.

[0008] The data fusion processing module is used to: perform a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate;

[0009] The adjustment control module is used to: reduce the grain breakage rate by adjusting the performance parameters of the harvester when the overall grain breakage rate is greater than a preset threshold, and perform closed-loop control based on the adjusted overall breakage rate.

[0010] The beneficial effects of the harvester grain breakage detection and adjustment system based on multi-sensor fusion of the present invention are as follows:

[0011] The system of this invention can effectively solve the problems of existing machine vision detection being limited in content, highly dependent on the environment, and lacking intelligent parameter adjustment mechanisms. It improves the efficiency and reliability of grain breakage detection in harvesters, thereby significantly enhancing the intelligence level of grain breakage detection and adjustment in harvesters and their ability to adapt to complex working conditions.

[0012] Based on the above solution, the harvester grain damage detection and adjustment system based on multi-sensor fusion of the present invention can be further improved as follows.

[0013] In one alternative approach, the data fusion processing module is specifically used for:

[0014] The image information is subjected to median filtering and the image damage rate is extracted based on edge detection and texture recognition algorithms;

[0015] The laser reflection signal is subjected to time-domain signal analysis to obtain the light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate.

[0016] The internal structure damage rate is obtained by filtering and feature extraction of the capacitance value change signal.

[0017] Assign corresponding weighting coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate;

[0018] The overall grain damage rate is obtained by weighting the image damage rate, the microcrack damage rate, and the internal structure damage rate according to each weight coefficient.

[0019] In one optional embodiment, the performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; the adjustment control module is specifically used for:

[0020] When the overall grain breakage rate exceeds a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of the following: a command to reduce vehicle speed, a command to reduce engine speed, and a command to reduce drum speed.

[0021] The actuator corresponding to the real-time control strategy adjusts at least one parameter among the harvester's vehicle speed, engine speed, and drum speed to reduce the harvester's grain breakage rate.

[0022] In one alternative approach, the multi-sensor acquisition module is specifically used for:

[0023] The machine vision sensor is used to acquire the image information characterizing the damage to the surface of the grain;

[0024] The laser reflection signal, which characterizes the microcracks on the surface of the grain, was collected using a laser displacement sensor.

[0025] A capacitance sensor is used to collect the capacitance value change signal, which characterizes the damage to the internal structure of the grain.

[0026] Secondly, this invention provides a method for detecting and adjusting grain breakage in harvesters based on multi-sensor fusion. The technical solution of this method is as follows:

[0027] Image information characterizing surface damage of grains, laser reflection signals characterizing microcracks on the grain surface, and capacitance value change signals characterizing internal structural damage of grains were collected respectively.

[0028] A weighted fusion algorithm is used to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate.

[0029] When the overall grain breakage rate exceeds a preset threshold, the performance parameters of the harvester are adjusted to reduce the grain breakage rate, and closed-loop control is performed based on the adjusted overall breakage rate.

[0030] The beneficial effects of the harvester grain breakage detection and adjustment method based on multi-sensor fusion of the present invention are as follows:

[0031] The method of this invention can effectively solve the problems of limited detection content, strong environmental dependence, and unintelligent parameter adjustment mechanism in existing machine vision systems. It improves the efficiency and reliability of grain breakage detection in harvesters, thereby significantly enhancing the intelligence level of grain breakage detection and adjustment in harvesters and their ability to adapt to complex working conditions.

[0032] Based on the above scheme, the harvester grain damage detection and adjustment method based on multi-sensor fusion of the present invention can be further improved as follows.

[0033] In one optional approach, the step of using a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate includes:

[0034] The image information is subjected to median filtering and the image damage rate is extracted based on edge detection and texture recognition algorithms;

[0035] The laser reflection signal is subjected to time-domain signal analysis to obtain the light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate.

[0036] The internal structure damage rate is obtained by filtering and feature extraction of the capacitance value change signal.

[0037] Assign corresponding weighting coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate;

[0038] The overall grain damage rate is obtained by weighting the image damage rate, the microcrack damage rate, and the internal structure damage rate according to each weight coefficient.

[0039] In one optional approach, the performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; when the overall grain breakage rate exceeds a preset threshold, the step of adjusting the harvester's performance parameters to reduce the grain breakage rate includes:

[0040] When the overall grain breakage rate exceeds a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of the following: a command to reduce vehicle speed, a command to reduce engine speed, and a command to reduce drum speed.

[0041] The actuator corresponding to the real-time control strategy adjusts at least one parameter among the harvester's vehicle speed, engine speed, and drum speed to reduce the harvester's grain breakage rate.

[0042] In one optional approach, the steps of acquiring image information characterizing surface damage of the grain, laser reflection signals characterizing microcracks on the grain surface, and capacitance value change signals characterizing internal structural damage of the grain include:

[0043] The machine vision sensor is used to acquire the image information characterizing the damage to the surface of the grain;

[0044] The laser reflection signal, which characterizes the microcracks on the surface of the grain, was collected using a laser displacement sensor.

[0045] A capacitance sensor is used to collect the capacitance value change signal, which characterizes the damage to the internal structure of the grain.

[0046] Thirdly, the technical solution of an electronic device according to the present invention is as follows:

[0047] It includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the harvester grain breakage detection and adjustment method based on multi-sensor fusion as described in this invention.

[0048] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows:

[0049] The computer-readable storage medium stores instructions that, when read by the computer-readable storage medium, cause the computer-readable storage medium to perform the steps of the harvester grain breakage detection and adjustment method based on multi-sensor fusion of the present invention.

[0050] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0051] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0052] Figure 1 This is a schematic diagram of an embodiment of a harvester grain breakage detection and adjustment system based on multi-sensor fusion according to the present invention;

[0053] Figure 2 This is a flowchart of the overall workflow;

[0054] Figure 3 This is a flowchart illustrating an embodiment of a harvester grain breakage detection and adjustment method based on multi-sensor fusion according to the present invention.

[0055] Figure 4 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0056] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0057] Figure 1 This diagram illustrates a structural schematic of an embodiment of a multi-sensor fusion-based harvester grain breakage detection and adjustment system 100 provided by the present invention. Figure 1 As shown, the system 100 includes: a multi-sensor acquisition module 110, a data fusion processing module 120, and an adjustment and control module 130;

[0058] The multi-sensor acquisition module 110 is used to: acquire image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain.

[0059] Here, "grain" refers to the seed particles of crops such as grains or beans processed during harvesting; for example, corn seeds threshed and collected by a corn harvester. "Image information" refers to digital image data acquired by an image sensor, used to characterize the surface morphology and damage of grains; for example, grayscale or RGB images of the surface of corn kernels captured by an industrial camera. "Laser reflection signal" refers to the light signal emitted by a laser displacement sensor and received from the grain surface, used to analyze microscopic deformation or cracks on the grain surface; for example, a laser sensor receiving a signal indicating a change in reflected light intensity due to microcracks on the surface of corn kernels. "Capacitance change signal" refers to the electrical signal detected by a capacitance sensor indicating a change in capacitance caused by changes in the internal structure of the grain; for example, damage inside a corn kernel leading to changes in moisture distribution, causing fluctuations in the capacitance sensor's output signal.

[0060] The data fusion processing module 120 is used to: perform comprehensive analysis on the image information, the laser reflection signal and the capacitance value change signal using a weighted fusion algorithm to generate the comprehensive grain breakage rate.

[0061] The overall grain breakage rate refers to a quantitative indicator obtained through multi-sensor data fusion processing, used to comprehensively evaluate the overall degree of damage to the surface, microcracks and internal structure of the grain, with a value range of 0%-100%.

[0062] The adjustment control module 130 is used to: reduce the grain breakage rate by adjusting the performance parameters of the harvester when the overall grain breakage rate is greater than a preset threshold, and perform closed-loop control based on the adjusted overall breakage rate.

[0063] The preset threshold refers to a reference value pre-set in the system to determine whether grain breakage exceeds the limit; for example, setting the threshold for the overall grain breakage rate to 5% triggers parameter adjustment if this value is exceeded. A harvester refers to agricultural machinery used to complete crop harvesting, threshing, cleaning, and collection; for example, a combine harvester performs harvesting and threshing operations in a cornfield. Performance parameters refer to adjustable parameters that affect the harvester's operational effectiveness and equipment operating status; for example, the harvester's vehicle speed, engine speed, and drum speed.

[0064] The technical solution of this embodiment can effectively solve the problems of existing machine vision detection content being limited, strong environmental dependence, and unintelligent parameter adjustment mechanisms, thereby improving the efficiency and reliability of grain damage detection in harvesters, and significantly enhancing the intelligence level of grain damage detection and adjustment in harvesters and their ability to adapt to complex working conditions.

[0065] In an alternative embodiment, the data fusion processing module 120 is specifically used for:

[0066] The image information is processed by median filtering, and the image damage rate is extracted based on edge detection and texture recognition algorithms.

[0067] Image damage rate refers to the numerical value extracted from image information through algorithmic processing, representing the degree of damage to the surface of the kernels; for example, determining the proportion of surface-damaged kernels in a batch of corn kernels through image analysis.

[0068] Specifically, the image information is processed by median filtering to obtain a filtered image. An edge detection algorithm is used to process the filtered image to extract edge features that represent the shape of intact grains and surface cracks. At the same time, a texture recognition algorithm is used to analyze the filtered image to obtain texture features. The edge features and texture features are fused to determine the damaged areas in the grain image. The image damage rate is calculated based on the ratio of the area of ​​the damaged area to the total area of ​​the grain region determined by the edge features that represent the shape of intact grains.

[0069] The laser reflection signal is subjected to time-domain signal analysis to obtain the light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate.

[0070] The light intensity change rate refers to the rate at which the light intensity of the laser reflection signal changes per unit time, used to identify microcracks on the kernel surface; for example, when scanning the surface of corn kernels with a laser, it represents the rate at which the reflected light intensity changes drastically due to microcracks. The microcrack breakage rate refers to the probability or ratio of breakage based on laser reflection signal analysis, indicating the presence of microcracks on the kernel surface; for example, the light intensity change rate can be used to calculate the percentage of kernels with microcracks in the current corn kernel stream.

[0071] Specifically, the laser reflection signal is processed by time-domain signal analysis. The light intensity change rate is obtained by calculating the differential or difference value of the laser reflection signal intensity per unit time. The light intensity change rate is classified and statistically analyzed based on a preset light intensity change rate threshold. The grains corresponding to the light intensity change rate exceeding the target threshold are judged to have microcracks. The microcrack damage rate is calculated based on the ratio of the number of grains judged to have microcracks to the total number of grains counted.

[0072] The internal structure damage rate is obtained by filtering and feature extraction of the capacitance value change signal.

[0073] The internal structure damage rate refers to the percentage of kernels whose internal structure has been damaged, derived from the analysis of capacitance value change signals; for example, the percentage of corn kernels with internal damage can be identified by changes in capacitance signals.

[0074] Specifically, the capacitance value change signal is filtered to obtain a filtered signal; time-domain or frequency-domain features are extracted from the filtered signal to obtain the stable capacitance value or capacitance change curve characteristic of the internal structure of the grain; the feature values ​​are classified based on a preset internal structure damage judgment threshold to determine whether the grain is damaged; the internal structure damage rate is calculated based on the ratio of the number of grains judged as internally damaged to the total number of grains detected.

[0075] Assign corresponding weighting coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate.

[0076] The weighting coefficient refers to the proportional coefficient that represents the importance of different sensor feature values ​​during the multi-sensor data fusion process; for example, the image damage rate, microcrack damage rate, and internal structure damage rate are assigned weights of 0.5, 0.3, and 0.2, respectively.

[0077] Specifically, based on the analytic hierarchy process (AHP) and combined with expert experience or historical experimental data, an importance judgment matrix for each damage rate influencing factor is constructed. The target eigenvector of the importance judgment matrix is ​​calculated and normalized to assign corresponding first, second, and third weight coefficients to the image damage rate, microcrack damage rate, and internal structure damage rate, respectively, with the sum of these weight coefficients being 1. Specifically, by solving for the largest eigenvalue of the importance judgment matrix and its corresponding target eigenvector, which contains three components corresponding to the image damage rate, microcrack damage rate, and internal structure damage rate, respectively, the target eigenvector is normalized so that the sum of each component is 1, thus obtaining the first, second, and third weight coefficients.

[0078] The overall grain damage rate is obtained by weighting the image damage rate, the microcrack damage rate, and the internal structure damage rate according to each weight coefficient.

[0079] Among the above-mentioned optional methods, image processing, laser reflection signal analysis, and capacitance value processing technologies are further utilized to accurately extract the damage rates of the grain surface, microcracks, and internal structure. A comprehensive damage rate is generated through a weighted fusion algorithm, which significantly improves the accuracy and reliability of damage detection.

[0080] In one optional embodiment, the performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; the adjustment control module 130 is specifically used for:

[0081] When the overall grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of the following: a command to reduce vehicle speed, a command to reduce engine speed, and a command to reduce drum speed.

[0082] The real-time control strategy refers to a set of control commands dynamically generated based on current detection results to adjust the performance parameters of the harvester. A speed reduction command is an instruction to control the harvester to reduce its travel speed; for example, controlling the travel hydraulic valve to reduce its opening to decrease speed. An engine speed reduction command is an instruction to control the harvester engine to reduce its speed; for example, adjusting the fuel injection quantity through the controller to reduce engine speed. A drum speed reduction command is an instruction to control the harvester threshing drum to reduce its speed; for example, adjusting the flow rate of the drum drive motor through a hydraulic valve to reduce its speed.

[0083] The actuator corresponding to the real-time control strategy adjusts at least one parameter among the harvester's vehicle speed, engine speed, and drum speed to reduce the harvester's grain breakage rate.

[0084] The vehicle speed value refers to the speed at which the harvester travels in the field; for example, the harvester operates at a speed of 1.5 meters per second. The engine speed value refers to the rotational speed of the harvester's engine; for example, the engine operates at a speed of 2200 revolutions per minute. The threshing drum speed value refers to the rotational speed of the harvester's threshing drum; for example, the threshing drum operates at a speed of 600 revolutions per minute.

[0085] Among the above-mentioned optional methods, the harvester speed, engine speed and drum speed are further adjusted in real time based on the overall grain breakage rate to achieve intelligent and dynamic performance parameter optimization, effectively reduce the grain breakage rate and improve harvest quality.

[0086] In one alternative embodiment, the multi-sensor acquisition module 110 is specifically used for:

[0087] The machine vision sensor is used to acquire image information characterizing the damage to the surface of the grain.

[0088] The machine vision sensor is installed at the grain outlet of the harvester, specifically above the conveying channel after the grain passes through the threshing drum and before entering the collection device. The machine vision sensor includes a high-resolution industrial camera, lens, light source (such as an LED supplemental light), and related support and fixing devices.

[0089] It should be noted that machine vision sensors are primarily used to detect surface damage to grains, including scratches, dents, and cracks. By capturing images of the grains using an industrial camera and applying uniform illumination, surface features can be clearly identified. Image processing algorithms (such as edge detection, color analysis, and texture recognition) can analyze the degree of damage in real time, providing surface damage data to support subsequent system adjustments.

[0090] The laser reflection signal, which characterizes the microcracks on the surface of the grain, is collected using a laser displacement sensor.

[0091] The laser displacement sensors are installed on both sides of the grain conveying channel, approximately 5-10 centimeters away from the grain flow path. The laser emitter and receiver of the laser displacement sensors are fixed on both sides of the conveying channel to ensure that the laser beam can cover the entire area of ​​grain flow.

[0092] It should be noted that the laser displacement sensor identifies microcracks on the grain surface by emitting a laser beam and detecting the rate of change in the intensity of the reflected light. The laser displacement sensor can quickly scan the grain surface and generate three-dimensional contour information of the grain. When microcracks exist on the grain surface, the intensity of the reflected laser light changes. By analyzing the rate of change in light intensity, it is possible to determine whether there is any minute damage to the grain. This detection method features high precision and fast response, and can compensate for the shortcomings of machine vision in microcrack detection.

[0093] A capacitance sensor is used to collect the capacitance value change signal, which characterizes the damage to the internal structure of the grain.

[0094] The capacitive sensor is installed at the bottom or side of the grain conveying channel, making direct contact with the grains. The electrode portion of the capacitive sensor is embedded in the material of the conveying channel, ensuring that a stable capacitive field is formed between the grains and the electrodes as they pass through.

[0095] It should be noted that the capacitive sensor determines whether there is damage to the internal structure of the grain by detecting changes in the capacitance between the grain and the electrode. When the internal structure of the grain is damaged, its internal physical properties, such as moisture distribution and density, will change, resulting in abnormal fluctuations in the capacitance value. The capacitive sensor can collect the capacitance signal of the grain in real time and analyze the internal damage of the grain through signal processing algorithms (such as filtering and feature extraction), providing the system with data on internal structural damage.

[0096] Among the above-mentioned optional methods, machine vision, laser displacement and capacitive sensors are further used to accurately collect information on the surface layer, surface microcracks and internal structure of the grains, respectively, to provide comprehensive and detailed raw data support for subsequent data fusion processing.

[0097] like Figure 2 As shown, the overall workflow of the technical solution in this embodiment is as follows:

[0098] 1) The harvester grain breakage detection and adjustment system based on multi-sensor fusion (hereinafter referred to as the system) is initially in the default working mode. First, it determines whether the grain breakage detection function is enabled. If enabled, it enters the normal working process. If not enabled, it enters the waiting mode. If the function is disabled, it exits the detection function.

[0099] 2) If the grain damage detection function is enabled, then further determine whether the system has been calibrated; if it has been calibrated, proceed directly to the normal working process; if it has not been calibrated, perform the calibration operation first, and then proceed to the normal working process.

[0100] 3) After entering the normal working process, determine whether the harvester is in operation mode; if not in operation mode, enter waiting mode; if in operation mode, start the system; if exiting operation mode during startup, exit the function.

[0101] 4) After the system starts working, the data fusion processing module performs fusion processing on the detection data from the machine vision sensor, laser displacement sensor and capacitive sensor, and simultaneously collects vehicle speed value, roller speed value and engine speed value.

[0102] 5) The data fusion processing module calculates the overall grain breakage rate in real time based on the detection data from machine vision sensors, laser displacement sensors, and capacitive sensors, and sends the breakage rate to the display screen for real-time display.

[0103] 6) If the overall grain breakage rate exceeds the preset threshold, the system automatically generates a real-time control strategy to adjust the performance parameters of the harvester (including vehicle speed, engine speed and drum speed) to reduce the grain breakage rate; after adjustment, it continues to monitor the overall grain breakage rate and forms a closed-loop control based on the feedback results.

[0104] Figure 3 This diagram illustrates a flowchart of an embodiment of a harvester grain breakage detection and adjustment method based on multi-sensor fusion provided by the present invention. Figure 3 As shown, it includes the following steps:

[0105] S1. Collect image information representing surface damage of grains, laser reflection signals representing microcracks on the surface of grains, and capacitance value change signals representing internal structural damage of grains, respectively.

[0106] S2. A weighted fusion algorithm is used to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate.

[0107] S3. When the overall grain breakage rate is greater than a preset threshold, the performance parameters of the harvester are adjusted to reduce the grain breakage rate, and closed-loop control is performed based on the adjusted overall breakage rate.

[0108] In one optional approach, the step of using a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate includes:

[0109] The image information is subjected to median filtering and the image damage rate is extracted based on edge detection and texture recognition algorithms;

[0110] The laser reflection signal is subjected to time-domain signal analysis to obtain the light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate.

[0111] The internal structure damage rate is obtained by filtering and feature extraction of the capacitance value change signal.

[0112] Assign corresponding weighting coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate;

[0113] The overall grain damage rate is obtained by weighting the image damage rate, the microcrack damage rate, and the internal structure damage rate according to each weight coefficient.

[0114] In one optional approach, the performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; when the overall grain breakage rate exceeds a preset threshold, the step of adjusting the harvester's performance parameters to reduce the grain breakage rate includes:

[0115] When the overall grain breakage rate exceeds a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of the following: a command to reduce vehicle speed, a command to reduce engine speed, and a command to reduce drum speed.

[0116] The actuator corresponding to the real-time control strategy adjusts at least one parameter among the harvester's vehicle speed, engine speed, and drum speed to reduce the harvester's grain breakage rate.

[0117] In one optional approach, the steps of acquiring image information characterizing surface damage of the grain, laser reflection signals characterizing microcracks on the grain surface, and capacitance value change signals characterizing internal structural damage of the grain include:

[0118] The machine vision sensor is used to acquire the image information characterizing the damage to the surface of the grain;

[0119] The laser reflection signal, which characterizes the microcracks on the surface of the grain, was collected using a laser displacement sensor.

[0120] A capacitance sensor is used to collect the capacitance value change signal, which characterizes the damage to the internal structure of the grain.

[0121] It should be noted that the beneficial effects of the harvester grain breakage detection and adjustment method based on multi-sensor fusion provided in the above embodiments are the same as those of the harvester grain breakage detection and adjustment system 100 based on multi-sensor fusion, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the method and system embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0122] The multi-sensor fusion-based harvester grain breakage detection and adjustment system 100 of the present invention can be a computer program (including program code) running on a computer device. For example, the multi-sensor fusion-based harvester grain breakage detection and adjustment system 100 of the present invention is an application software that can be used to execute the corresponding steps in the multi-sensor fusion-based harvester grain breakage detection and adjustment method of the present invention.

[0123] In some embodiments, the harvester grain breakage detection and adjustment system 100 based on multi-sensor fusion of the present invention can be implemented in a combination of hardware and software. As an example, the harvester grain breakage detection and adjustment system 100 based on multi-sensor fusion of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the harvester grain breakage detection and adjustment method based on multi-sensor fusion of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0124] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0125] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned harvester grain damage detection and adjustment methods based on multi-sensor fusion. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the harvester grain damage detection and adjustment method based on multi-sensor fusion shown in any embodiment of the present invention by calling the computer program.

[0126] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0127] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0128] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0129] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0130] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0131] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0132] It should be noted that, Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0133] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned harvester grain damage detection and adjustment methods based on multi-sensor fusion.

[0134] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0135] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned harvester grain breakage detection and adjustment method based on multi-sensor fusion.

[0136] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0139] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0140] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0141] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0142] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0143] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A harvester grain breakage detection and adjustment system based on multi-sensor fusion, characterized in that, include: Multi-sensor acquisition module, data fusion processing module, and adjustment control module; The multi-sensor acquisition module is used to: acquire image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain, respectively. The data fusion processing module is used to: perform a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate; The adjustment control module is used to: reduce the grain breakage rate by adjusting the performance parameters of the harvester when the overall grain breakage rate is greater than a preset threshold, and perform closed-loop control based on the adjusted overall breakage rate; The data fusion processing module is specifically used for: The image information is subjected to median filtering and the image damage rate is extracted based on edge detection and texture recognition algorithms; The laser reflection signal is subjected to time-domain signal analysis to obtain the light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate. The internal structure damage rate is obtained by filtering and feature extraction of the capacitance value change signal. Assign corresponding weighting coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The overall grain damage rate is obtained by weighting the image damage rate, the microcrack damage rate, and the internal structure damage rate according to each weight coefficient. The performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; the adjustment control module is specifically used for: When the overall grain breakage rate exceeds a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of the following: a command to reduce vehicle speed, a command to reduce engine speed, and a command to reduce drum speed. The actuator corresponding to the real-time control strategy adjusts at least one parameter among the harvester's vehicle speed, engine speed, and drum speed to reduce the harvester's grain breakage rate.

2. The harvester grain breakage detection and adjustment system based on multi-sensor fusion according to claim 1, characterized in that, The multi-sensor acquisition module is specifically used for: The machine vision sensor is used to acquire the image information characterizing the damage to the surface of the grain; The laser reflection signal, which characterizes the microcracks on the surface of the grain, was collected using a laser displacement sensor. A capacitance sensor is used to collect the capacitance value change signal, which characterizes the damage to the internal structure of the grain.

3. A method for detecting and adjusting grain breakage in a harvester based on multi-sensor fusion, characterized in that, include: Image information characterizing surface damage of grains, laser reflection signals characterizing microcracks on the grain surface, and capacitance value change signals characterizing internal structural damage of grains were collected respectively. A weighted fusion algorithm is used to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate. When the overall grain breakage rate exceeds a preset threshold, the performance parameters of the harvester are adjusted to reduce the grain breakage rate, and closed-loop control is performed based on the adjusted overall breakage rate. The step of using a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate the overall grain breakage rate includes: The image information is subjected to median filtering and the image damage rate is extracted based on edge detection and texture recognition algorithms; The laser reflection signal is subjected to time-domain signal analysis to obtain the light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate. The internal structure damage rate is obtained by filtering and feature extraction of the capacitance value change signal. Assign corresponding weighting coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The overall grain damage rate is obtained by weighting the image damage rate, the microcrack damage rate, and the internal structure damage rate according to each weight coefficient. The performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; when the overall grain breakage rate exceeds a preset threshold, the step of adjusting the harvester's performance parameters to reduce the grain breakage rate includes: When the overall grain breakage rate exceeds a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of the following: a command to reduce vehicle speed, a command to reduce engine speed, and a command to reduce drum speed. The actuator corresponding to the real-time control strategy adjusts at least one parameter among the harvester's vehicle speed, engine speed, and drum speed to reduce the harvester's grain breakage rate.

4. The harvester grain breakage detection and adjustment method based on multi-sensor fusion according to claim 3, characterized in that, The steps of acquiring image information characterizing surface damage of grains, laser reflection signals characterizing microcracks on the grain surface, and capacitance value changes characterizing internal structural damage of grains include: The machine vision sensor is used to acquire the image information characterizing the damage to the surface of the grain; The laser reflection signal, which characterizes the microcracks on the surface of the grain, was collected using a laser displacement sensor. A capacitance sensor is used to collect the capacitance value change signal, which characterizes the damage to the internal structure of the grain.

5. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the harvester grain breakage detection and adjustment method based on multi-sensor fusion as described in claim 3 or 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer-readable storage medium to implement the harvester grain breakage detection and adjustment method based on multi-sensor fusion as described in claim 3 or 4.

Images