Harvester grain breakage detecting and adjusting system based on multi-sensor fusion

Abstract

The invention relates to the technical field of harvester intelligent detection, and discloses a harvester grain breakage detection and adjustment system based on multi-sensor fusion. Respectively acquiring image information representing grain surface damage, a laser reflection signal representing grain surface microcracks and a capacitance value change signal representing grain internal structure damage; the data fusion processing module is used for comprehensively analyzing the image information, the laser reflection signal and the capacitance value change signal by adopting a weighted fusion algorithm to generate a grain comprehensive breakage rate; and the adjustment control module is used for reducing the grain breakage rate by adjusting performance parameters of the harvester when the grain comprehensive breakage rate is greater than a preset threshold value, and performing closed-loop control according to the adjusted comprehensive breakage rate. The grain breakage detection efficiency and reliability of the harvester can be improved, so that the intelligent level of grain breakage detection and adjustment of the harvester and the capacity of adapting to complex working conditions are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection of harvesters, and in particular to a harvester grain damage detection and adjustment system based on multi-sensor fusion. Background Art

[0002] Currently, intelligent detection in grain harvesters mainly relies on machine vision technology. This technology uses a camera to capture image information of grains and uses image processing algorithms to analyze the damage to the surface of the grains. This allows for rapid detection of damaged grains, but it has significant shortcomings. On the one hand, machine vision can only detect damage to the surface of grains and cannot identify microcracks or damage to the internal structure. The detection content is limited, making it difficult to comprehensively assess the damage to the grains. On the other hand, machine vision is highly dependent on environmental conditions. Under complex working conditions such as insufficient lighting, impurities attached to the grain surface, vibration interference, or complex backgrounds, the detection accuracy and reliability will be significantly reduced, and false detection or missed detection are prone to occur, resulting in low accuracy of the detection results. In addition, the existing parameter adjustment mechanism lacks intelligence and real-time performance, and cannot comprehensively optimize performance parameters based on the test results, making it difficult to meet the needs of intelligent detection and adjustment under high-precision and complex working conditions.

[0003] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a harvester grain damage detection and adjustment system based on multi-sensor fusion.

[0005] In a first aspect, the present invention provides a harvester grain damage detection and adjustment system based on multi-sensor fusion, the technical solution of which is as follows: It includes: multi-sensor acquisition module, data fusion processing module and adjustment control module; The multi-sensor acquisition module is used to respectively acquire image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain; The data fusion processing module is used to: use a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal and the capacitance value change signal to generate a comprehensive grain breakage rate; The adjustment control module is used to: when the comprehensive grain breakage rate is greater than a preset threshold, adjust the performance parameters of the harvester to reduce the grain breakage rate, and perform closed-loop control according to the adjusted comprehensive breakage rate.

[0006] The beneficial effects of the harvester grain damage detection and adjustment system based on multi-sensor fusion of the present invention are as follows: The system of the present invention can effectively solve the problems of existing machine vision detection, such as single content, strong environmental dependence and unintelligent parameter adjustment mechanism, and improve the efficiency and reliability of harvester grain damage detection, thereby significantly improving the intelligence level of harvester grain damage detection and adjustment and the ability to adapt to complex working conditions.

[0007] On the basis of the above scheme, the harvester grain damage detection and adjustment system based on multi-sensor fusion of the present invention can also be improved as follows.

[0008] In an optional manner, the data fusion processing module is specifically configured to: Performing median filtering on the image information and extracting the image damage rate based on edge detection and texture recognition algorithms; Performing time domain signal analysis processing on the laser reflection signal to obtain a light intensity change rate, and obtaining a microcrack breakage rate based on the light intensity change rate; Filtering and feature extraction processing are performed on the capacitance value change signal to obtain an internal structure damage rate; assigning corresponding weight coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The image damage rate, the microcrack damage rate and the internal structure damage rate are weightedly calculated according to each weight coefficient to obtain the comprehensive damage rate of the grain.

[0009] In an optional manner, the performance parameters include: a vehicle speed value, an engine speed value, and a drum speed value of the harvester; and the adjustment control module is specifically configured to: When the comprehensive grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of a vehicle speed reduction instruction, an engine speed reduction instruction, and a drum speed reduction instruction; The execution mechanism corresponding to the real-time control strategy is controlled to adjust at least one parameter of the vehicle speed value, engine speed value and drum speed value of the harvester to reduce the grain breakage rate of the harvester.

[0010] In an optional manner, the multi-sensor acquisition module is specifically used to: Using a machine vision sensor to collect the image information representing the damage to the surface of the grain; Using a laser displacement sensor, collecting the laser reflection signal representing the microcracks on the surface of the grain; A capacitance sensor is used to collect the capacitance value change signal indicating damage to the internal structure of the grain.

[0011] In a second aspect, the present invention provides a method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion. The technical solution of the method is as follows: The image information representing the damage of the grain surface, the laser reflection signal representing the micro-cracks on the grain surface and the capacitance value change signal representing the damage of the grain internal structure are collected respectively; A weighted fusion algorithm is used to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate a comprehensive grain breakage rate; When the comprehensive grain breakage rate is greater than a preset threshold, the performance parameters of the harvester are adjusted to reduce the grain breakage rate, and closed-loop control is performed based on the adjusted comprehensive breakage rate.

[0012] The beneficial effects of the harvester grain damage detection and adjustment method based on multi-sensor fusion of the present invention are as follows: The method of the present invention can effectively solve the problems of existing machine vision detection, such as single content, strong environmental dependence and unintelligent parameter adjustment mechanism, and improve the efficiency and reliability of harvester grain damage detection, thereby significantly improving the intelligence level of harvester grain damage detection and adjustment and the ability to adapt to complex working conditions.

[0013] On the basis of the above scheme, the harvester grain damage detection and adjustment method based on multi-sensor fusion of the present invention can also be improved as follows.

[0014] In an optional manner, the step of using a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate a comprehensive grain breakage rate includes: Performing median filtering on the image information and extracting the image damage rate based on edge detection and texture recognition algorithms; Performing time domain signal analysis processing on the laser reflection signal to obtain a light intensity change rate, and obtaining a microcrack breakage rate based on the light intensity change rate; Filtering and feature extraction processing are performed on the capacitance value change signal to obtain an internal structure damage rate; assigning corresponding weight coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The image damage rate, the microcrack damage rate and the internal structure damage rate are weightedly calculated according to each weight coefficient to obtain the comprehensive damage rate of the grain.

[0015] In an optional embodiment, the performance parameters include: a vehicle speed value, an engine speed value, and a drum speed value of the harvester; when the comprehensive grain breakage rate is greater than a preset threshold, the step of reducing the grain breakage rate by adjusting the performance parameters of the harvester includes: When the comprehensive grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of a vehicle speed reduction instruction, an engine speed reduction instruction, and a drum speed reduction instruction; The execution mechanism corresponding to the real-time control strategy is controlled to adjust at least one parameter of the vehicle speed value, engine speed value and drum speed value of the harvester to reduce the grain breakage rate of the harvester.

[0016] In an optional manner, the steps of respectively collecting image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain include: Using a machine vision sensor to collect the image information representing the damage to the surface of the grain; Using a laser displacement sensor, collecting the laser reflection signal representing the microcracks on the surface of the grain; A capacitance sensor is used to collect the capacitance value change signal indicating damage to the internal structure of the grain.

[0017] In a third aspect, the technical solution of an electronic device of the present invention is as follows: It includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps of the harvester grain damage detection and adjustment method based on multi-sensor fusion as described in the present invention.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution: Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the harvester grain damage detection and adjustment method based on multi-sensor fusion according to the present invention.

[0019] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings: Figure 1 Schematic diagram of the structure of an embodiment of a harvester grain damage detection and adjustment system based on multi-sensor fusion according to the present invention; Figure 2 It is the overall workflow diagram; Figure 3 Schematic diagram of a process of an embodiment of a harvester grain damage detection and adjustment method based on multi-sensor fusion according to the present invention; Figure 4 The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0021] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0022] Figure 1 FIG. 1 shows a schematic structural diagram of an embodiment of a harvester grain damage detection and adjustment system 100 based on multi-sensor fusion provided by the present invention. Figure 1 As shown, the system 100 includes: a multi-sensor acquisition module 110, a data fusion processing module 120 and an adjustment control module 130; The multi-sensor acquisition module 110 is used to respectively acquire image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain.

[0023] Here, "seeds" refers to the seed particles of crops such as grains or beans that are processed during harvester operations; for example, corn seeds threshed and collected by a corn harvester during operation. Image information refers to digital image data collected by an image sensor that characterizes the surface morphology and damage of the kernels; for example, a grayscale or RGB image of the surface of a corn kernel captured by an industrial camera. A laser reflection signal refers to the light signal reflected from the kernel surface by a laser displacement sensor emitting a laser beam. This signal is used to analyze microscopic deformations or cracks on the kernel surface; for example, a laser sensor receives a signal indicating a change in reflected light intensity due to microcracks on the kernel surface. A capacitance change signal refers to an electrical signal detected by a capacitance sensor indicating a change in capacitance due to changes in the kernel's internal structure; for example, damage to a kernel can cause a change in moisture distribution, which in turn can cause fluctuations in the output signal of the capacitance sensor.

[0024] The data fusion processing module 120 is used to: use a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal and the capacitance value change signal to generate a comprehensive grain breakage rate.

[0025] Among them, the comprehensive grain damage rate refers to: a quantitative indicator obtained through multi-sensor data fusion processing, which is used to comprehensively evaluate the overall damage degree of the grain surface, microcracks and internal structure, and the value range is 0%-100%.

[0026] The adjustment control module 130 is used to: when the comprehensive grain breakage rate is greater than a preset threshold, adjust the performance parameters of the harvester to reduce the grain breakage rate, and perform closed-loop control based on the adjusted comprehensive breakage rate.

[0027] A preset threshold refers to a pre-set reference value in the system used to determine whether kernel breakage exceeds a specified limit. For example, a threshold for the combined kernel breakage rate is set to 5%, and exceeding this value triggers parameter adjustment. A harvester is an agricultural machine used to harvest, thresh, clean, and collect crops; for example, a combine harvester performs harvesting and threshing operations in a corn field. Performance parameters are adjustable parameters that affect the harvester's performance and operating status; for example, the harvester's vehicle speed, engine speed, and drum speed.

[0028] The technical solution of this embodiment can effectively solve the problems of single content, strong environmental dependence and unintelligent parameter adjustment mechanism of existing machine vision inspection, improve the efficiency and reliability of harvester grain damage detection, and thus significantly improve the intelligence level of harvester grain damage detection and adjustment and the ability to adapt to complex working conditions.

[0029] In an optional manner, the data fusion processing module 120 is specifically configured to: The image information is subjected to median filtering and the image damage rate is extracted based on edge detection and texture recognition algorithms.

[0030] The image damage rate refers to a numerical value representing the degree of damage to the surface of the kernels, extracted through algorithmic processing based on image information; for example, the proportion of kernels with surface damage in a batch of corn kernels is determined through image analysis.

[0031] Specifically, the image information is subjected to median filtering to obtain a filtered image, and the filtered image is operated using an edge detection algorithm to extract edge features that characterize the complete kernel shape and surface cracks. At the same time, the filtered image is analyzed using a texture recognition algorithm to obtain texture features. The edge features and texture features are fused to determine the damaged area in the kernel image. The image damage rate is calculated based on the ratio of the area of ​​the damaged area to the total area of ​​the kernel determined by the edge features that characterize the complete kernel shape.

[0032] The laser reflection signal is subjected to time domain signal analysis processing to obtain a light intensity change rate, and the microcrack breakage rate is obtained based on the light intensity change rate.

[0033] The light intensity change rate refers to the rate at which the laser reflected signal's intensity changes per unit time. This is used to identify microcracks on the kernel surface. For example, when scanning a corn kernel's surface with a laser, the reflected light intensity changes dramatically due to microcracks. The microcrack breakage rate is the probability or ratio of breakage indicating the presence of microcracks on the kernel surface, derived from analysis of the laser reflected signal. For example, the light intensity change rate is used to calculate the percentage of kernels with microcracks in the current corn kernel stream.

[0034] Specifically, the laser reflection signal is subjected to time domain signal analysis and processing, and the light intensity change rate is obtained by calculating the differential or difference value of the laser reflection signal intensity per unit time; the light intensity change rate is classified and counted based on a preset light intensity change rate threshold, and the grains corresponding to the light intensity change rate exceeding the target threshold are judged to have microcracks; the microcrack breakage rate is calculated based on the ratio of the number of grains judged to have microcracks to the total number of grains counted.

[0035] The capacitance value change signal is filtered and feature extracted to obtain the internal structure damage rate.

[0036] The internal structure damage rate refers to the ratio of damage to the internal structure of the kernel, obtained based on the analysis of the capacitance value change signal; for example, the percentage of corn kernels with internal damage identified by the capacitance signal change.

[0037] Specifically, the capacitance value change signal is filtered to obtain a filtered signal; time domain or frequency domain feature extraction is performed on the filtered signal to obtain a stable capacitance value or characteristic value of the capacitance change curve that characterizes the internal structure of the grain; the characteristic value is classified based on a preset internal structure damage judgment threshold to determine whether the inside of the grain is damaged; the internal structure damage rate is calculated based on the ratio of the number of grains determined to be internally damaged to the total number of grains detected.

[0038] Corresponding weight coefficients are assigned to the image damage rate, the microcrack damage rate, and the internal structure damage rate.

[0039] The weight coefficient refers to the proportional coefficient assigned to different sensor eigenvalues ​​to indicate their importance during the multi-sensor data fusion process; for example, weights of 0.5, 0.3, and 0.2 are assigned to the image damage rate, microcrack damage rate, and internal structure damage rate, respectively.

[0040] Specifically, based on the analytic hierarchy process and combined with expert experience or historical experimental data, an importance judgment matrix of each damage rate influencing factor is constructed. By calculating the target eigenvector of the importance judgment matrix and normalizing it, the first, second, and third weight coefficients are assigned to the image damage rate, microcrack damage rate, and internal structure damage rate, respectively, and the sum of the weight coefficients is 1. The first, second, and third weight coefficients are obtained by solving the maximum eigenvalue of the importance judgment matrix and its corresponding target eigenvector, which contains three components corresponding to the image damage rate, microcrack damage rate, and internal structure damage rate, respectively. The target eigenvector is normalized so that the sum of the components is 1.

[0041] The image damage rate, the microcrack damage rate and the internal structure damage rate are weightedly calculated according to each weight coefficient to obtain the comprehensive damage rate of the grain.

[0042] In the above optional method, image processing, laser reflection signal analysis and capacitance value processing technology are further used to accurately extract the damage rate of the grain surface, microcracks and internal structure, and generate a comprehensive damage rate through a weighted fusion algorithm, which significantly improves the accuracy and reliability of damage detection.

[0043] In an optional manner, the performance parameters include: a vehicle speed value, an engine speed value, and a drum speed value of the harvester; the adjustment control module 130 is specifically configured to: When the comprehensive grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of a vehicle speed reduction instruction, an engine speed reduction instruction, and a drum speed reduction instruction.

[0044] The real-time control strategy refers to a set of control instructions dynamically generated based on current detection results and used to adjust the harvester's performance parameters. A speed reduction instruction is an instruction to control the harvester to reduce its travel speed; for example, controlling the travel hydraulic valve to reduce its opening to reduce vehicle speed. An engine speed reduction instruction is an instruction to control the harvester's engine to reduce its speed; for example, a controller can adjust the fuel injection rate to reduce engine speed. A drum speed reduction instruction is an instruction to control the harvester's threshing drum to reduce its speed; for example, a hydraulic valve can be used to adjust the flow rate of the drum drive motor to reduce its speed.

[0045] The execution mechanism corresponding to the real-time control strategy is controlled to adjust at least one parameter of the vehicle speed value, engine speed value and drum speed value of the harvester to reduce the grain breakage rate of the harvester.

[0046] The vehicle speed value refers to the speed of the harvester while operating in the field; for example, a harvester harvests at a speed of 1.5 meters per second. The engine speed value refers to the rotational speed of the harvester's engine; for example, the engine runs at 2200 revolutions per minute. The drum speed value refers to the rotational speed of the harvester's threshing drum; for example, the threshing drum threshes at 600 revolutions per minute.

[0047] In the above optional method, the harvester speed, engine speed and drum speed are further adjusted in real time based on the comprehensive grain breakage rate to achieve intelligent and dynamic performance parameter optimization, effectively reduce the grain breakage rate and improve the harvest quality.

[0048] In an optional manner, the multi-sensor acquisition module 110 is specifically configured to: The image information representing the damage to the surface of the grain is collected using a machine vision sensor.

[0049] The machine vision sensor is installed at the harvester's grain outlet, above the conveyor belt after the grain passes through the threshing drum and before entering the collection device. The machine vision sensor consists of a high-resolution industrial camera, lens, light source (such as an LED fill light), and related mounting fixtures.

[0050] It should be noted that machine vision sensors are primarily used to detect surface damage on kernels, including scratches, dents, and cracks. By capturing images of kernels with industrial cameras and combining them with uniform illumination, surface features can be clearly captured. Image processing algorithms (such as edge detection, color analysis, and texture recognition) can analyze the extent of kernel damage in real time, providing surface damage data to support subsequent system adjustments.

[0051] A laser displacement sensor is used to collect the laser reflection signal representing the microcracks on the surface of the grain.

[0052] Laser displacement sensors are installed on either side of the grain conveyor channel, approximately 5-10 cm from the grain flow path. The laser transmitter and receiver heads of the laser displacement sensors are fixed on either side of the conveyor channel, ensuring that the laser beam covers the entire area where the grain flows.

[0053] It should be noted that laser displacement sensors identify microcracks on the surface of grains by emitting a laser beam and detecting the rate of change in the intensity of the reflected light. Laser displacement sensors can rapidly scan the surface of grains, generating a three-dimensional profile of the grains. When microcracks are present on the grain surface, the intensity of the reflected laser light changes. By analyzing the rate of change in light intensity, the presence of minor damage can be determined. This detection method offers high precision and rapid response, complementing the limitations of machine vision in microcrack detection.

[0054] A capacitance sensor is used to collect the capacitance value change signal indicating damage to the internal structure of the grain.

[0055] The capacitive sensor is installed at the bottom or side of the seed conveying channel, making direct contact with the seed. The electrode portion of the capacitive sensor is embedded in the material of the conveying channel, ensuring that the seed forms a stable capacitive field with the electrode as it passes through.

[0056] It's important to note that capacitive sensors detect changes in the capacitance between the seed and the electrode to determine whether the seed's internal structure is damaged. When the seed's internal structure is damaged, its internal physical properties, such as moisture distribution and density, change, leading to abnormal fluctuations in the capacitance value. Capacitive sensors collect the seed's capacitance signal in real time and analyze the seed's internal damage using signal processing algorithms (such as filtering and feature extraction), providing the system with internal structural damage data.

[0057] In the above optional methods, machine vision, laser displacement and capacitive sensors are further used to accurately collect information on the surface of the grain, surface microcracks and internal structure, respectively, to provide comprehensive and detailed original data support for subsequent data fusion processing.

[0058] like Figure 2 As shown, the overall workflow of the technical solution of this embodiment is: 1) The harvester grain damage detection and adjustment system based on multi-sensor fusion (abbreviated as: system) is initially in the default working mode. First, it determines whether the grain damage detection function is turned on. If it is turned on, it enters the normal working process; if it is not turned on, it enters the waiting mode; if the function is turned off, it exits the detection function.

[0059] 2) If the kernel breakage detection function is enabled, further determine whether the system has been calibrated. If so, directly enter the normal working process. If not, perform the calibration operation first, and then enter the normal working process after completion.

[0060] 3) After entering the normal working process, determine whether the harvester is in the working state; if it is not in the working state, enter the waiting mode; if it is in the working state, start the system; if it exits the working state during the startup process, exit this function.

[0061] 4) After the system is started, the data fusion processing module fuses the detection data from the machine vision sensor, laser displacement sensor and capacitive sensor, and synchronously collects the vehicle speed value, drum speed value and engine speed value.

[0062] 5) The data fusion processing module calculates the comprehensive grain breakage rate in real time based on the detection data of the machine vision sensor, laser displacement sensor and capacitive sensor, and sends the breakage rate to the display screen for real-time display.

[0063] 6) If the combined grain breakage rate exceeds a preset threshold, the system automatically generates a real-time control strategy to adjust the harvester’s performance parameters (including vehicle speed, engine speed, and drum speed) to reduce the grain breakage rate. After the adjustment, the system continues to monitor the combined grain breakage rate and forms a closed-loop control based on the feedback results.

[0064] Figure 3 The figure shows a flow chart of an embodiment of a method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion provided by the present invention. Figure 3 As shown, the following steps are included: S1, respectively collecting image information representing the damage of the surface layer of the grain, laser reflection signals representing the microcracks on the surface of the grain, and capacitance value change signals representing the damage of the internal structure of the grain; S2. Using a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate a comprehensive grain breakage rate; S3. When the comprehensive grain breakage rate is greater than a preset threshold, the performance parameters of the harvester are adjusted to reduce the grain breakage rate, and closed-loop control is performed based on the adjusted comprehensive breakage rate.

[0065] In an optional manner, the step of using a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate a comprehensive grain breakage rate includes: Performing median filtering on the image information and extracting the image damage rate based on edge detection and texture recognition algorithms; Performing time domain signal analysis processing on the laser reflection signal to obtain a light intensity change rate, and obtaining a microcrack breakage rate based on the light intensity change rate; Filtering and feature extraction processing are performed on the capacitance value change signal to obtain an internal structure damage rate; assigning corresponding weight coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The image damage rate, the microcrack damage rate and the internal structure damage rate are weightedly calculated according to each weight coefficient to obtain the comprehensive damage rate of the grain.

[0066] In an optional embodiment, the performance parameters include: a vehicle speed value, an engine speed value, and a drum speed value of the harvester; when the comprehensive grain breakage rate is greater than a preset threshold, the step of reducing the grain breakage rate by adjusting the performance parameters of the harvester includes: When the comprehensive grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of a vehicle speed reduction instruction, an engine speed reduction instruction, and a drum speed reduction instruction; The execution mechanism corresponding to the real-time control strategy is controlled to adjust at least one parameter of the vehicle speed value, engine speed value and drum speed value of the harvester to reduce the grain breakage rate of the harvester.

[0067] In an optional manner, the steps of respectively collecting image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain include: Using a machine vision sensor to collect the image information representing the damage to the surface of the grain; Using a laser displacement sensor, collecting the laser reflection signal representing the microcracks on the surface of the grain; A capacitance sensor is used to collect the capacitance value change signal indicating damage to the internal structure of the grain.

[0068] It should be noted that the beneficial effects of the method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion provided by the above embodiment are the same as the beneficial effects of the system 100 for detecting and adjusting grain damage in a harvester based on multi-sensor fusion, which will not be described in detail here. In addition, when the system provided by the above embodiment realizes its functions, it only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the method and system embodiments provided by the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment, which will not be described in detail here.

[0069] Among them, the harvester grain damage detection and adjustment system 100 based on multi-sensor fusion of the present invention can be a computer program (including program code) running in a computer device. For example, the harvester grain damage detection and adjustment system 100 based on multi-sensor fusion of the present invention is an application software that can be used to execute the corresponding steps in the harvester grain damage detection and adjustment method based on multi-sensor fusion of the present invention.

[0070] In some embodiments, the harvester grain damage detection and adjustment system 100 based on multi-sensor fusion of the present invention can be implemented by a combination of software and hardware. As an example, the harvester grain damage detection and adjustment system 100 based on multi-sensor fusion of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the harvester grain damage detection and adjustment method based on multi-sensor fusion of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0071] The modules described in the embodiments of the present invention may be implemented in software or hardware, and the name of a module does not necessarily limit the module itself.

[0072] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the above-mentioned methods for detecting and adjusting grain damage in a harvester based on multi-sensor fusion is implemented. That is, an electronic device according to an embodiment of the present invention may include but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion as shown in any embodiment of the present invention by calling the computer program.

[0073] In an alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0074] Processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 4001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0075] Bus 4002 may include a path for transmitting information between the above components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 In the figure, only one thick line is used to represent the bus 4002, but this does not mean that there is only one bus or one type of bus.

[0076] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0077] The memory 4003 is used to store application code (computer program) for executing the solution of the present invention, and is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0078] Among them, the electronic device can also be a terminal device, and the terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart TV, and smart car-mounted device.

[0079] It should be noted that Figure 4 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0080] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements any of the above-mentioned harvester grain damage detection and adjustment methods based on multi-sensor fusion.

[0081] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0082] In an exemplary embodiment, a computer program product or computer program is also provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the above-described harvester grain damage detection and adjustment method based on multi-sensor fusion.

[0083] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0085] The computer-readable storage medium provided in the embodiments of the present invention may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0086] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0087] The above description is merely an illustration of preferred embodiments of the present invention and the underlying technical principles. Those skilled in the art should understand that the scope of the present invention is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned concepts. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this invention.

[0088] It should be noted that the terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects and to define a specific order or precedence. Where appropriate, the order used for similar objects may be interchanged, such that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0089] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present invention may be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may be implemented as a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0090] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A harvester grain damage detection and adjustment system based on multi-sensor fusion, characterized in that: include: Multi-sensor acquisition module, data fusion processing module and adjustment control module; The multi-sensor acquisition module is used to respectively acquire image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain; The data fusion processing module is used to: use a weighted fusion algorithm to comprehensively analyze the image information, the laser reflection signal and the capacitance value change signal to generate a comprehensive grain breakage rate; The adjustment control module is used to: when the comprehensive grain breakage rate is greater than a preset threshold, adjust the performance parameters of the harvester to reduce the grain breakage rate, and perform closed-loop control according to the adjusted comprehensive breakage rate.

2. The harvester grain damage detection and adjustment system based on multi-sensor fusion according to claim 1 is characterized in that: The data fusion processing module is specifically used for: Performing median filtering on the image information and extracting the image damage rate based on edge detection and texture recognition algorithms; Performing time domain signal analysis processing on the laser reflection signal to obtain a light intensity change rate, and obtaining a microcrack breakage rate based on the light intensity change rate; Filtering and feature extraction processing are performed on the capacitance value change signal to obtain an internal structure damage rate; assigning corresponding weight coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The image damage rate, the microcrack damage rate and the internal structure damage rate are weightedly calculated according to each weight coefficient to obtain the comprehensive damage rate of the grain.

3. The harvester grain damage detection and adjustment system based on multi-sensor fusion according to claim 1 is characterized in that: The performance parameters include: the harvester's vehicle speed, engine speed, and drum speed; the adjustment control module is specifically used to: When the comprehensive grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of a vehicle speed reduction instruction, an engine speed reduction instruction, and a drum speed reduction instruction; The execution mechanism corresponding to the real-time control strategy is controlled to adjust at least one parameter of the vehicle speed value, engine speed value and drum speed value of the harvester to reduce the grain breakage rate of the harvester.

4. The harvester grain damage detection and adjustment system based on multi-sensor fusion according to any one of claims 1 to 3, characterized in that: The multi-sensor acquisition module is specifically used for: Using a machine vision sensor to collect the image information representing the damage to the surface of the grain; Using a laser displacement sensor, collecting the laser reflection signal representing the microcracks on the surface of the grain; A capacitance sensor is used to collect the capacitance value change signal indicating damage to the internal structure of the grain.

5. A harvester grain damage detection and adjustment method based on multi-sensor fusion, characterized in that: include: The image information representing the damage of the grain surface, the laser reflection signal representing the micro-cracks on the grain surface and the capacitance value change signal representing the damage of the grain internal structure are collected respectively; A weighted fusion algorithm is used to comprehensively analyze the image information, the laser reflection signal, and the capacitance value change signal to generate a comprehensive grain breakage rate; When the comprehensive grain breakage rate is greater than a preset threshold, the performance parameters of the harvester are adjusted to reduce the grain breakage rate, and closed-loop control is performed based on the adjusted comprehensive breakage rate.

6. The method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion according to claim 5, characterized in that: The step of comprehensively analyzing the image information, the laser reflection signal, and the capacitance value change signal using a weighted fusion algorithm to generate a comprehensive grain breakage rate includes: Performing median filtering on the image information and extracting the image damage rate based on edge detection and texture recognition algorithms; Performing time domain signal analysis processing on the laser reflection signal to obtain a light intensity change rate, and obtaining a microcrack breakage rate based on the light intensity change rate; Filtering and feature extraction processing are performed on the capacitance value change signal to obtain an internal structure damage rate; assigning corresponding weight coefficients to the image damage rate, the microcrack damage rate, and the internal structure damage rate; The image damage rate, the microcrack damage rate and the internal structure damage rate are weightedly calculated according to each weight coefficient to obtain the comprehensive damage rate of the grain.

7. The method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion according to claim 5, characterized in that: The performance parameters include: a vehicle speed value, an engine speed value, and a drum speed value of the harvester; when the comprehensive grain breakage rate is greater than a preset threshold, the step of reducing the grain breakage rate by adjusting the performance parameters of the harvester includes: When the comprehensive grain breakage rate is greater than a preset threshold, a real-time control strategy for the harvester is generated; the real-time control strategy includes at least one of a vehicle speed reduction instruction, an engine speed reduction instruction, and a drum speed reduction instruction; The execution mechanism corresponding to the real-time control strategy is controlled to adjust at least one parameter of the vehicle speed value, engine speed value and drum speed value of the harvester to reduce the grain breakage rate of the harvester.

8. The method for detecting and adjusting grain damage in a harvester based on multi-sensor fusion according to any one of claims 5 to 7, characterized in that: The steps of respectively collecting image information representing damage to the surface of the grain, laser reflection signals representing microcracks on the surface of the grain, and capacitance value change signals representing damage to the internal structure of the grain include: Using a machine vision sensor to collect the image information representing the damage to the surface of the grain; Using a laser displacement sensor, collecting the laser reflection signal representing the microcracks on the surface of the grain; A capacitance sensor is used to collect the capacitance value change signal indicating damage to the internal structure of the grain.

9. An electronic device, characterized in that: The electronic device includes a processor, which is coupled to a memory. The memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device can implement the harvester grain damage detection and adjustment method based on multi-sensor fusion as described in any one of claims 5 to 8.

10. A computer-readable storage medium, characterized in that At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by the processor so that the computer-readable storage medium implements the harvester grain damage detection and adjustment method based on multi-sensor fusion as described in any one of claims 5 to 8.

Images