Image processing method and device of picking equipment
By using adaptive illumination estimation and a global-local cooperative image enhancement model, the problem of image processing for agricultural harvesting equipment under non-uniform lighting conditions was solved, achieving efficient fruit recognition and localization and improving the working efficiency of the harvesting equipment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-03-17
AI Technical Summary
Existing agricultural harvesting equipment struggles to balance local details and overall contrast in non-uniform lighting conditions, resulting in insufficient accuracy in fruit identification and positioning, which affects harvesting efficiency.
An adaptive regularization factor illumination estimation model based on edge pixel ratio is adopted, combined with a local illumination enhancement model and a global contrast optimization model. Image processing is optimized through a multi-objective balance function to achieve global-local collaborative enhancement.
With low computing resources and without the need for deep learning models, adaptive optimization of image degradation improves the fruit positioning and grasping accuracy of harvesting equipment in low-light and occluded scenarios, thereby increasing harvesting efficiency.
Smart Images
Figure CN121685352A_ABST
Abstract
Description
Technical Field
[0001] The technical field of this disclosure, more specifically, relates to an image processing method and apparatus for a harvesting device. Background Technology
[0002] With the rapid development of agricultural harvesting equipment, image enhancement technology can be used to identify and locate fruits in complex environments. Currently, existing image enhancement methods for agricultural harvesting equipment mainly rely on histogram enhancement, Retinex transform, or deep learning enhancement models. Specifically, while histogram equalization is computationally simple, it uniformly adjusts global illumination changes, which can lead to local overexposure or underexposure in greenhouse environments, and it also improperly handles shading areas at the junction of tomatoes and leaves. Retinex-based methods can separate illumination and reflection components, but their iterative computational complexity is high (processing time for a single image typically exceeds 200ms), and they are highly sensitive to parameters, making it difficult to adapt to the dynamic lighting conditions of mixed natural and artificial light in greenhouses. While deep learning enhancement models have better performance, they require a large amount of labeled data for training and have long inference times (typically 80-150ms / frame), making it difficult to meet the needs of real-time harvesting. In addition, existing methods generally lack optimization mechanisms for the specific color combination of red fruit and green leaves, which can easily cause color saturation distortion while enhancing brightness, affecting the judgment of ripeness. Therefore, existing technologies have limited enhancement capabilities in non-uniform lighting scenarios, making it difficult to balance local details with overall contrast, and thus failing to meet the positioning and grasping requirements of agricultural harvesting equipment, thereby affecting the working efficiency of the harvesting equipment. Summary of the Invention
[0003] One objective of this disclosure is to provide a new technical solution for image processing of harvesting equipment.
[0004] According to a first aspect of this disclosure, an image processing method for a harvesting device is provided, the method comprising: The image acquisition device associated with the harvesting equipment is used to acquire a sampled image of the target area; The sampled image is input into a preset illumination estimation model to obtain an initial illumination map; The initial illumination map is input into a preset local illumination enhancement model to obtain an enhanced brightness map; The initial illumination map and the enhanced brightness map are input into a preset global contrast optimization model to obtain a global contrast image; Based on the global comparison image, the operating parameters of the harvesting equipment are set; The illumination estimation model employs an adaptive regularization factor based on the edge pixel ratio, the local illumination enhancement model introduces saturation statistical characteristics to control the reflection term, and the global contrast optimization model establishes a multi-objective balance function of information entropy, structural similarity, and edge loss.
[0005] Optionally, before inputting the sampled image into a preset illumination estimation model to obtain an initial illumination map, the method further includes: Determine the pixel intensity, maximum pixel value, and minimum pixel value of the sampled image in the color channel; Based on the pixel intensity, the maximum pixel value, and the minimum pixel value, the color channel is linearly normalized to obtain a normalized sampled image.
[0006] Optionally, inputting the sampled image into a preset illumination estimation model to obtain an initial illumination map includes: The sampled image is estimated by multi-scale weighted guided filtering based on the edge pixel ratio using a preset illumination estimation model to obtain the first filtered image. If the first filtering result does not meet the set image accuracy condition, the first filtered image is used as input, and the step of performing multi-scale weighted guided filtering estimation on the sampled image based on the edge pixel ratio is executed cyclically until the output second filtered image meets the image accuracy condition, and the second filtered image is used as the initial illumination image.
[0007] Optionally, the illumination estimation model includes: in, Let I(x,y) represent the initial illumination map, and I(x,y) represent the sampled image. (x,y) represents the expression at the nth scale. (x,y) represents the output of the standard guided filter for the guided graph. Represents structure-aware weights. (x,y) represents the normalized sampled image.
[0008] Optionally, the illumination estimation model further includes: in, It is the first k Each scale corresponds to a filtering window. k It is the number of pixels in the window. and These are the mean and variance of the regional illumination, respectively. It is a regularization factor. It is the ratio of edge pixels. It is the initial regularization factor.
[0009] Optionally, the local illumination enhancement model includes: in, T ( x,y The brightness and reflectance diagram is derived from the initial illumination diagram. The brightness enhancement adjustment coefficient, The value range is [0, 1].
[0010] Optionally, after inputting the initial illumination map into a preset local illumination enhancement model to obtain an enhanced brightness map, the method further includes: The enhanced brightness map is input into a preset feature extraction model to obtain the first matrix feature and the second matrix feature; The first matrix features and the second matrix features are input into a preset feature fusion model to obtain the fused enhanced brightness map; The feature extraction model includes: in, Indicates the first i The average gray level of the enhanced brightness map Indicates the first j The average gray level of the enhanced brightness map n To increase the total number of pixels in the luminance map, Indicates the first i The first enhanced brightness map k The brightness value of each pixel. Indicates the first j The first enhanced brightness map k The brightness value of each pixel; The variance matrix is represented by a first matrix feature extracted from the variance matrix constructed from one of the enhanced brightness maps, and a second matrix feature extracted from the variance matrix constructed from another enhanced brightness map. The feature extraction model includes: in, ( x,y () represents the enhanced brightness map after fusion. ( x,y ) represents the feature of the first matrix. ( x,y ) represents the feature of the second matrix. and The weighting coefficients and .
[0011] Optionally, the global comparison optimization model includes: in, K It is the initial illumination map and the enhanced brightness map ( x,y The corresponding contrast factor is: Entropy represents image information entropy, SSIM represents the structural similarity index, and EdgeLoss represents the edge loss term. , and The weighting coefficients and ; This represents the optimal contrast factor.
[0012] Optionally, setting the operating parameters of the harvesting device based on the global comparison image includes: Determine the key indicator values of the global comparison image; If the value of the key indicator does not meet the set indicator conditions, at least one of the illumination estimation model, the local illumination enhancement model, and the global contrast optimization model shall be adjusted based on the value of the key indicator. The sampled image is then used as input again and fed into the adjusted illumination estimation model, local illumination enhancement model, and global contrast optimization model until the final output global contrast image meets the specified index conditions. Based on the final global comparison image, the visual positioning parameters and grasping parameters of the harvesting device are set.
[0013] According to a second aspect of this disclosure, an image processing apparatus for a harvesting device is also provided, the apparatus comprising: The acquisition module is used to acquire a sampled image of the target area through an image acquisition device associated with the harvesting equipment; The first obtaining module is used to input the sampled image into a preset illumination estimation model to obtain an initial illumination map; The second obtaining module is used to input the initial illumination map into a preset local illumination enhancement model to obtain an enhanced brightness map; The third module is used to input the initial illumination map and the enhanced brightness map into a preset global contrast optimization model to obtain a global contrast image; The setting module is used to set the operating parameters of the harvesting equipment based on the global comparison image; wherein, the illumination estimation model adopts an adaptive regularization factor based on the edge pixel ratio, the local illumination enhancement model introduces saturation statistical characteristics to control the reflection term, and the global comparison optimization model establishes a multi-objective balance function of information entropy, structural similarity and edge loss.
[0014] According to a third aspect of this disclosure, a computer system is also provided, the computer system including a processor, which, when the processor executes program instructions or code, implements the image processing method of the picking device in the first aspect.
[0015] For example, the computer system also includes a memory for storing program instructions or code.
[0016] According to a fourth aspect of this disclosure, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to execute the image processing method of the harvesting device described above when it is run.
[0017] According to a fifth aspect of this disclosure, a computer program product is also provided, comprising a computer program that, when executed, causes a computer to perform the image processing method steps of the harvesting device described above.
[0018] According to a sixth aspect of this disclosure, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the image processing method of the harvesting device through the computer program.
[0019] One beneficial effect of this disclosure is that the image processing method for harvesting equipment provided by the present invention can achieve a global-local collaborative image enhancement mechanism by setting up an illumination estimation model, a local illumination enhancement model, and a global contrast optimization model. This achieves adaptive optimization of image degradation problems in low-light and occlusion scenarios during tomato harvesting operations without relying on high computing resources or deep learning models, effectively meeting the positioning and grasping requirements of the harvesting equipment and significantly improving its working efficiency.
[0020] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the embodiments of the present disclosure.
[0022] Figure 1A flowchart illustrating an image processing method for a harvesting device according to some embodiments is shown; Figure 2 A schematic diagram of the structure of an image processing apparatus for a harvesting device according to some embodiments is shown; Figure 3 A schematic diagram of the hardware structure of an electronic device according to some embodiments is shown. Detailed Implementation
[0023] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the invention.
[0024] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.
[0025] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0026] In all the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.
[0027] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0028] <Method Implementation> Figure 1 This is a flowchart illustrating an image processing method for a harvesting device according to one embodiment. The implementing entity is a smart terminal, such as a personal computer, mobile phone, tablet, or server.
[0029] like Figure 1 As shown, the image processing method of the harvesting device in this embodiment may include the following steps S110 to S150: Step S110: Obtain a sampled image of the target area through the image acquisition device associated with the harvesting equipment.
[0030] In some embodiments, prior to step S110, the method further includes the following steps S210 and S220: Step S210: Determine the pixel intensity, maximum pixel value, and minimum pixel value of the sampled image in the color channel.
[0031] Step S220: Based on pixel intensity, maximum pixel value and minimum pixel value, linear normalization is performed on the color channel to obtain a normalized sampled image.
[0032] In this embodiment, the harvesting device can be positioned in front of any plant, with its camera fixed at a certain distance, such as 0.4m-0.8m, to ensure coverage of the entire plant. The original sampled image, i.e., the RGB image Iraw(x,y)=[R,G,B], is acquired at a resolution of 1280×740 pixels. Linear normalization is then performed on the three channels to eliminate gain differences between different bands, generating a normalized sampled image. I norm ( x , y ), for the follow-up HSI The transformation establishes a stable input. The expression for linear normalization is as follows: in, It is a normalized sampled image I norm ( x , y The normalized channel values of ) It is a color channel C (Red Channel) R Green Channel G and the blue channel B Pixels on ) x , y The pixel intensity of ) and, and These are the color channels in the sampled image. C The minimum and maximum pixel values.
[0033] Step S120: Input the sampled image into the preset illumination estimation model to obtain the initial illumination map.
[0034] In some embodiments, step S120 may include the following steps S310 and S320: Step S310: Using a preset illumination estimation model, the sampled image is subjected to multi-scale weighted guided filtering estimation based on the edge pixel ratio to obtain the first filtered image. The illumination estimation model employs an adaptive regularization factor based on the edge pixel ratio.
[0035] Step S320: If the first filtering result does not meet the set image accuracy conditions, the first filtered image is used as input, and the step of performing multi-scale weighted guided filtering estimation on the sampled image based on the edge pixel ratio is executed cyclically until the output second filtered image meets the image accuracy conditions, and the second filtered image is used as the initial illumination image.
[0036] In some embodiments, the illumination estimation model includes: in, Let I(x,y) represent the initial illumination map, and I(x,y) represent the sampled image. (x,y) represents the expression at the nth scale. (x,y) represents the output of the standard guided filter for the guided graph. Represents structure-aware weights. (x,y) represents the normalized sampled image.
[0037] In this embodiment, the normalized sampled image I norm ( x , y ) Transfer to HSI Space, then in J Normalized sampled images at each scale I norm ( x , y The initial illumination map is estimated using a multi-scale weighted guided filter based on the edge pixel ratio. The filtering process can be iterated a maximum of 5 times, and the following image accuracy conditions must be met: reprojection error < 0.5 pixels, rotation estimation accuracy ± 0.1°, and ≥ 50 sets of valid pixels must be retained. The process can then terminate and output the final initial illumination map. This is to ensure that the output initial lighting map reflects the lighting intensity more accurately.
[0038] In some embodiments, the illumination estimation model further includes: in, It is the first k Each scale corresponds to a filtering window. k It is the number of pixels in the window. and These are the mean and variance of the regional illumination, respectively. It is a regularization factor. It is the ratio of edge pixels. It is the initial regularization factor.
[0039] In this embodiment, the weights of each scale are adaptively controlled, and the structural intensity in the sampled image is measured by comparing illumination and local statistics, so as to achieve adaptive fusion of structural information at each scale in the final output initial illumination map.
[0040] In this embodiment, the adaptive regularization factor It is based on the proportion of edge pixels within the filter window. Dynamic adjustments are made. The initial regularization factor can be 0.01, adaptively adjusted in regions rich in edges (e.g., This ensures enhanced structural sensitivity in complex occlusion regions and effectively improves the SSIM score compared to fixed regularization, thereby further enhancing the filter's response at structural edges.
[0041] Step S130: Input the initial illumination map into the preset local illumination enhancement model to obtain the enhanced brightness map. The local illumination enhancement model introduces saturation statistical characteristics to control the reflection term.
[0042] In some embodiments, the local illumination enhancement model includes: Where 255 is the upper limit of grayscale for an 8-bit image. T ( x,y The brightness reflection diagram is derived from the initial illumination diagram and is used to adjust the nonlinear intensity of the enhancement function. The brightness enhancement adjustment coefficient, The value range is [0, 1].
[0043] In this embodiment, the local illumination enhancement model can be the Weber-Fechner model that incorporates statistical properties of saturation, i.e., the initial illumination map. Mapped to enhanced brightness map L en ( x , y ).
[0044] In this embodiment, two sets of enhancement coefficients are set. (Moderate degeneration) and (Severe degradation) can generate an enhanced brightness map. L en1 ( x , y ) and enhanced brightness map L en2 ( x , y ).coefficient The selection is based on the joint optimization of the information entropy-enhancement coefficient and SSIM-enhancement coefficient curves: when the information entropy > 7.0 and SSIM > 0.7, the enhancement is deemed effective, and the processing time for a single image is approximately 10ms.
[0045] In this embodiment, a brightness adjustment mechanism based on saturation statistical characteristics is introduced to adjust the overall average saturation of the initial illumination map. Control of reflection terms T (x , y To make the initial lighting pattern Mapped to enhanced brightness map L en ( x , y The process is more adaptable.
[0046] In some embodiments, after step S130, the method further includes the following steps S410 and S420: Step S410: Input the enhanced brightness map into the preset feature extraction model to obtain the first matrix feature and the second matrix feature.
[0047] Step S420: Input the first matrix features and the second matrix features into the preset feature fusion model to obtain the fused enhanced brightness map.
[0048] The feature extraction models include: in, Indicates the first i The average gray level of the enhanced brightness map Indicates the first j The average gray level of the enhanced brightness map n To increase the total number of pixels in the luminance map, Indicates the first i The first enhanced brightness map k The brightness value of each pixel. Indicates the first j The first enhanced brightness map k The brightness value of each pixel; The variance matrix is represented by the first matrix feature extracted from the variance matrix constructed from one enhanced brightness map, and the second matrix feature extracted from the variance matrix constructed from another enhanced brightness map. The feature extraction models include: in, ( x,y () represents the enhanced brightness map after fusion. ( x,y ) represents the characteristic of the first matrix. ( x, y ) represents the characteristic of the second matrix. and The weighting coefficients and .
[0049] In this embodiment, the enhanced brightness map is obtained through principal component analysis. L en1 ( x ,y )and L en2 ( x , y Vectorization to construct pixel-level covariance matrix The covariance matrix is subjected to eigenvalue decomposition to extract eigenvalues and eigenvectors. The principal eigenvector corresponding to the largest eigenvalue is selected as the fusion weight, and finally fused into a fused enhanced brightness map. (x,y).
[0050] In some examples, after step S420, the method further includes step S430: step S430, restoring the fused enhanced luminance map from the HSI color space to the RGB color space.
[0051] In this embodiment, by setting a feature extraction model and a feature efflorescence model, the image's main direction of change is followed, and the fused output enhanced brightness map not only inherits the high brightness enhancement effect but also effectively reduces the occurrence of excessive information compression and artifact accumulation.
[0052] Step S140: Input the initial illumination map and the enhanced brightness map into a preset global contrast optimization model to obtain a global contrast image. The global contrast optimization model establishes a multi-objective balance function of information entropy, structural similarity, and edge loss.
[0053] In some embodiments, the global comparison optimization model includes: in, K It is the initial lighting diagram. and enhanced brightness map ( x,y The corresponding contrast factor is: Entropy represents image information entropy, which measures the amount of information in the image. SSIM represents the structural similarity index, which measures the structural fidelity between the enhanced image and the original image. EdgeLoss represents the edge loss term, which suppresses artifacts and blurring caused by over-enhancement. , and The weighting coefficients and ; This represents the optimal contrast factor.
[0054] In this embodiment, the optimal contrast factor It can be obtained by searching based on the improved bat algorithm (population size 20-50, iterations 50-100, convergence threshold 0.01, initial comparison adjustment factor range 0.5-2.0; sinusoidal perturbation is introduced in position update).
[0055] Step S150: Based on the global comparison image, set the operating parameters of the harvesting equipment.
[0056] In some embodiments, step S150 may include the following steps S510 to S540: Step S510: Determine the key indicator values of the global comparison image.
[0057] Step S520: If the value of the key indicator does not meet the set indicator conditions, adjust at least one of the following models based on the value of the key indicator: illumination estimation model, local illumination enhancement model, and global contrast optimization model.
[0058] Step S530: The sampled image is re-inputted into the adjusted illumination estimation model, local illumination enhancement model, and global contrast optimization model until the final output global contrast image meets the index conditions.
[0059] Step S540: Using the final global comparison image, set the visual positioning parameters and grasping parameters of the harvesting device.
[0060] In this embodiment, global comparison images are used. Calculate key metric values, such as information entropy, SSIM, and EdgeLoss. If any key metric fails to reach a preset threshold (Entropy > 7.5; SSIM > 0.8; EdgeLoss < 0.1), meaning it does not meet the set metric conditions, the algorithm automatically backtracks to fine-tuning the regularization factor or the number of iterations until the maximum number of retries is met or reached. Finally, it displays the global comparison image. The data is fed into the visual positioning model and grasping module of the harvesting equipment. Under an Intel i7-8700 CPU environment, the end-to-end average inference latency is approximately 47.9ms / frame, meeting the requirements for real-time harvesting at the second level.
[0061] <Equipment Example 1> Figure 2 This is a schematic block diagram of an image processing device for a harvesting apparatus according to one embodiment. (e.g.) Figure 2 As shown, the image processing device 200 of the harvesting equipment may include: The acquisition module 210 is used to acquire a sampled image of the target area through an image acquisition device associated with the harvesting equipment; The first obtaining module 220 is used to input the sampled image into a preset illumination estimation model to obtain an initial illumination map; The second module 230 is used to input the initial illumination map into a preset local illumination enhancement model to obtain an enhanced brightness map; The third module 240 is used to input the initial illumination map and the enhanced brightness map into a preset global contrast optimization model to obtain a global contrast image; The setting module 250 is used to set the operating parameters of the harvesting equipment based on the global contrast image. Among them, the illumination estimation model adopts an adaptive regularization factor based on the edge pixel ratio, the local illumination enhancement model introduces saturation statistical characteristics to control the reflection term, and the global contrast optimization model establishes a multi-objective balance function of information entropy, structural similarity and edge loss.
[0062] Optionally, the image processing device 200 of the harvesting equipment further includes a normalization module for determining the pixel intensity, maximum pixel value, and minimum pixel value of the sampled image in the color channel; and performing linear normalization on the color channel based on the pixel intensity, maximum pixel value, and minimum pixel value to obtain a normalized sampled image.
[0063] Optionally, the first obtaining module 220 is further configured to perform multi-scale weighted guided filtering estimation on the sampled image based on the edge pixel ratio using a preset illumination estimation model to obtain a first filtered image; if the first filtering result does not meet the set image accuracy conditions, the first filtered image is used as input to repeatedly execute the step of performing multi-scale weighted guided filtering estimation on the sampled image based on the edge pixel ratio until the output second filtered image meets the image accuracy conditions, and the second filtered image is used as the initial illumination image.
[0064] Optionally, the image processing device 200 of the harvesting equipment further includes a fusion module, used to input the enhanced brightness map into a preset feature extraction model to obtain a first matrix feature and a second matrix feature; and input the first matrix feature and the second matrix feature into a preset feature fusion model to obtain a fused enhanced brightness map; Optionally, the setting module 250 is also used to determine the key index values of the global comparison image; if the key index values do not meet the set index conditions, adjust at least one of the illumination estimation model, local illumination enhancement model, and global contrast optimization model based on the key index values; re-input the sampled image as input to the adjusted illumination estimation model, local illumination enhancement model, and global contrast optimization model until the final output global comparison image meets the index conditions; and set the visual positioning parameters and grasping parameters of the picking device based on the final global comparison image.
[0065] <Equipment Example 2> Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to another embodiment.
[0066] like Figure 3As shown, the electronic device 300 includes a processor 310 and a memory 320, the memory 320 being used to store an executable computer program, and the processor 310 being used to execute methods as described in any of the above method embodiments under the control of the computer program.
[0067] Each module of the image processing device 200200 of the above-mentioned harvesting equipment can be implemented by the processor 310 in this embodiment executing the computer program stored in the memory 320, or it can be implemented by other structures, which are not limited here.
[0068] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.
[0069] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
[0070] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.
[0071] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.
[0072] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0073] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0074] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0075] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.
[0076] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.
Claims
1. An image processing method of a picking apparatus, characterized by, The method includes: The image acquisition device associated with the harvesting equipment is used to acquire a sampled image of the target area; The sampled image is input into a preset illumination estimation model to obtain an initial illumination map; The initial illumination map is input into a preset local illumination enhancement model to obtain an enhanced brightness map; The initial illumination map and the enhanced brightness map are input into a preset global contrast optimization model to obtain a global contrast image; Based on the global comparison image, the operating parameters of the harvesting equipment are set; The illumination estimation model employs an adaptive regularization factor based on the edge pixel ratio, the local illumination enhancement model introduces saturation statistical characteristics to control the reflection term, and the global contrast optimization model establishes a multi-objective balance function of information entropy, structural similarity, and edge loss.
2. The method of claim 1, wherein, Before inputting the sampled image into a preset illumination estimation model to obtain an initial illumination map, the method further includes: Determine the pixel intensity, maximum pixel value, and minimum pixel value of the sampled image in the color channel; Based on the pixel intensity, the maximum pixel value, and the minimum pixel value, the color channel is linearly normalized to obtain a normalized sampled image.
3. The method of claim 1, wherein, The step of inputting the sampled image into a preset illumination estimation model to obtain an initial illumination map includes: The sampled image is estimated by multi-scale weighted guided filtering based on the edge pixel ratio using a preset illumination estimation model to obtain the first filtered image. If the first filtering result does not meet the set image accuracy condition, the first filtered image is used as input, and the step of performing multi-scale weighted guided filtering estimation on the sampled image based on the edge pixel ratio is executed cyclically until the output second filtered image meets the image accuracy condition, and the second filtered image is used as the initial illumination image.
4. The method of claim 3, wherein, The illumination estimation model includes: wherein, represents an initial light map, I ( x,y ) represents a sampled image, ( x,y ) represents an output of a standard guided filter with ( x,y ) as a guide image, represents a structure-aware weight, ( x,y ) represents a normalized sampled image.
5. The method of claim 1, wherein, The illumination estimation model also includes: wherein, is the filter window corresponding to the k th scale, k is the number of pixels in the window, and are the mean and variance of the regional illumination, respectively, is the regularization factor, is the proportion of edge pixels, is the initial regularization factor.
6. The method of claim 1, wherein, The local illumination enhancement model includes: wherein, T x,y is a luminance reflectance map derived from the initial light map, is a luminance enhancement adjustment coefficient, has a value in the range [0, 1]. 7. The method according to claim 1, characterized in that, After inputting the initial illumination map into a preset local illumination enhancement model to obtain the enhanced brightness map, the method further includes: The enhanced brightness map is input into a preset feature extraction model to obtain the first matrix feature and the second matrix feature; The first matrix features and the second matrix features are input into a preset feature fusion model to obtain the fused enhanced brightness map; The feature extraction model includes: in, Indicates the first i The average gray level of the enhanced brightness map Indicates the first j The average gray level of the enhanced brightness map n To increase the total number of pixels in the luminance map, Indicates the first i The first enhanced brightness map k The brightness value of each pixel. Indicates the first j The first enhanced brightness map k The brightness value of each pixel; The variance matrix is represented by a first matrix feature extracted from the variance matrix constructed from one of the enhanced brightness maps, and a second matrix feature extracted from the variance matrix constructed from another enhanced brightness map. The feature extraction model includes: in, ( x,y () represents the enhanced brightness map after fusion. ( x,y ) represents the feature of the first matrix. ( x,y ) represents the feature of the second matrix. and The weighting coefficients and .
8. The method according to claim 1, characterized in that, The global comparison optimization model includes: in, K It is the initial illumination map and the enhanced brightness map ( x,y The corresponding contrast factor is: Entropy represents image information entropy, SSIM represents the structural similarity index, and EdgeLoss represents the edge loss term. , and The weighting coefficients and ; This represents the optimal contrast factor.
9. The method according to claim 1, characterized in that, The step of setting the operating parameters of the harvesting equipment based on the global comparison image includes: Determine the key indicator values of the global comparison image; If the value of the key indicator does not meet the set indicator conditions, at least one of the illumination estimation model, the local illumination enhancement model, and the global contrast optimization model shall be adjusted based on the value of the key indicator. The sampled image is then used as input again and fed into the adjusted illumination estimation model, local illumination enhancement model, and global contrast optimization model until the final output global contrast image meets the specified index conditions. Based on the final global comparison image, the visual positioning parameters and grasping parameters of the harvesting device are set.
10. An image processing device for a harvesting equipment, characterized in that, The device includes: The acquisition module is used to acquire a sampled image of the target area through an image acquisition device associated with the harvesting equipment; The first obtaining module is used to input the sampled image into a preset illumination estimation model to obtain an initial illumination map; The second obtaining module is used to input the initial illumination map into a preset local illumination enhancement model to obtain an enhanced brightness map; The third module is used to input the initial illumination map and the enhanced brightness map into a preset global contrast optimization model to obtain a global contrast image; The setting module is used to set the operating parameters of the harvesting equipment based on the global comparison image; wherein, the illumination estimation model adopts an adaptive regularization factor based on the edge pixel ratio, the local illumination enhancement model introduces saturation statistical characteristics to control the reflection term, and the global comparison optimization model establishes a multi-objective balance function of information entropy, structural similarity and edge loss.