Image splicing coefficient determination method and image splicing method based on line-scan digital camera

Abstract

The invention relates to the technical field of image processing, and provides an image splicing coefficient determination method and an image splicing method based on a line-scan digital camera, and the image splicing coefficient determination method comprises the steps: determining a basic channel for image splicing from a plurality of single channels obtained from the line-scan digital camera in advance; using the splicing coefficient of the single channel adjacent to the basic channel as a unit splicing coefficient, and splicing all other single channels to the basic channel to obtain a corresponding spliced image; calculating a correlation coefficient of a basic channel and at least one other single channel in the spliced image; and obtaining multiple groups of unit splicing coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, performing polynomial fitting based on the multiple groups of unit splicing coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, and determining an optimal unit splicing coefficient based on a polynomial fitting result.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for determining an image stitching coefficient and an image stitching method based on a line array camera. Background Art

[0002] When the color sorter sorts materials on the production line, the line array camera will take pictures of the materials and identify them, and then further sort the materials. For example, a three-channel color line array camera has three channels, R, G, and B. The images of the three channels need to be stitched together to obtain a clear image for identification and sorting. Due to the large difference in density of different materials, the speeds when passing through the color sorter will be different. Therefore, when sorting different materials, if the same stitching coefficient is used for image stitching, the captured images will be distorted, which will affect the recognition accuracy of the color sorter. Therefore, when changing materials or debugging the color sorter online, the image stitching coefficient needs to be adjusted according to the adaptability of the product.

[0003] Normally, in order to solve the problem of image distortion, technicians need to adjust the stitching coefficients of different materials to obtain clear images. However, manual adjustment is not only difficult to operate, but also has low adjustment efficiency, which limits the automation and intelligence level of the production line and affects the sorting efficiency of the color sorter. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first purpose of the present invention is to propose a method for determining an image stitching coefficient, so as to efficiently determine the image stitching coefficient of a linear array camera and get rid of the dependence on the experience of the staff; optimize the unit stitching coefficient of a single channel of the linear array camera, and stitch images according to the optimized unit stitching coefficient to obtain an image with higher resolution, thereby effectively improving the accuracy of the color sorter in identifying materials, and improving the accuracy and efficiency of the color sorter in sorting.

[0005] The second objective of the present invention is to provide an image stitching method based on a line array camera.

[0006] To achieve the above object, a first embodiment of the present invention provides a method for determining image splicing coefficients, comprising:

[0007] Determine a basic channel for image stitching from a plurality of single channels pre-acquired from a line array camera;

[0008] Take the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, splice all other single channels to the basic channel to obtain the corresponding spliced ​​image;

[0009] Calculate the correlation coefficient between the base channel and at least one other single channel in the stitched image;

[0010] A plurality of groups of unit splicing coefficients and a plurality of groups of correlation coefficients corresponding to the plurality of groups of unit splicing coefficients are obtained, polynomial fitting is performed based on the plurality of groups of unit splicing coefficients and the plurality of groups of correlation coefficients corresponding to the plurality of groups of unit splicing coefficients, and an optimal unit splicing coefficient is determined based on the polynomial fitting result.

[0011] In addition, the image stitching coefficient determination method according to the above embodiment of the present invention may also have the following additional technical features:

[0012] According to some embodiments of the present invention, the stitching coefficient of a single channel adjacent to a basic channel is used as a unit stitching coefficient, and all other single channels are stitched onto the basic channel to obtain a corresponding stitched image, including:

[0013] Determine the position of the base channel and the relative positions of all other single channels relative to the base channel;

[0014] Based on the relative position and the unit splicing coefficient, determine the channel splicing coefficient between the other single channels and the basic channel;

[0015] All other single channels are spliced ​​onto the basic channel according to the channel splicing coefficients to obtain a spliced ​​image.

[0016] According to some embodiments of the present invention, all other single channels are spliced ​​onto the basic channel according to the channel splicing coefficients, including:

[0017] Each single channel is moved N unit pixel distances toward the direction of the basic channel according to its corresponding channel stitching coefficient; wherein the unit pixel distance is the moving pixel distance corresponding to the unit stitching coefficient, and there are (N-1) other single channels between the single channel and the basic channel, and N is a positive integer greater than or equal to 1.

[0018] According to some embodiments of the present invention, calculating the correlation coefficient between the base channel and at least one other single channel in the stitched image further includes:

[0019] Based on the original components of multiple single channels, the change points of image splicing are located, and the correlation coefficients are calculated only for the change points.

[0020] According to some embodiments of the present invention, locating a change point of image stitching based on a plurality of single-channel original components includes:

[0021] Perform median filtering on the original components of the multiple single channels respectively to obtain single channel components after the median filtering of the multiple single channels;

[0022] Calculate the difference between the original components of multiple single channels and the single channel components to obtain the difference component of each single channel;

[0023] Calculate the standard deviation of the difference components of each single channel and retain the largest standard deviation std;

[0024] Initialize a binary image of the same size as the stitched image, traverse all pixels in the binary image, and when the value of any single channel component of the pixel is greater than N×std, set the pixel value of the pixel to 1, otherwise the pixel is 0, and obtain the filtered image; where N is a preset empirical value.

[0025] According to some embodiments of the present invention, the method further comprises:

[0026] Perform at least one morphological dilation operation on the filtered image to obtain a mask image; wherein the pixel points with a pixel value of 1 in the mask image are the change points of the spliced ​​image.

[0027] According to some embodiments of the present invention, calculating the correlation coefficient between a base channel and at least one other single channel in a stitched image includes:

[0028] Calculate the corresponding basic channel component x and the selected single channel component y whose correlation coefficient is to be calculated according to the spliced ​​image;

[0029] According to the correlation coefficient calculation formula, the correlation coefficient between the basic channel and the selected single channel under the unit splicing coefficient is calculated. The formula is:

[0030]

[0031] in, They are respectively the average value of the basic channel component x of all pixels that need to be calculated in the current stitched image and the average value of the selected single channel component y.

[0032] According to some embodiments of the present invention, obtaining values ​​of multiple groups of unit splicing coefficients and corresponding correlation coefficients, and performing polynomial fitting based on the values ​​of the multiple groups of unit splicing coefficients and corresponding correlation coefficients includes:

[0033] Calculate the correlation coefficients of the corresponding basic channel and the selected single channel under different unit splicing coefficients respectively;

[0034] The quadratic polynomial relationship between the unit splicing coefficient a and the correlation coefficient b is fitted using the least squares method, namely:

[0035] b=p1x 2 +p2x+p3

[0036] According to some embodiments of the present invention, determining the optimal splicing coefficient based on the polynomial fitting result includes:

[0037] The extreme value of the fitted binomial is obtained to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is obtained as the optimal unit splicing coefficient.

[0038] According to an image stitching coefficient determination method provided by an embodiment of the present invention, a basic channel for image stitching is determined from a plurality of single channels obtained in advance from a line array camera; the stitching coefficient of a single channel adjacent to the basic channel is used as a unit stitching coefficient, and all other single channels are stitched onto the basic channel to obtain a corresponding stitched image; the correlation coefficient between the basic channel and at least one other single channel in the stitched image is calculated; multiple groups of unit stitching coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit stitching coefficients are obtained, polynomial fitting is performed based on the multiple groups of unit stitching coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit stitching coefficients, and the optimal unit stitching coefficient is determined based on the polynomial fitting result. The embodiment of the present invention can determine the optimal unit stitching coefficient of a single channel by calculating the correlation coefficient of a single channel of a line array camera and fitting a polynomial, thereby improving the accuracy and quality of image stitching and ensuring that the stitched image is more visually coherent and consistent.

[0039] To achieve the above object, a second embodiment of the present invention provides an image stitching method based on a line array camera, comprising:

[0040] Obtaining the single-channel parameters corresponding to the target object photographed by the line array camera;

[0041] Determine the base channel for image stitching;

[0042] Determine the corresponding splicing coefficients of other single channels to the basic channel;

[0043] The stitching coefficients of the single channels are stitched together to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image taken by the linear array camera.

[0044] According to some embodiments of the present invention, respectively determining the splicing coefficients corresponding to splicing other single channels to the basic channel includes:

[0045] The splicing coefficient of the single channel adjacent to the basic channel is used as the unit splicing coefficient, and the channel splicing coefficients of other single channels spliced ​​to the basic channel are determined;

[0046] Under multiple groups of different unit stitching coefficients, the images are stitched separately;

[0047] Select at least one single channel as a selected single channel;

[0048] Calculate the correlation coefficient between the base channel and the selected single channel under each set of stitched images;

[0049] Based on the numerical relationship between multiple groups of unit splicing coefficients and corresponding correlation coefficients, find the optimal unit splicing coefficient;

[0050] Based on the optimal unit splicing coefficient, the channel splicing coefficients of other single channels spliced ​​to the basic channel are determined.

[0051] According to some embodiments of the present invention, taking a single channel splicing coefficient adjacent to a basic channel as a unit splicing coefficient and determining a channel splicing coefficient of other single channels spliced ​​to the basic channel includes:

[0052] Determine the position of the base channel and the relative positions of all other single channels relative to the base channel;

[0053] Based on the relative position and the unit splicing coefficient, determine the channel splicing coefficient of other single channels to the basic channel;

[0054] Each single channel is moved N unit pixel distances toward the direction of the basic channel according to its corresponding channel stitching coefficient; wherein the unit pixel distance is the moving pixel distance corresponding to the unit stitching coefficient, and there are (N-1) other single channels between the single channel and the basic channel, and N is a positive integer greater than or equal to 1.

[0055] According to some embodiments of the present invention, calculating the correlation coefficient between the base channel and the selected single channel under each group of stitched images further includes:

[0056] Based on the original components of multiple single channels, the change points of image splicing are located, and the correlation coefficients are calculated only for the change points;

[0057] Performing median filtering on the original components of the multiple single channels to obtain single channel components after the median filtering of the multiple single channels;

[0058] Calculate the difference between the original components of multiple single channels and the single channel components to obtain the single channel difference components;

[0059] Calculate the standard deviation of each single channel difference component and retain the largest standard deviation std;

[0060] Initialize a binary image of the same size as the stitched image, traverse all pixels in the binary image, and when the value of any single channel component of the pixel is greater than N×std, set the pixel value of the pixel to 1, otherwise the pixel is 0, and obtain the filtered image; where N is a preset empirical value;

[0061] Perform at least one morphological dilation operation on the filtered image to obtain a mask image; wherein the pixel points with a pixel value of 1 in the mask image are the change points of the spliced ​​image.

[0062] According to some embodiments of the present invention, calculating the correlation coefficient between the base channel and the selected single channel under each group of stitched images further includes:

[0063] Calculate the corresponding basic channel component x and the selected single channel component y whose correlation coefficient is to be calculated according to the spliced ​​image;

[0064] According to the correlation coefficient calculation formula, the correlation coefficient between the basic channel and the selected single channel under the unit splicing coefficient is calculated. The formula is:

[0065]

[0066] in, They are respectively the average value of the basic channel component x of all pixels that need to be calculated in the current stitched image and the average value of the selected single channel component y.

[0067] According to some embodiments of the present invention, based on the numerical relationship between multiple groups of unit splicing coefficients and corresponding correlation coefficients, finding the optimal unit splicing coefficient includes:

[0068] Calculate the correlation coefficients of the corresponding basic channel and the selected single channel under different unit splicing coefficients respectively;

[0069] The quadratic polynomial relationship between the unit splicing coefficient a and the correlation coefficient b is fitted using the least squares method, namely:

[0070] b=p1x 2 +p2x+p3

[0071] The extreme value of the fitted binomial is obtained to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is obtained as the optimal unit splicing coefficient.

[0072] According to an image stitching method based on a line array camera provided by an embodiment of the present invention, firstly, the parameters of each single channel corresponding to the target object photographed by the line array camera are obtained; further, the basic channel for image stitching is determined; then, the stitching coefficients corresponding to the stitching of other single channels to the basic channel are determined respectively; finally, stitching is performed according to the stitching coefficients of each single channel to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image photographed by the line array camera. The embodiment of the present invention optimizes the unit stitching coefficient of the single channel of the line array camera, and stitches the image according to the optimized unit stitching coefficient to obtain an image with higher resolution, thereby effectively improving the accuracy of the color sorter in identifying materials, and improving the accuracy and efficiency of the color sorter in sorting.

[0073] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0075] Figure 1 A schematic diagram of the principle of image acquisition by a color three-line array camera provided in an embodiment of the present invention;

[0076] Figure 2 A schematic diagram of an image obtained by directly stitching data collected by a color three-line array camera provided in an embodiment of the present invention;

[0077] Figure 3 A flow chart of a method for determining image splicing coefficients provided by an embodiment of the present invention;

[0078] Figure 4 A schematic diagram of channels of a six-channel linear array camera provided in an embodiment of the present invention;

[0079] Figure 5 A schematic diagram of a flow chart of a change point for locating image stitching provided by an embodiment of the present invention;

[0080] Figure 6 A schematic diagram of the R, G, and B channel components provided by an embodiment of the present invention before median filtering;

[0081] Figure 7 A schematic diagram of R, G, and B channel components after median filtering according to an embodiment of the present invention;

[0082] Figure 8 A schematic diagram of difference components corresponding to R, G, and B channel components provided in an embodiment of the present invention;

[0083] Fig. 9 A schematic diagram of a filtered image provided by an embodiment of the present invention;

[0084] Fig.10 A schematic diagram of a mask image provided by an embodiment of the present invention;

[0085] Fig.11 A schematic diagram of a device for determining image splicing coefficients provided by an embodiment of the present invention;

[0086] Fig.12A more specific schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0087] Fig.13 A flow chart of an image stitching method based on a line array camera provided in an embodiment of the present invention;

[0088] Fig.14 A schematic diagram of a process flow for determining a splicing coefficient corresponding to splicing other single channels to a basic channel provided by an embodiment of the present invention;

[0089] Fig.15 An image obtained by splicing according to the optimal splicing coefficient provided in an embodiment of the present invention;

[0090] Fig.16 A schematic diagram of an image stitching device based on a line array camera provided in an embodiment of the present invention;

[0091] Fig.17 A more specific schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0092] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0093] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0094] As described in the background technology section, when the linear array camera of the color sorter photographs and distinguishes materials on the production line, due to the large differences in the density of the materials to be sorted, when materials of different densities pass through the color sorter at a uniform speed, their running speeds may be different, which in turn causes the linear array camera to take pictures of the materials. There is color distortion in the pictures taken by the linear array camera when photographing the materials. Usually, technicians are required to repeatedly adjust the stitching coefficients of different materials so that the linear array camera can take clear pictures of the materials and identify and sort the materials more accurately. On the production line, manual adjustment is difficult and inefficient, and the technical threshold for manual labor is high, which limits the automation and intelligence level of the production line and is not conducive to the long-term use of the color sorter.

[0095] In the process of implementing the present invention, the applicant discovered that by calculating the correlation coefficient and polynomial fitting of a single channel of a linear array camera, the optimal unit stitching coefficient of a single channel can be determined, and the optimized unit stitching coefficient can be used for image stitching, thereby improving the accuracy and quality of image stitching and ensuring that the stitched image is more visually coherent and consistent.

[0096] The following is a detailed description of the technical solution of the method for determining image splicing coefficients of the present invention through specific embodiments.

[0097] The sensor of a line scan camera is linear and is responsible for receiving light and converting it into electrical signals. In this process, different parts or areas on the sensor can be regarded as different "channels", and each channel is responsible for capturing part of the image information. The line scan camera captures light point by point and converts it into electrical signals by scanning the linear sensor line by line. These electrical signals are then converted into digital signals for image reconstruction and processing. In this process, each channel plays an important role to ensure the integrity and accuracy of the image information. Line scan cameras can generally be set to have 1, 2, 3, 4, 6, 8, or 16 channels as needed. When taking pictures, each channel forms multiple rows of sampling positions on the same plane facing each other, and the multiple rows of sampling positions are spaced at the same interval. When used for color imaging, a three-channel color line scan camera is generally used, and its channels are R, G, and B channels representing red (Red), green (Green), and blue (Blue), which are the basis of color images. In the application of line scan cameras, if the camera is equipped with an RGB sensor, each channel (red, green, and blue) will capture a color component of the image.

[0098] The color trilinear camera is based on a trilinear CCD (Charge-Coupled Device) sensor, which is arranged into three linear arrays, namely red (R), green (G), and blue (B) channels. Each linear array observes the object at a different angle (or time, which can be understood as a different scanning line for static objects), thereby capturing the three color components of the object, R, G, and B. However, due to the special structure of the trilinear camera, the R, G, and B components collected at the same time do not correspond to the same point on the object.

[0099] like Figure 1 The figure shows the principle diagram of image acquisition by a color three-line array camera. It can be seen that the R, G, and B components collected at the same time do not correspond to the same point of the object. The captured images need to be stitched together with strict stitching coefficients to form an image with normal color.

[0100] Color sorters can sort materials with different densities, such as ore and tea. Due to the large density differences between different materials, the speeds of the materials passing through the color sorter will be different, which will cause distortion in the pictures taken when sorting different materials. Figure 2 The figure shows a schematic diagram of an image obtained by directly stitching data collected by a color three-line array camera. It can be seen that the color distortion of the edge of the material is obvious, which affects the recognition accuracy of the subsequent sorting algorithm.

[0101] refer to Figure 3 , is a flow chart of a method for determining image splicing coefficients provided in an embodiment of the present invention.

[0102] Determine a basic channel for image stitching from a plurality of single channels pre-acquired from a line array camera;

[0103] Taking the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, all other single channels are spliced ​​onto the basic channel to obtain a corresponding spliced ​​image;

[0104] Calculating the correlation coefficient between the basic channel and at least one other single channel in the stitched image;

[0105] Acquire multiple groups of unit splicing coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, perform polynomial fitting based on the multiple groups of unit splicing coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, and determine the optimal unit splicing coefficients based on the polynomial fitting results.

[0106] After determining the basic channel, this embodiment can determine the image stitching scheme for each channel, and then complete the data processing and stitching of each channel based on the assumed unit stitching coefficient value, so as to obtain the corresponding stitched image, and obtain the relationship between the unit stitching coefficient and the stitching image quality (correlation coefficient) by evaluating the stitching image quality, obtain multiple groups of such relationship values, obtain the numerical relationship between the two parameters based on polynomial fitting, and determine the optimal unit stitching coefficient based on the fitting result, thereby being able to evaluate the appropriate optimal stitching coefficient for different materials, avoiding the problems of low efficiency and unquantifiable results when adjusting the stitching parameters based on manual experience, and improving the imaging quality.

[0107] Specifically, the method for determining the image stitching coefficient provided in this embodiment includes:

[0108] Step S301 : determining a basic channel for image stitching from a plurality of single channels pre-acquired from a line array camera.

[0109] First, from the multiple single channels acquired by the line scan camera, one is selected as the basis for image stitching. This base channel will serve as the reference point for stitching, and other channels will be stitched around it.

[0110] For example, if a color three-line array camera captures data from three channels, R, G, and B, these data can represent the intensity information of red, green, and blue in the image respectively. Among these three channels, any one can be selected as the base channel. Generally, since the green channel (G) is the most sensitive in the human eye and contains the most image detail information, it can be preferred as the base channel. After having the base channel, the data of other channels (such as the R channel and the B channel) need to be spliced ​​onto the base channel. Figure 1 It can be known that there are gaps between the objects collected by the three channels of R, G, and B. If the data of the three channels are directly spliced, it can be considered that the image data at three different positions are spliced ​​together to form an image, so there will be unclear situations. Splicing the data of other channels to the basic channel is the process of adjusting the data of each channel to its original position. When performing image stitching, it is necessary to obtain the parameters of the linear array camera for a certain period of time to form a certain domain. The stitching process can be considered as stitching the image after making a certain displacement of the data of other channels along the time axis.

[0111] Step S302: Using the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, all other single channels are spliced ​​onto the basic channel to obtain a corresponding spliced ​​image.

[0112] Specifically, the unit stitching coefficient refers to a single channel directly adjacent to the base channel (for example, if the base channel is the green channel G, then the unit stitching coefficient may refer to the stitching coefficient between the green channel G and the red channel R or the blue channel B, depending on which channel is directly adjacent to the green channel G).

[0113] As an optional embodiment, the stitching coefficient of the single channel adjacent to the basic channel is used as the unit stitching coefficient, and all other single channels are stitched onto the basic channel to obtain a corresponding stitched image, including: determining the position of the basic channel and the relative positions of all other single channels relative to the basic channel; determining the channel stitching coefficients between other single channels and the basic channel based on the relative positions and the unit stitching coefficient; and stitching all other single channels onto the basic channel according to the channel stitching coefficients to obtain a stitched image.

[0114] Specifically, the sensors of color three-line array cameras are usually arranged in a specific arrangement, such as RGB linear arrangement, BGR linear arrangement or staggered arrangement. In the image data structure, each channel has a specific position or index. For the basic channel, its position is known, usually at the beginning of the data structure or a specific position. The relative position refers to the position relationship of other single channels relative to the basic channel in the image data structure. For a linear array camera, the physical intervals of adjacent channels are consistent on the object plane facing the camera. Therefore, when determining the stitching coefficients of other channels based on the basic stitching coefficients, it is only necessary to know the relative position relationship of all channels.

[0115] For example, for Figure 1 In the three-channel linear array camera shown, the R channel is used as the basic channel, and the stitching coefficient of the B channel to the R channel is defined as the unit stitching coefficient. Then, the stitching coefficient of the G channel to the R channel is 2 times the unit stitching coefficient.

[0116] As an optional embodiment, all other single channels are spliced ​​onto the basic channel according to the channel splicing coefficients, including: each single channel is moved N unit pixel distances toward the basic channel according to its corresponding channel splicing coefficient; wherein the unit pixel distance is the moving pixel distance corresponding to the unit splicing coefficient, and the single channel and the basic channel are separated by (N-1) other single channels, and N is a positive integer greater than or equal to 1.

[0117] For example Figure 4The six-channel line array camera shown takes the second channel 22 as the basic channel, and the stitching coefficient of the third channel 23 stitching the basic channel 22 as the unit stitching coefficient a. Since the fourth channel 24 is separated from the basic channel 22 by another channel (the third channel 23), its channel stitching coefficient is 2a. Therefore, it can be determined that the channel stitching coefficient of the fifth channel 25 is 3a, and the channel stitching coefficient of the sixth channel 26 is 4a. Since the first channel 21 and the third channel 23 are located on both sides of the basic channel 22, the value of its channel stitching coefficient is still a, but considering the opposite direction, it should be set to -a during calculation.

[0118] Specifically, the unit pixel distance refers to the moving pixel distance corresponding to the unit stitching coefficient. The specific value of the distance does not need to be accurately calculated. This definition is introduced in this embodiment only to facilitate the understanding of the technical solution. In actual processing, there is no need to consider the specific value of the pixel distance. It is only necessary to evaluate the quality of the processed image to determine whether the value of the unit stitching coefficient is appropriate.

[0119] Step S303, calculating the correlation coefficient between the basic channel and at least one other single channel in the stitched image.

[0120] The correlation coefficient is a statistic that measures the strength and direction of the linear relationship between two single channel components. In image processing, calculating the correlation coefficient between the base channel and other single channels can determine the similarity and difference between the single channels. Step S304, obtaining multiple groups of unit splicing coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, performing polynomial fitting based on the multiple groups of unit splicing coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, and determining the optimal unit splicing coefficient based on the polynomial fitting result.

[0121] In the embodiment of the present invention, multiple groups of unit stitching coefficients may be collected or generated, and these coefficients represent stitching effects under different parameter settings in the image stitching task, or represent other parameter combinations that need to be optimized.

[0122] For each set of unit splicing coefficients, a correlation coefficient associated with it is obtained, and polynomial fitting is performed using the collected multiple sets of unit splicing coefficients and correlation coefficients as data points. Polynomial fitting is a mathematical method used to find a set of polynomial coefficients that can best describe the relationship between data points by minimizing errors. Based on the results of polynomial fitting, the unit splicing coefficient that maximizes the correlation coefficient (or minimizes a certain error index) can be found. This coefficient can be used as the optimal unit splicing coefficient.

[0123] Specifically, calculating the correlation coefficient between the basic channel and at least one other single channel in the stitched image includes: finding the corresponding basic channel component x and the selected single channel component y whose correlation coefficient is to be calculated according to the stitched image; calculating the correlation coefficient between the basic channel and the selected single channel under the unit stitching coefficient according to the correlation coefficient calculation formula, the formula is:

[0124]

[0125] in, They are respectively the average value of the basic channel component x of all pixels that need to be calculated in the current stitched image and the average value of the selected single channel component y.

[0126] Specifically, extract the basic channel component x and the selected single channel component y from the stitched image. These components can represent different color channels (such as red, green, blue) or other feature channels of the image. Further, calculate the average value x of the basic channel component x and the average value of the selected single channel component y of all pixels to be calculated in the current stitched image. The average value can be obtained by summing the corresponding channel components of all pixels and then dividing by the total number of pixels. The correlation coefficient calculation formula is used to calculate the correlation coefficient between the basic channel and the selected single channel. The correlation coefficient is a statistic that measures the degree of linear correlation between two variables, and its value is between -1 and 1. The closer the correlation coefficient value is to 1, the stronger the linear relationship between the basic channel and the selected single channel (positive correlation); the closer it is to -1, the stronger the linear relationship (negative correlation); close to 0, there is almost no linear relationship.

[0127] As an optional embodiment, multiple sets of unit splicing coefficients and corresponding correlation coefficients are obtained, and polynomial fitting is performed based on the multiple sets of unit splicing coefficients and corresponding correlation coefficients, including: calculating the correlation coefficients of the corresponding basic channel and the selected single channel under different unit splicing coefficients respectively; and fitting the quadratic polynomial relationship between the unit splicing coefficient a and the correlation coefficient b by the least squares method, that is:

[0128] b=p1a 2 +p2a+p3

[0129] Among them, taking the R channel as the base channel and the G channel as an example, a is the unit splicing coefficient, b is the correlation coefficient of the corresponding R channel and G channel components, and q1, q2, and q3 are the coefficients of the quadratic term, the linear term, and the zero-order term in the formula respectively.

[0130] As an optional embodiment, the optimal stitching coefficient is determined based on the polynomial fitting result, including: finding the extreme value of the fitted binomial to obtain the unit stitching coefficient a corresponding to the maximum value of the correlation coefficient b; if the correlation coefficients of multiple selected single channels and the basic channel are calculated, then the average value of the multiple unit stitching coefficients a is calculated as the optimal unit stitching coefficient.

[0131] Fit quadratic polynomials to the correlation coefficients of the R channel and G channel, and the R channel and B channel components, and solve for the maximum value and the corresponding splicing coefficient. The specific solution process is as follows:

[0132] The least squares method is used to fit the correlation coefficient to a quadratic polynomial with the splicing coefficient as a variable. The least squares method is used to fit the quadratic polynomial formula as follows:

[0133] Let a1, a2, ..., a n are splicing coefficients, b1, b2, ..., b n is the corresponding correlation coefficient, and the fitting quadratic polynomial is b=p1a 2 +p2a+p3, then the mean square error is E=∑ ( b i -b i ' ) 2 =∑(b i -p1a i2 -p2a i -p3), and its partial derivative is By setting the three partial derivatives to 0, we can find p1, p2, and p3, and the maximum value is The splicing coefficient corresponding to the maximum value is

[0134] The final splicing coefficient is the average of the splicing coefficients corresponding to the maximum values ​​of the correlation coefficients between the R channel and the G channel, and between the R channel and the B channel. The formula is as follows:

[0135]

[0136] Among them, offset RG The splicing coefficient obtained for the R channel and G channel components, offset RB The splicing coefficients obtained for the R channel and B channel components.

[0137] According to the obtained stitching coefficient offset, the R channel component remains unchanged (R channel is the basic channel), the B channel component moves the offset, and the G channel component moves 2×offset, and then they are re-stitched into an image.

[0138] If offset is a decimal, the concatenation formula is as follows:

[0139]

[0140] Among them, img'(i,j) represents the image after splicing at (i,j), img(i,j) represents the image to be spliced ​​at (i,j), and {*} represents taking the decimal part. Indicates rounding down. Indicates rounding up.

[0141] Furthermore, in the process of calculating the correlation coefficient, since all pixels are involved in the calculation, the amount of calculation is large, the computing power required is high, and the program runs slowly. In addition, for the application scenario of the color sorter, the material area in the image is generally not too high relative to the background area of ​​the conveyor belt, which causes the value of the pixel points in the background area to make the calculation result of the correlation coefficient higher than the theoretical expectation, affecting the calculation of the final unit splicing coefficient.

[0142] As an optional embodiment, the calculation of the correlation coefficient between the basic channel and at least one other single channel in the stitched image further includes: locating the change points of the image stitching based on the original components of multiple single channels, and calculating the correlation coefficient only for the change points.

[0143] The change point of image splicing refers to the position where discontinuity or mutation may occur in the image splicing process. In this embodiment, in order to reduce the amount of data calculation, it is preferred to calculate the correlation coefficient of the change point and the subsequent splicing process, and at the same time, a more accurate correlation coefficient can be obtained.

[0144] refer to Figure 5 , is a schematic diagram of a process for locating change points of image stitching provided by an embodiment of the present invention.

[0145] As an optional embodiment, locating the change point of image splicing based on the original components of multiple single channels includes:

[0146] Step S501, performing median filtering on the original components of multiple single channels respectively to obtain single channel components after median filtering of multiple single channels;

[0147] Step S502, calculating the difference between the original components of multiple single channels and the single channel components to obtain the difference component of each single channel;

[0148] Step S503, calculating the standard deviation of the difference components of each single channel, and retaining the largest standard deviation std;

[0149] Step S504, initialize a binary image of the same size as the stitched image, traverse all pixels in the binary image, and when the value of any single channel component of the pixel is greater than N×std, the pixel value of the pixel is set to 1, otherwise the pixel is 0, and a filtered image is obtained; wherein N is a preset empirical value.

[0150] Based on the above steps, a binary image can be obtained, in which the pixel points with a pixel value of 1 can be considered as the change points in the image stitching process. Only the correlation coefficients of the basic channel and the selected single channel of the relevant pixel points are calculated, so as to obtain a more accurate value reflecting the correlation between the two channels.

[0151] First, median filtering is performed on multiple single channels (including basic channels and other single channels) in the spliced ​​image. Median filtering is a nonlinear filtering technique that can effectively remove noise (such as salt and pepper noise, random noise) in images while retaining the edge information of the image.

[0152] refer to Figure 6 , is a schematic diagram of the R, G, and B channel components before median filtering. Figure 6 (a) is a schematic diagram of the R channel component before median filtering. Figure 6 (b) is a schematic diagram of the G channel component before median filtering. Figure 6 (c) is a schematic diagram of the B channel component before median filtering. One of the main advantages of median filtering is that it can effectively remove noise from the image. If median filtering is not performed, the noise may remain in the image, affecting the visual quality of the image and the effect of subsequent processing.

[0153] refer to Figure 7 , is a schematic diagram of the R, G, and B channel components after median filtering. Figure 7 (a) is a schematic diagram of the R channel component after median filtering, 7(b) is a schematic diagram of the G channel component after median filtering, and 7(c) is a schematic diagram of the B channel component after median filtering. The median filtered component of each single channel can be obtained by median filtering. Figure 7 (a) Figure 7 (b) and Figure 7 (c) Compared with 6(a), Figure 6 (b) and Figure 6 (c) Remove noise from the image while retaining the edge information of the image.

[0154] Furthermore, the difference between the original component of each single channel and the component after median filtering is calculated to obtain the difference component of each single channel. Figure 8, which is a schematic diagram of the difference components corresponding to the R, G, and B channel components. Figure 8 (a) is a schematic diagram of the difference component corresponding to the R channel component. Figure 8 (b) is a schematic diagram of the difference component corresponding to the G channel component. Figure 8 (c) is a schematic diagram of the difference component corresponding to the B channel component. The difference component reflects the difference between the original image and the filtered image, which contains the information of the image splicing change point. Then, the standard deviation of the difference component of each single channel is calculated. The standard deviation is a statistic that measures the degree of data dispersion. Here it can help identify which single channel difference components contain more change information. The single channel difference component with the largest standard deviation is retained because it is most likely to contain the main change point of image splicing.

[0155] As an optional embodiment, in order to locate the change point more intuitively, a binary image of the same size as the spliced ​​image can be initialized. Then, all the pixels in the binary image are traversed, and the value of any single channel component of each pixel (usually the single channel difference component corresponding to the maximum standard deviation previously retained) is checked. If the value of the single channel component of a certain pixel is greater than a preset empirical value N (N is usually set according to the specific situation of the image and the processing requirements), the pixel value of the pixel is set to 1 (indicating a change point), otherwise it is set to 0 (indicating a non-change point). In this way, a filtered image is obtained, which only contains the main change points of the image splicing. Specifically, a binary image of the same size as the initial image is created, and the comparison value assigned is determined to be N×std according to the standard deviation std, where N is an integer, and in the embodiment of the present invention, N is preferably 5.

[0156] refer to Fig. 9 , which is a schematic diagram of the filtered image. The white area (pixel value is 1) in the binary image represents the pixels with significant fluctuations in any single channel component in the initial image. These pixels may be the edges, textures or noise in the image when the initial splicing coefficient is used for splicing. The black area (pixel value is 0) in the binary image represents the pixels with no fluctuations or negligible fluctuations in any single channel component in the image. These pixels are the smooth areas or backgrounds in the image when the initial splicing coefficient is used for splicing.

[0157] Furthermore, in order to obtain a more accurate value of the correlation coefficient, the neighboring pixels of the pixel points in the binary image can be calculated as change points, or the pixel points can be amplified in other ways. For example, as an optional embodiment, the filtered image is subjected to at least one morphological dilation operation to obtain a mask image; wherein the pixel points with a pixel value of 1 in the expanded mask image are the change points of the image.

[0158] Perform at least one morphological dilation operation on the filtered image. Morphological dilation is a nonlinear operation that is usually used to enlarge bright areas or foreground objects in an image. It is achieved by sliding a structure element (usually a small rectangle or disk) over the image and assigning the maximum value of the area covered by the structure element to the pixel at the center of the structure element. After the morphological dilation operation, some areas in the image may become brighter or more prominent. Then, a mask image is generated based on the dilated image. In this mask image, pixels with a pixel value of 1 represent the change points of the spliced ​​image, that is, those areas that become more prominent due to morphological dilation.

[0159] Specifically, a circle with a structure element radius of d (the radius of the circle is preferably 5) is used to perform morphological dilation on the binary image, that is, the white area in the target binary image is enlarged by a circle with a radius of 5, and the following is obtained: Fig.10 The expanded mask image shown in FIG. 1 , wherein the pixel value of the pixel in the mask image is 1.

[0160] According to an image stitching coefficient determination method provided by an embodiment of the present invention, a basic channel for image stitching is determined from a plurality of single channels obtained in advance from a line array camera, and a stitching scheme of the entire stitched image is determined according to the position of the basic channel; the stitching coefficient of a single channel adjacent to the basic channel is used as a unit stitching coefficient, and all other single channels are stitched onto the basic channel to obtain a corresponding stitched image; the correlation coefficient between the basic channel and at least one other single channel in the stitched image is calculated; multiple groups of unit stitching coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit stitching coefficients are obtained, polynomial fitting is performed based on the multiple groups of unit stitching coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit stitching coefficients, and the optimal unit stitching coefficient is determined based on the polynomial fitting result. The embodiment of the present invention can determine the optimal unit stitching coefficient of a single channel by calculating the correlation coefficient of a single channel of a line array camera and fitting the polynomial, thereby improving the accuracy and quality of image stitching and ensuring that the stitched image is more visually coherent and consistent.

[0161] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.

[0162] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] Based on the same inventive concept, corresponding to the method provided in any of the above embodiments, the present invention further provides a device for determining image splicing coefficients.

[0164] refer to Fig.11 , is a schematic diagram of an image splicing coefficient determination device provided by an embodiment of the present invention.

[0165] The image splicing coefficient determination device includes: an acquisition module 1110 , a splicing module 1120 , a calculation module 1130 and a determination module 1140 .

[0166] The acquisition module 1111 is configured to determine a basic channel for image stitching from a plurality of single channels pre-acquired from the line array camera;

[0167] The stitching module 1120 is configured to use the stitching coefficient of the single channel adjacent to the basic channel as the unit stitching coefficient, stitch all other single channels to the basic channel, and obtain a corresponding stitched image;

[0168] The calculation module 1130 is configured to calculate the correlation coefficient between the base channel and at least one other single channel in the stitched image;

[0169] The determination module 1140 is configured to obtain multiple groups of unit splicing coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, perform polynomial fitting based on the multiple groups of unit splicing coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, and determine the optimal unit splicing coefficients based on the polynomial fitting results.

[0170] For the convenience of description, the above device is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0171] The image stitching coefficient determination device of the above embodiment is used to implement the corresponding image stitching coefficient determination method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0172] Based on the same inventive concept, corresponding to the method for determining the image stitching coefficients described in any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining the image stitching coefficients described in any of the above embodiments is implemented.

[0173] Fig.12 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1210, a memory 1220, an input / output interface 1230, a communication interface 1240, and a bus 1250. The processor 1210, the memory 1220, the input / output interface 1230, and the communication interface 1240 are connected to each other through the bus 1250 in the device.

[0174] The processor 1210 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0175] The memory 1220 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1220 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1220 and are called and executed by the processor 1210.

[0176] The input / output interface 1230 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0177] The communication interface 1240 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0178] The bus 1250 includes a path that transmits information between the various components of the device (eg, the processor 1210 , the memory 1220 , the input / output interface 1230 , and the communication interface 1240 ).

[0179] It should be noted that, although the above device only shows the processor 1210, the memory 1220, the input / output interface 1230, the communication interface 1240 and the bus 1250, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0180] The electronic device of the above embodiment is used to implement the corresponding method for determining the image stitching coefficient in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0181] Based on the same inventive concept, corresponding to the method for determining the image stitching coefficient described in any of the above embodiments, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for determining the image stitching coefficient described in any of the above embodiments.

[0182] The above-mentioned computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0183] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image stitching coefficient determination method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0184] The following is a detailed description of the technical solution of the image stitching method based on a line array camera of the present invention through specific embodiments.

[0185] refer to Fig.13 , is a flow chart of an image stitching method based on a line array camera provided in an embodiment of the present invention.

[0186] Obtaining the single-channel parameters corresponding to the target object photographed by the line array camera;

[0187] Determine the base channel for image stitching;

[0188] Determine the corresponding splicing coefficients of other single channels to the basic channel;

[0189] The stitching is performed according to the stitching coefficients of each single channel to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image taken by the line array camera.

[0190] This embodiment determines the stitching scheme for each channel to obtain a stitched image by selecting a basic channel, and then determines the specific stitching coefficient of each channel, thereby completing the pattern stitching of the line array camera, and can obtain a clear stitched image, which is conducive to image-based product identification and sorting.

[0191] Furthermore, the image stitching method based on the line array camera provided in this embodiment includes:

[0192] Step S1301, obtaining the single-channel parameters corresponding to the target object photographed by the line array camera.

[0193] First, a line array camera can be used to shoot the target object and obtain the single-channel images or data obtained by the shooting. These single channels may represent different color channels (such as red, green, and blue), or other specific image feature channels.

[0194] Step S1302, determining a basic channel for image stitching.

[0195] First, one of the multiple single channels acquired by the line array camera can be selected as the basis for image stitching. This basic channel will be used as a reference point for stitching, and other channels will be stitched around it. For example, in this embodiment, the R channel is used as the basic channel, and the data of the G and B channels are processed separately to be stitched onto the R channel to obtain a stitched image.

[0196] Step S1303, respectively determine the splicing coefficients corresponding to splicing other single channels to the basic channel.

[0197] For details, please refer to the attached Fig.14 , which is a schematic diagram of a process for determining a splicing coefficient corresponding to splicing other single channels to a basic channel provided by an embodiment of the present invention.

[0198] The respectively determining the splicing coefficients corresponding to splicing other single channels to the basic channel includes:

[0199] Step S1401, taking the splicing coefficient of a single channel adjacent to a basic channel as a unit splicing coefficient, and determining the channel splicing coefficients of other single channels spliced ​​to the basic channel;

[0200] Step S1402, stitching the images respectively under a plurality of different unit stitching coefficients;

[0201] Step S1403, selecting at least one of the single channels as a selected single channel;

[0202] Step S1404, calculating the correlation coefficient between the basic channel and the selected single channel in each group of stitched images;

[0203] Step S1405, finding the optimal unit splicing coefficient based on the numerical relationship between the multiple groups of unit splicing coefficients and the corresponding correlation coefficients;

[0204] Step S1406: determining channel splicing coefficients for splicing other single channels to the basic channel based on the optimal unit splicing coefficient.

[0205] This embodiment stitches images at different stitching coefficients and obtains corresponding correlation coefficients, thereby quantifying the quality of the stitched image, and determining the optimal stitching coefficient based on the numerical relationship between the stitching coefficient and the correlation coefficient. Therefore, the appropriate stitching coefficient can be accurately determined through data processing, reducing dependence on the experience of on-site debugging personnel, meeting the requirements of data switching for different material scenarios, and improving the flexibility of production line products.

[0206] Specifically, the scheme of performing image stitching and calculating correlation coefficients with unit stitching coefficients in this embodiment is the same as that in the above embodiment. This implementation also includes the purpose of highlighting the location change points of the stitching to reduce the amount of data processing and improve the accuracy of the correlation coefficient, and the specific scheme is not repeated here.

[0207] Step S1304 , performing stitching according to the stitching coefficients of the single channels to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image taken by the line array camera.

[0208] According to the obtained stitching coefficient offset, the R channel component remains unchanged (R channel is the basic channel), the B channel component moves the offset, and the G channel component moves 2×offset, and then they are re-stitched into an image, such as Fig.15 shown. Fig.15 is the image obtained by stitching according to the optimal stitching coefficient. Figure 2 , Fig.15 The material edges are clear and the resolution is high.

[0209] If offset is a decimal, the concatenation formula is as follows:

[0210]

[0211] Among them, img'(i,j) represents the image after splicing at (i,j), img(i,j) represents the image to be spliced ​​at (i,j), and {*} represents taking the decimal part. Indicates rounding down. Indicates rounding up.

[0212] According to an image stitching method based on a line array camera provided by an embodiment of the present invention, firstly, the parameters of each single channel corresponding to the target object photographed by the line array camera are obtained; further, the basic channel for image stitching is determined; then, the stitching coefficients corresponding to the stitching of other single channels to the basic channel are determined respectively; finally, stitching is performed according to the stitching coefficients of each single channel to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image photographed by the line array camera. The embodiment of the present invention optimizes the unit stitching coefficient of the single channel of the line array camera, and stitches the image according to the optimized unit stitching coefficient to obtain an image with higher resolution, thereby effectively improving the accuracy of the color sorter in identifying materials, and improving the accuracy and efficiency of the color sorter in sorting.

[0213] According to an image stitching method based on a line array camera provided by an embodiment of the present invention, firstly, the parameters of each single channel corresponding to the target object photographed by the line array camera are obtained; further, the basic channel for image stitching is determined; then, the stitching coefficients corresponding to the stitching of other single channels to the basic channel are determined respectively; finally, stitching is performed according to the stitching coefficients of each single channel to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image photographed by the line array camera. The embodiment of the present invention optimizes the unit stitching coefficient of the single channel of the line array camera, and stitches the image according to the optimized unit stitching coefficient to obtain an image with higher resolution, thereby effectively improving the accuracy of the color sorter in identifying materials, and improving the accuracy and efficiency of the color sorter in sorting.

[0214] It should be noted that the image stitching method based on a line array camera of the embodiment of the present invention can be executed by a single device, such as a computer or a server. The image stitching method based on a line array camera of the present embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only execute one or more steps of the image stitching method based on a line array camera of the embodiment of the present invention, and the multiple devices will interact with each other to complete the image stitching method based on a line array camera.

[0215] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0216] Based on the same inventive concept, corresponding to the image stitching method based on the line array camera provided in any of the above embodiments, the present invention further provides an image stitching coefficient determination device.

[0217] refer to Fig.16 , is a schematic diagram of an image stitching device based on a line array camera provided in an embodiment of the present invention.

[0218] The image stitching device based on a line array camera includes: an acquisition module 1610 , a first determination module 1620 , a second determination module 1630 and a stitching module 1640 .

[0219] The acquisition module 1610 is configured to acquire each single channel parameter corresponding to the target object photographed by the line array camera;

[0220] The first determination module 1620 is configured to determine a basic channel for image stitching;

[0221] The second determination module 1630 is configured to respectively determine the splicing coefficients corresponding to splicing other single channels to the basic channel;

[0222] The stitching module 1640 is configured to stitch together the stitching coefficients of each single channel to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image taken by the line array camera.

[0223] For the convenience of description, the above device is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0224] The device of the above embodiment is used to implement the corresponding image stitching method based on line array cameras in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0225] Based on the same inventive concept, corresponding to the image stitching method based on the line array camera described in any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image stitching method based on the line array camera described in any of the above embodiments is implemented.

[0226] Fig.17A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1710, a memory 1720, an input / output interface 1730, a communication interface 1740, and a bus 1750. The processor 1710, the memory 1720, the input / output interface 1730, and the communication interface 1740 are connected to each other through the bus 1750 in the device.

[0227] The processor 1710 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0228] The memory 1720 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1720 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1720 and are called and executed by the processor 1710.

[0229] The input / output interface 1730 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0230] The communication interface 1740 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0231] The bus 1750 includes a path that transmits information between the various components of the device (eg, the processor 1710, the memory 1720, the input / output interface 1730, and the communication interface 1740).

[0232] It should be noted that, although the above device only shows the processor 1710, the memory 1720, the input / output interface 1730, the communication interface 1740 and the bus 1750, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0233] The electronic device of the above embodiment is used to implement the corresponding image stitching method based on the line array camera in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0234] Based on the same inventive concept, corresponding to the image stitching method based on the line array camera described in any of the above embodiments, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image stitching method based on the line array camera described in any of the above embodiments.

[0235] The above-mentioned computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0236] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image stitching method based on a line array camera as described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0237] In addition, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0238] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0239] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0240] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and this division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. A method for determining image splicing coefficients, characterized in that: include: Determine a basic channel for image stitching from a plurality of single channels pre-acquired from a line array camera; Taking the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, all other single channels are spliced ​​onto the basic channel to obtain a corresponding spliced ​​image; Calculating the correlation coefficient between the basic channel and at least one other single channel in the stitched image; Acquire multiple groups of unit splicing coefficients and multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, perform polynomial fitting based on the multiple groups of unit splicing coefficients and the multiple groups of correlation coefficients corresponding to the multiple groups of unit splicing coefficients, and determine the optimal unit splicing coefficients based on the polynomial fitting results.

2. The method for determining image splicing coefficients according to claim 1, characterized in that: The method uses the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, splices all other single channels to the basic channel, and obtains a corresponding spliced ​​image, including: Determine the position of the basic channel and the relative positions of all other single channels relative to the basic channel; Determine a channel stitching coefficient between another single channel and the basic channel based on the relative position and the unit stitching coefficient; All other single channels are spliced ​​onto the basic channel according to the channel splicing coefficients to obtain the spliced ​​image.

3. The method for determining image splicing coefficients according to claim 2, characterized in that: The step of splicing all other single channels onto the basic channel according to the channel splicing coefficients includes: Each single channel is moved N unit pixel distances toward the direction of the basic channel according to the corresponding channel stitching coefficient; wherein the unit pixel distance is the moving pixel distance corresponding to the unit stitching coefficient, and the single channel is spaced from the basic channel by (N-1) other single channels, and N is a positive integer greater than or equal to 1.

4. The method for determining image splicing coefficients according to claim 3, characterized in that: The calculating of the correlation coefficient between the basic channel and at least one other single channel in the stitched image further includes: Based on the original components of the multiple single channels, the change points of image splicing are located, and the correlation coefficient is calculated only for the change points.

5. The method for determining image splicing coefficients according to claim 4, characterized in that: The step of locating the change point of image splicing based on the original components of the multiple single channels includes: Performing median filtering on the original components of the multiple single channels respectively to obtain single channel components after median filtering of the multiple single channels; Calculating the difference between the original components of the multiple single channels and the single channel components to obtain the difference component of each single channel; Calculate the standard deviation of the difference component of each single channel and retain the largest standard deviation std; Initialize a binary image of the same size as the stitched image, traverse all pixels in the binary image, and when the value of any single-channel component of the pixel is greater than N×std, set the pixel value of the pixel to 1, otherwise the pixel is 0, to obtain a filtered image; wherein N is a preset empirical value.

6. The method for determining image splicing coefficients according to claim 5, characterized in that: The method further comprises: Perform at least one morphological dilation operation on the filtered image to obtain a mask image; wherein the pixel points with a pixel value of 1 in the mask image are the change points of the spliced ​​image.

7. The method for determining image splicing coefficients according to any one of claims 1 or 4 to 6, characterized in that: The calculating the correlation coefficient between the basic channel and at least one other single channel in the stitched image comprises: Calculate the corresponding basic channel component x and the selected single channel component y whose correlation coefficient is to be calculated according to the stitched image; According to the correlation coefficient calculation formula, the correlation coefficient between the basic channel and the selected single channel under the unit splicing coefficient is calculated. The formula is: in, They are respectively the average value of the basic channel component x of all pixels that need to be calculated in the current stitched image and the average value of the selected single channel component y.

8. The method for determining image splicing coefficients according to claim 7, characterized in that: The obtaining of values ​​of multiple groups of unit splicing coefficients and corresponding correlation coefficients, and performing polynomial fitting based on the values ​​of the multiple groups of unit splicing coefficients and corresponding correlation coefficients, includes: Calculate the correlation coefficients of the corresponding basic channel and the selected single channel under different unit splicing coefficients respectively; The quadratic polynomial relationship between the unit splicing coefficient a and the correlation coefficient b is fitted using the least squares method, namely: b=p1x 2 +p2x+p3。 9. The method for determining image splicing coefficients according to claim 8, characterized in that: The determining of the optimal splicing coefficient based on the polynomial fitting result includes: The extreme value of the fitted binomial is obtained to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is obtained as the optimal unit splicing coefficient.

10. An image stitching method based on a line array camera, characterized in that: include: Obtaining the single-channel parameters corresponding to the target object photographed by the line array camera; Determine the base channel for image stitching; Determine the corresponding splicing coefficients of other single channels to the basic channel; The stitching is performed according to the stitching coefficients of each single channel to obtain a stitched image of the target object; wherein the resolution of the stitched image is higher than the initial image taken by the line array camera.

11. The image stitching method based on a line array camera according to claim 10, characterized in that: The respectively determining the splicing coefficients corresponding to splicing other single channels to the basic channel includes: Taking the splicing coefficient of a single channel adjacent to the basic channel as a unit splicing coefficient, and determining the channel splicing coefficients of other single channels spliced ​​to the basic channel; Under multiple groups of different unit stitching coefficients, the images are stitched separately; selecting at least one of the single channels as a selected single channel; Calculating the correlation coefficient between the basic channel and the selected single channel under each group of stitched images; Finding the optimal unit splicing coefficient based on the numerical relationship between the multiple groups of unit splicing coefficients and the corresponding correlation coefficients; Based on the optimal unit splicing coefficient, channel splicing coefficients for splicing other single channels to the basic channel are determined.

12. The image stitching method based on a line array camera according to claim 11, characterized in that: The method of taking the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient and determining the channel splicing coefficients of other single channels spliced ​​to the basic channel includes: Determine the position of the basic channel and the relative positions of all other single channels relative to the basic channel; Based on the relative position and the unit splicing coefficient, determining a channel splicing coefficient for splicing the other single channels to the basic channel; Each single channel is moved N unit pixel distances toward the direction of the basic channel according to the corresponding channel stitching coefficient; wherein the unit pixel distance is the moving pixel distance corresponding to the unit stitching coefficient, and the single channel is spaced from the basic channel by (N-1) other single channels, and N is a positive integer greater than or equal to 1.

13. The image stitching method based on a line array camera according to claim 11, characterized in that: The step of calculating the correlation coefficient between the basic channel and the selected single channel under each group of stitched images further includes: Based on the original components of multiple single channels, locate the change points of image splicing, and calculate the correlation coefficient only for the change points; The positioning image splicing change point includes: Performing median filtering on the original components of the multiple single channels to obtain single channel components after median filtering of the multiple single channels; Calculating the difference between the original components of the multiple single channels and the single channel component to obtain a single channel difference component; Calculate the standard deviation of each of the single-channel difference components and retain the largest standard deviation std; Initialize a binary image of the same size as the stitched image, traverse all pixels in the binary image, and when the value of any single channel component of the pixel is greater than N×std, set the pixel value of the pixel to 1, otherwise the pixel is 0, to obtain a filtered image; wherein N is a preset empirical value; Perform at least one morphological dilation operation on the filtered image to obtain a mask image; wherein the pixel points with a pixel value of 1 in the mask image are the change points of the spliced ​​image.

14. The image stitching method based on a line array camera according to any one of claims 11 to 13, characterized in that: The step of calculating the correlation coefficient between the basic channel and the selected single channel under each group of stitched images further includes: Calculate the corresponding basic channel component x and the selected single channel component y whose correlation coefficient is to be calculated according to the stitched image; According to the correlation coefficient calculation formula, the correlation coefficient between the basic channel and the selected single channel under the unit splicing coefficient is calculated. The formula is: in, They are respectively the average value of the basic channel component x of all pixels that need to be calculated in the current stitched image and the average value of the selected single channel component y.

15. The image stitching method based on a line array camera according to claim 14, characterized in that: The step of finding the optimal unit splicing coefficient based on the numerical relationship between the plurality of groups of the unit splicing coefficients and the corresponding correlation coefficients includes: Calculate the correlation coefficients of the corresponding basic channel and the selected single channel under different unit splicing coefficients respectively; The quadratic polynomial relationship between the unit splicing coefficient a and the correlation coefficient b is fitted using the least squares method, namely: b=p1x 2 +p2x+p3 The extreme value of the fitted binomial is obtained to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is obtained as the optimal unit splicing coefficient.

Images