Image stitching coefficient determination method and image stitching method based on linear scan camera

Abstract

This invention relates to the field of image processing technology, and provides a method for determining image stitching coefficients and an image stitching method based on a line scan camera. The method for determining image stitching coefficients includes: determining a base channel for image stitching from multiple single channels pre-acquired from a line scan camera; using the stitching coefficient of the single channel adjacent to the base channel as the unit stitching coefficient, stitching all other single channels onto the base channel to obtain a corresponding stitched image; calculating the correlation coefficient between the base channel and at least one other single channel in the stitched image; obtaining multiple sets of unit stitching coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit stitching coefficients; performing polynomial fitting based on the multiple sets of unit stitching coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit stitching coefficients; and determining the optimal unit stitching coefficient based on the polynomial fitting result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for determining image stitching coefficients and an image stitching method based on a line scan camera. Background Technology

[0002] When materials are sorted on the production line by a color sorter, a line scan camera photographs and identifies the materials for further sorting. For example, a three-channel color line scan camera has three channels: R, G, and B. The images from the three channels need to be stitched together to obtain a clear image before identification and sorting. Due to the significant density differences between different materials, their speeds through the color sorter will vary. This means that if the same stitching coefficient is used for image stitching when sorting different materials, the captured images will be distorted, affecting the color sorter's recognition accuracy. Therefore, when changing materials or debugging the color sorter, the image stitching coefficient needs to be adjusted according to the product's adaptability.

[0003] To solve the problem of distorted images, technicians typically need to adjust the splicing coefficients of different materials to obtain clear images. However, manual adjustment is not only difficult to operate but also inefficient, limiting the automation and intelligence level of the production line and affecting the sorting efficiency of the color sorter. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, the first objective of this invention is to propose an image stitching coefficient determination method to efficiently determine the image stitching coefficient of a line scan camera and eliminate reliance on operator experience; it optimizes the unit stitching coefficient of a single channel of the line scan camera and stitches images according to the optimized unit stitching coefficient to obtain higher resolution images, thereby effectively improving the accuracy of material identification by color sorters and increasing the sorting accuracy and efficiency of color sorters.

[0005] The second objective of this invention is to propose an image stitching method based on a line scan camera.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for determining image stitching coefficients, comprising:

[0007] The base channel for image stitching is determined from multiple single channels pre-acquired from the line scan camera;

[0008] Using the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, all other single channels are spliced ​​onto the basic channel to obtain the corresponding spliced ​​image;

[0009] Calculate the correlation coefficient between the base channel and at least one other single channel in the stitched image;

[0010] Obtain multiple sets of unit splicing coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients. Perform polynomial fitting based on the multiple sets of unit splicing coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients. Determine the optimal unit splicing coefficient based on the polynomial fitting results.

[0011] In addition, the image stitching coefficient determination method according to the above embodiments of the present invention may also have the following additional technical features:

[0012] According to some embodiments of the present invention, using the stitching coefficient of a single channel adjacent to the base channel as the unit stitching coefficient, all other single channels are stitched onto the base channel to obtain the corresponding stitched image, including:

[0013] Determine the location of the base channel and the relative positions of all other single channels to the base channel;

[0014] Based on relative position and unit splicing coefficient, determine the channel splicing coefficient between other single channels and the basic channel;

[0015] All other single channels are stitched onto the base channel according to the channel stitching coefficient to obtain a stitched image.

[0016] According to some embodiments of the present invention, all other single channels are spliced ​​onto the basic channel according to the channel splicing coefficient, including:

[0017] Each single channel is moved N unit pixels towards the base channel according to its corresponding channel splicing coefficient; where the unit pixel distance is the moving pixel distance corresponding to the unit splicing coefficient, and there is an interval of (N-1) other single channels between the single channel and the base channel, where N is a positive integer greater than or equal to 1.

[0018] According to some embodiments of the present invention, calculating the correlation coefficient between a basic channel and at least one other single channel in a stitched image further includes:

[0019] Based on the original components of multiple single channels, the change points of image stitching are located, and the correlation coefficient is calculated only for the change points.

[0020] According to some embodiments of the present invention, based on multiple single-channel raw components, locating the change points of image stitching includes:

[0021] The original components of multiple single channels are subjected to median filtering to obtain the single-channel components after median filtering.

[0022] Calculate the difference between the original components and the single-channel components of multiple single channels to obtain the difference component of each single channel;

[0023] Calculate the standard deviation of the difference components for each single channel, and retain the largest standard deviation std;

[0024] Initialize a binarized image of the same size as the stitched image. Iterate through all pixels in the binarized image. If the value of any single-channel component of a pixel is greater than N×std, set the pixel value of the pixel to 1; otherwise, set the pixel value to 0 to obtain the filtered image. Here, N is a preset empirical value.

[0025] According to some embodiments of the present invention, the method further includes:

[0026] Perform at least one morphological dilation operation on the filtered image to obtain a mask image; where pixels with a value of 1 in the mask image are the points of change in the stitched image.

[0027] According to some embodiments of the present invention, calculating the correlation coefficient between a basic channel and at least one other single channel in a stitched image includes:

[0028] The corresponding basic channel component x and the selected single channel component y to be calculated for the correlation coefficient are obtained from the stitched image.

[0029] The correlation coefficient between the base channel and the selected single channel is calculated using the correlation coefficient calculation formula, under a unit splicing coefficient. The formula is as follows:

[0030]

[0031] in, These are the average values ​​of the base channel components x and the average values ​​of the selected single channel components y for all pixels that need to be calculated in the current stitched image, respectively.

[0032] According to some embodiments of the present invention, multiple sets of unit splicing coefficients and corresponding correlation coefficients are obtained, and polynomial fitting is performed based on the multiple sets of unit splicing coefficients and corresponding correlation coefficients, including:

[0033] Calculate the correlation coefficients of the base channel and the selected single channel under different unit splicing coefficients;

[0034] The quadratic polynomial relationship between the unit splicing coefficient 'a' and the correlation coefficient 'b' is obtained by fitting using the least squares method, namely:

[0035] b = p1x 2 +p2x+p3

[0036] According to some embodiments of the present invention, determining the optimal splicing coefficients based on polynomial fitting results includes:

[0037] Find the extreme value of the fitted binomial to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels with the basic channel are calculated, then the average value of the multiple unit splicing coefficients a is taken as the optimal unit splicing coefficient.

[0038] According to an embodiment of the present invention, an image stitching coefficient determination method is provided. A base channel for image stitching is determined from multiple single channels pre-acquired from a line scan camera. The stitching coefficient of the single channel adjacent to the base channel is used as the unit stitching coefficient. All other single channels are stitched onto the base channel to obtain the corresponding stitched image. The correlation coefficient between the base channel and at least one other single channel in the stitched image is calculated. Multiple sets of unit stitching coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit stitching coefficients are obtained. Polynomial fitting is performed based on the multiple sets of unit stitching coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit stitching coefficients. The optimal unit stitching coefficient is determined based on the polynomial fitting result. This embodiment of the present invention, by calculating the correlation coefficient of the single channel of the line scan camera and performing polynomial fitting, can determine the optimal unit stitching coefficient for a single channel, thereby improving the accuracy and quality of image stitching and ensuring that the stitched image is visually more coherent and consistent.

[0039] To achieve the above objectives, a second aspect of the present invention provides an image stitching method based on a line scan camera, comprising:

[0040] Obtain the single-channel parameters corresponding to the target object captured by the linear scan camera;

[0041] Determine the base channels used for image stitching;

[0042] Determine the splicing coefficients for splicing other single channels to the basic channel;

[0043] The stitched image of the target object is obtained by stitching the images together according to the stitching coefficients of each single channel; the resolution of the stitched image is higher than that of the initial image captured by the line scan camera.

[0044] According to some embodiments of the present invention, the splicing coefficients corresponding to splicing other single channels to the basic channel are determined respectively, including:

[0045] The splicing coefficient of the single channel adjacent to the basic channel is used as the unit splicing coefficient, and the splicing coefficient of other single channels splicing to the basic channel is determined.

[0046] Image stitching was performed under multiple sets of different unit stitching coefficients;

[0047] Select at least one single channel as the selected single channel;

[0048] Calculate the correlation coefficient between the base channel and the selected single channel for each set of stitched images;

[0049] Based on the numerical relationship between multiple sets of unit splicing coefficients and their corresponding correlation coefficients, the optimal unit splicing coefficient is found.

[0050] Based on the optimal unit splicing coefficient, determine the channel splicing coefficient for splicing other single channels to the basic channel.

[0051] According to some embodiments of the present invention, the splicing coefficient of a single channel adjacent to the base channel is used as the unit splicing coefficient, and the channel splicing coefficients of other single channels splicing to the base channel are determined, including:

[0052] Determine the location of the base channel and the relative positions of all other single channels to the base channel;

[0053] Based on relative position and unit splicing coefficient, determine the channel splicing coefficient for splicing other single channels to the basic channel;

[0054] Each single channel is moved N unit pixels towards the base channel according to its corresponding channel splicing coefficient; where the unit pixel distance is the moving pixel distance corresponding to the unit splicing coefficient, and there is an interval of (N-1) other single channels between the single channel and the base channel, where N is a positive integer greater than or equal to 1.

[0055] According to some embodiments of the present invention, calculating the correlation coefficient between the base channel and the selected single channel in each group of stitched images further includes:

[0056] Based on the original components of multiple single channels, the change points of image stitching are located, and the correlation coefficient is calculated only for the change points;

[0057] Median filtering is performed on the original components of multiple single channels to obtain the single-channel components after median filtering.

[0058] Calculate the difference between the original components of multiple single channels and the single-channel components to obtain the single-channel difference components;

[0059] Calculate the standard deviation of each single-channel difference component and retain the largest standard deviation (std).

[0060] Initialize a binarized image of the same size as the stitched image. Iterate through all pixels in the binarized image. If the value of any single-channel component of a pixel is greater than N×std, set the pixel value of the pixel to 1; otherwise, set the pixel value to 0 to obtain the filtered image. Here, N is a preset empirical value.

[0061] Perform at least one morphological dilation operation on the filtered image to obtain a mask image; where pixels with a value of 1 in the mask image are the points of change in the stitched image.

[0062] According to some embodiments of the present invention, calculating the correlation coefficient between the base channel and the selected single channel in each group of stitched images further includes:

[0063] The corresponding basic channel component x and the selected single channel component y to be calculated for the correlation coefficient are obtained from the stitched image.

[0064] The correlation coefficient between the base channel and the selected single channel is calculated using the correlation coefficient calculation formula, under a unit splicing coefficient. The formula is as follows:

[0065]

[0066] in, These are the average values ​​of the base channel components x and the average values ​​of the selected single channel components y for all pixels that need to be calculated in the current stitched image, respectively.

[0067] According to some embodiments of the present invention, the optimal unit splicing coefficient is found based on the numerical relationship between multiple sets of unit splicing coefficients and their corresponding correlation coefficients, including:

[0068] Calculate the correlation coefficients of the base channel and the selected single channel under different unit splicing coefficients;

[0069] The quadratic polynomial relationship between the unit splicing coefficient 'a' and the correlation coefficient 'b' is obtained by fitting using the least squares method, namely:

[0070] b = p1x 2 +p2x+p3

[0071] Find the extreme value of the fitted binomial to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels with the basic channel are calculated, then the average value of the multiple unit splicing coefficients a is taken as the optimal unit splicing coefficient.

[0072] According to an embodiment of the present invention, an image stitching method based on a line scan camera is provided. First, the single-channel parameters corresponding to the target object captured by the line scan camera are obtained. Further, a base channel for image stitching is determined. Then, stitching coefficients corresponding to stitching other single channels to the base channel are determined. Finally, the stitched image of the target object is obtained by stitching according to the stitching coefficients of each single channel. The resolution of the stitched image is higher than that of the initial image captured by the line scan camera. This embodiment of the present invention optimizes the unit stitching coefficient of the single channel of the line scan camera and stitches the image according to the optimized unit stitching coefficient to obtain a higher resolution image, thereby effectively improving the accuracy of material identification by the color sorter and increasing the accuracy and efficiency of the color sorter.

[0073] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This is a schematic diagram illustrating the image acquisition principle of a color three-line array camera provided in an embodiment of the present invention;

[0076] Figure 2 This is a schematic diagram of an image obtained by directly stitching together data acquired by a color three-line array camera according to an embodiment of the present invention.

[0077] Figure 3 This is a flowchart of the image stitching coefficient determination method provided in an embodiment of the present invention;

[0078] Figure 4 A schematic diagram of the channels of a six-channel line scan camera provided in an embodiment of the present invention;

[0079] Figure 5 This is a schematic diagram of the change point process for positioning image stitching provided in an embodiment of the present invention;

[0080] Figure 6 A schematic diagram of the R, G, and B channel components before median filtering processing provided in an embodiment of the present invention;

[0081] Figure 7 A schematic diagram showing the median filtering process applied to the R, G, and B channel components provided in an embodiment of the present invention.

[0082] Figure 8 A schematic diagram of the difference components corresponding to the R, G, and B channel components provided in an embodiment of the present invention;

[0083] Figure 9 This is a schematic diagram of a filtered image provided in an embodiment of the present invention;

[0084] Figure 10 This is a schematic diagram of a mask image provided in an embodiment of the present invention;

[0085] Figure 11 This is a schematic diagram of the image stitching coefficient determination device provided in an embodiment of the present invention;

[0086] Figure 12A more specific schematic diagram of the hardware structure of an electronic device is provided for an embodiment of the present invention;

[0087] Figure 13 This is a flowchart of an image stitching method based on a line scan camera provided in an embodiment of the present invention;

[0088] Figure 14 This is a schematic diagram illustrating the process of determining the splicing coefficients corresponding to splicing other single channels to the basic channel, provided in an embodiment of the present invention.

[0089] Figure 15 This is an image obtained by stitching together according to the optimal stitching coefficient, as provided in an embodiment of the present invention.

[0090] Figure 16 This is a schematic diagram of an image stitching device based on a line scan camera provided in an embodiment of the present invention;

[0091] Figure 17 This is a schematic diagram of a more specific electronic device hardware structure provided for an embodiment of the present invention. Detailed Implementation

[0092] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0093] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0094] As described in the background section, when the line scan camera of a color sorter photographs and distinguishes materials on the production line, the materials to be sorted have large differences in density. When materials of different densities pass through the color sorter at a uniform speed, their running speeds may differ, resulting in color distortion in the images captured by the line scan camera. Technicians usually need to repeatedly adjust the splicing coefficient for different materials to ensure that the line scan camera can capture clear material photos for more accurate identification and sorting. On the production line, manual adjustment is difficult and inefficient, and requires a high level of technical expertise, limiting the automation and intelligence level of the production line and hindering the long-term use of the color sorter.

[0095] In the process of realizing this invention, the applicant discovered that by calculating the correlation coefficient and polynomial fitting of a single channel of a linear array camera, the optimal unit stitching coefficient for a single channel can be determined. By using the optimized unit stitching coefficient for image stitching, the accuracy and quality of image stitching can be improved, ensuring that the stitched image is more visually coherent and consistent.

[0096] The following detailed description of the image stitching coefficient determination method of the present invention will be provided through specific embodiments.

[0097] A line scan camera's sensor is linear, responsible for receiving light and converting it into electrical signals. In this process, different parts or areas of the sensor can be considered different "channels," each responsible for capturing a portion of the image information. Line scan cameras capture light point by point and convert it into electrical signals by scanning line by line with the linear sensor. These electrical signals are then converted into digital signals for image reconstruction and processing. Each channel plays a crucial role in ensuring the integrity and accuracy of the image information. Line scan cameras are typically configured with 1, 2, 3, 4, 6, 8, or 16 channels. During image capture, each channel forms multiple rows of sampling positions on the same facing plane, with equal intervals between the rows. For color imaging, a three-channel color line scan camera is generally used, with the channels representing red (R), green (G), and blue (B) channels, which form the basis of a color image. In line scan camera applications, if the camera is equipped with an RGB sensor, each channel (red, green, blue) will capture one color component of the image.

[0098] A color three-line array camera is based on a three-line array CCD (Charge-Coupled Device) sensor, which is arranged in three linear arrays, representing the red (R), green (G), and blue (B) channels. Each linear array observes the object at a different angle (or at a different time, which can be understood as different scan lines for a static object), thereby capturing the object's three color components: R, G, and B. However, due to the special structure of the three-line array camera, the R, G, and B components acquired at the same moment do not correspond to the same point on the object.

[0099] like Figure 1 The diagram shows the principle of image acquisition using a color three-line array camera. It can be seen that the R, G, and B components acquired at the same time do not correspond to the same point on the object. The captured images need to be stitched together using strict stitching coefficients to form an image with normal color.

[0100] Color sorters separate materials with significant density differences, such as high-density materials (e.g., ores) and low-density materials (e.g., tea leaves). Because of these density differences, the materials pass through the color sorter at different speeds, which can cause distortion in the images taken during the sorting process. Figure 2 The image shown is a schematic diagram of an image obtained by directly stitching together data collected by a color three-line array camera. It can be seen that the color distortion at the material edges is significant, thus affecting the recognition accuracy of subsequent sorting algorithms.

[0101] refer to Figure 3 The above is a flowchart of the image stitching coefficient determination method provided in an embodiment of the present invention.

[0102] The base channel for image stitching is determined from multiple single channels pre-acquired from the line scan camera;

[0103] Using the splicing coefficient of the single channel adjacent to the base channel as the unit splicing coefficient, all other single channels are spliced ​​onto the base channel to obtain the corresponding spliced ​​image;

[0104] Calculate the correlation coefficient between the basic channel and at least one other single channel in the stitched image;

[0105] Obtain multiple sets of unit splicing coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients. Perform polynomial fitting based on the multiple sets of unit splicing coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients. Determine the optimal unit splicing coefficient based on the polynomial fitting result.

[0106] This embodiment determines the image stitching scheme for each channel after identifying the basic channels. Then, based on the assumed unit stitching coefficient, it completes the data processing and stitching of each channel to obtain the corresponding stitched image. By evaluating the quality of the stitched image, it obtains the relationship between the unit stitching coefficient and the quality of the stitched image (correlation coefficient), obtains multiple sets of such relationship values, obtains the numerical relationship between the two parameters based on polynomial fitting, and determines the optimal unit stitching coefficient based on the fitting results. This allows for the evaluation of suitable optimal stitching coefficients for different materials, avoiding the problems of low efficiency and unquantifiable results when adjusting stitching parameters based on manual experience, thus improving imaging quality.

[0107] Specifically, the image stitching coefficient determination method provided in this implementation includes:

[0108] Step S301: Determine the base channel for image stitching from multiple single channels pre-acquired from the line scan camera.

[0109] First, from the multiple single channels acquired by the line scan camera, one is selected as the basis for image stitching. This base channel will serve as the reference point for stitching, and the other channels will be stitched around it.

[0110] For example, if a color three-line array camera captures data from the R, G, and B channels respectively, these data can represent the intensity information of red, green, and blue in the image. Any one of these three channels can be chosen as the base channel. Generally, because the green channel (G) is the most sensitive to the human eye and contains the most image detail, it is preferred as the base channel. After obtaining the base channel, the data from other channels (such as the R and B channels) needs to be concatenated onto it, referring to... Figure 1 It's known that the objects acquired by the R, G, and B channels have gaps. If the data from these three channels are directly stitched together, it's like stitching together image data from three different locations to form an image, which can result in blurriness. Stitching data from other channels onto the base channels aims to adjust the data from each channel to its original position. In the actual image stitching process, it's necessary to acquire the parameters of the line scan camera over a certain time period, thus forming a specific area. The stitching process can be considered as shifting the data from other channels along the time axis before stitching the images together.

[0111] Step S302: Using the splicing coefficient of the single channel adjacent to the basic channel as the unit splicing coefficient, all other single channels are spliced ​​onto the basic channel to obtain the corresponding spliced ​​image.

[0112] Specifically, the unit splicing factor refers to the single channel directly adjacent to the base channel (for example, if the base channel is green channel G, then the unit splicing factor may refer to the splicing factor between green channel G and red channel R or blue channel B, depending on which channel is directly adjacent to green channel G).

[0113] As an optional embodiment, the stitching coefficient of the single channel adjacent to the base channel is used as the unit stitching coefficient. All other single channels are stitched onto the base channel to obtain the corresponding stitched image. This includes: determining the position of the base channel and the relative positions of all other single channels relative to the base channel; determining the channel stitching coefficient between the other single channels and the base channel based on the relative positions and the unit stitching coefficient; and stitching all other single channels onto the base channel according to the channel stitching coefficient to obtain the stitched image.

[0114] Specifically, the sensors of a color three-line scan camera are typically arranged in a specific pattern, such as RGB linear arrangement, BGR linear arrangement, or staggered arrangement. In the image data structure, each channel has a specific position or index. For the base channel, its position is known, usually at the beginning of the data structure or at a specific location. Relative position refers to the positional relationship of other single channels relative to the base channel within the image data structure. For a line scan camera, the physical spacing between adjacent channels is consistent on the plane of the object directly in front of the camera. Therefore, when determining the stitching coefficients of other channels based on the base stitching coefficients, only the relative positional relationships of all channels need to be known.

[0115] For example, for Figure 1 The three-channel line scan camera shown uses the R channel as the base channel. The stitching coefficient of the B channel to the R channel is defined as the unit stitching coefficient. Then the stitching coefficient of the G channel to the R channel is twice the unit stitching coefficient.

[0116] As an optional embodiment, all other single channels are spliced ​​onto the base channel according to the channel splicing coefficient, including: moving each single channel toward the base channel by N unit pixel distances according to its corresponding channel splicing coefficient; wherein, the unit pixel distance is the moving pixel distance corresponding to the unit splicing coefficient, and there is an interval of (N-1) other single channels between the single channel and the base channel, where N is a positive integer greater than or equal to 1.

[0117] For example Figure 4The six-channel line scan camera shown uses the second channel 22 as the base channel and the stitching coefficient of the third channel 23 stitching to the base channel 22 as the unit stitching coefficient 'a'. Since the fourth channel 24 is separated from the base channel 22 by another channel (the third channel 23), its channel stitching coefficient is 2a. Therefore, the channel stitching coefficient of the fifth channel 25 is 3a, and the channel stitching coefficient of the sixth channel 26 is 4a. The first channel 21 is located on both sides of the base channel 22 with the third channel 23, so its channel stitching coefficient is still 'a' in numerical value, but considering the opposite direction, it should be set to -a in the calculation.

[0118] Specifically, the unit pixel distance refers to the distance of the moving pixels corresponding to the unit stitching coefficient. The specific value of this distance does not need to be calculated accurately. This definition is introduced in this embodiment only to facilitate the understanding of the technical solution. In actual processing, it is not necessary to consider the specific value of the pixel distance. It is only necessary to evaluate the quality of the processed image and judge whether the value of the unit stitching coefficient is appropriate.

[0119] Step S303: Calculate the correlation coefficient between the basic channel and at least one other single channel in the stitched image.

[0120] The correlation coefficient is a statistic that measures the strength and direction of the linear relationship between two single-channel components. In image processing, calculating the correlation coefficient between the base channel and other single channels can determine the similarity and differences between single channels. Step S304 involves obtaining multiple sets of unit stitching coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit stitching coefficients, performing polynomial fitting based on the multiple sets of unit stitching coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit stitching coefficients, and determining the optimal unit stitching coefficient based on the polynomial fitting result.

[0121] In this embodiment of the invention, multiple sets of unit stitching coefficients can be collected or generated. These coefficients characterize the stitching effect under different parameter settings in the image stitching task, or represent other parameter combinations that need to be optimized.

[0122] For each set of unit concatenation coefficients, a correlation coefficient was obtained. Using the collected multiple sets of unit concatenation coefficients and correlation coefficients as data points, polynomial fitting was performed. Polynomial fitting is a mathematical method used to find a set of polynomial coefficients that best describe the relationship between data points by minimizing the error. Based on the results of the polynomial fitting, the unit concatenation coefficients that maximize the correlation coefficient (or minimize a certain error metric) can be found. This coefficient can be considered the optimal unit concatenation coefficient.

[0123] Specifically, calculating the correlation coefficient between the base channel and at least one other single channel in the stitched image includes: determining the corresponding base channel component x and the selected single channel component y for which the correlation coefficient is to be calculated based on the stitched image; and calculating the correlation coefficient between the base channel and the selected single channel under a unit stitching coefficient using the correlation coefficient calculation formula, which is:

[0124]

[0125] in, These are the average values ​​of the base channel components x and the average values ​​of the selected single channel components y for all pixels that need to be calculated in the current stitched image, respectively.

[0126] Specifically, the basic channel component x and selected single-channel component y are extracted from the stitched image. These components can represent different color channels (such as red, green, and blue) or other feature channels of the image. Further, the average value x of the basic channel component x and the average value y of the selected single-channel component y are calculated for all pixels in the current stitched image that require calculation. The average value can be obtained by summing the corresponding channel components of all pixels and then dividing by the total number of pixels. The correlation coefficient is calculated using the correlation coefficient formula between the base channel and the selected single channel. The correlation coefficient is a statistic that measures the degree of linear correlation between two variables, with values ​​ranging from -1 to 1. The closer the correlation coefficient is to 1, the stronger the linear relationship (positive correlation) between the base channel and the selected single channel; the closer it is to -1, the stronger the linear relationship (negative correlation); and close to 0, there is almost no linear relationship.

[0127] As an optional embodiment, multiple sets of unit splicing coefficients and corresponding correlation coefficients are obtained. Polynomial fitting is then performed based on these values, including: calculating the correlation coefficients of the corresponding base channel and selected single channel under different unit splicing coefficients; and fitting a quadratic polynomial relationship between the unit splicing coefficient a and the correlation coefficient b using the least squares method, i.e.:

[0128] b = p1a 2 +p2a+p3

[0129] Taking the R channel as the base channel and the G channel as examples, a is the unit splicing coefficient, b is the correlation coefficient between the corresponding R channel and G channel components, and q1, q2, and q3 are the coefficients of the quadratic, linear, and zero-order terms in the formula, respectively.

[0130] As an optional embodiment, the optimal splicing coefficient is determined based on the polynomial fitting result, including: finding the extreme value of the fitted binomial to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b; if the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is calculated as the optimal unit splicing coefficient.

[0131] Quadratic polynomials were fitted to the correlation coefficients of the R-channel and G-channel, and the R-channel and B-channel components, respectively, and the maximum values ​​and corresponding splicing coefficients were solved. The specific solution process is as follows:

[0132] The least squares method is used to fit a quadratic polynomial with the concatenation coefficient as the variable. The derivation of the formula for fitting the quadratic polynomial using the least squares method is as follows:

[0133] Let a1, a2, ..., a n These are the splicing coefficients, b1, b2, ..., b n For the corresponding correlation coefficient, the fitted quadratic polynomial is b = p1a 2 +p2a+p3, then the mean square error is E=∑ ( b i -b i ′ ) 2 =∑(b i -p1a i2 -p2a i -p3), taking its partial derivative is: By setting the three partial derivatives to zero, we can find p1, p2, and p3, and the maximum value is... The splicing coefficient corresponding to the maximum value is

[0134] The final splicing coefficient is the average of the splicing coefficients corresponding to the maximum correlation coefficients of the R-channel and G-channel, and the R-channel and B-channel components, as shown in the following formula:

[0135]

[0136] Among them, offset RG The splicing coefficients obtained for the R and G channel components, offset RB The splicing coefficients are obtained for the R and B channel components.

[0137] Based on the calculated splicing coefficient offset, the R channel component remains unchanged (R channel is the base channel), the B channel component is moved by offset, and the G channel component is moved by 2× offset, and then spliced ​​into a new image.

[0138] If offset is a decimal, the concatenation formula is as follows:

[0139]

[0140] Where img'(i,j) represents the image to be stitched together at position (i,j), img(i,j) represents the image to be stitched together at position (i,j), and {*} indicates taking the decimal part. Indicates rounding down. This indicates rounding up to the nearest integer.

[0141] Furthermore, the calculation of the correlation coefficient involves all pixels, resulting in a large computational load, high computing power requirements, and slow program execution speed. Additionally, in color sorter applications, the material area in the image typically does not constitute a large proportion of the conveyor belt background area. This causes the values ​​of background pixels to lead to a higher-than-expected correlation coefficient, affecting the calculation of the final unit splicing coefficient.

[0142] As an optional embodiment, the calculation of the correlation coefficient between the basic channel and at least one other single channel in the stitched image further includes: locating the change points of the image stitching based on the original components of multiple single channels, and calculating the correlation coefficient only for the change points.

[0143] The change points in image stitching refer to the locations where discontinuities or abrupt changes may occur during the stitching process. In this embodiment, to reduce the amount of data computation, it is preferable to calculate the correlation coefficient for the change points and perform subsequent stitching processing, thereby obtaining more accurate correlation coefficients.

[0144] refer to Figure 5 This is a schematic diagram of the change point process for positioning image stitching provided in an embodiment of the present invention.

[0145] As an optional embodiment, based on multiple single-channel raw components, the change points of image stitching are located, including:

[0146] Step S501: Perform median filtering on the original components of multiple single channels to obtain median-filtered single-channel components of multiple single channels.

[0147] Step S502: Calculate the difference between the original components and the single-channel components of multiple single channels to obtain the difference component of each single channel.

[0148] Step S503: Calculate the standard deviation of the difference components of each single channel and retain the largest standard deviation std;

[0149] Step S504: Initialize a binarized image of the same size as the stitched image, traverse all pixels in the binarized image, and set the pixel value of the pixel to 1 if the value of any single-channel component of the pixel is greater than N×std; otherwise, set the pixel value to 0 to obtain the filtered image; where N is a preset empirical value.

[0150] Based on the above steps, a binary image can be obtained, where pixels with a value of 1 can be considered as points of change in the image stitching process. The correlation coefficients of the basic channels and selected single channels of the relevant pixels are calculated only, thereby obtaining a more accurate value that reflects the correlation between the two channels.

[0151] First, median filtering is applied to multiple single channels (including the base channel and other single channels) in the stitched image. Median filtering is a non-linear filtering technique that can effectively remove noise (such as salt-and-pepper noise and random noise) from the image while preserving the image's edge information.

[0152] refer to Figure 6 This is a schematic diagram showing the R, G, and B channel components before median filtering. Figure 6 (a) is a schematic diagram of the R channel components before median filtering. Figure 6 (b) is a schematic diagram of the G channel components before median filtering. Figure 6 (c) is a schematic diagram of the B channel components before median filtering. One of the main advantages of median filtering is its ability to effectively remove noise from an image. Without median filtering, this noise may remain in the image, affecting its visual quality and the effectiveness of subsequent processing.

[0153] refer to Figure 7 This is a schematic diagram showing the median filtering process applied to the R, G, and B channel components. Figure 7 (a) is a schematic diagram of the median-filtered component corresponding to the R channel; (b) is a schematic diagram of the median-filtered component corresponding to the G channel; and (c) is a schematic diagram of the median-filtered component corresponding to the B channel. Median filtering yields the median-filtered component for each single channel. Figure 7 (a) Figure 7 (b) and Figure 7 (c) Compared to 6(a), respectively Figure 6 (b) and Figure 6 (c) Remove noise from the image while preserving the image's edge information.

[0154] Furthermore, the difference between the original component and the median-filtered component of each single channel is calculated to obtain the difference component for each single channel. (Reference) Figure 8This is a schematic diagram of the difference components corresponding to the R, G, and B channel components. Figure 8 (a) is a schematic diagram of the difference components corresponding to the R channel components. Figure 8 (b) is a schematic diagram of the difference components corresponding to the G channel components. Figure 8 (c) is a schematic diagram of the difference components corresponding to the B channel components. The difference components reflect the differences between the original image and the filtered image, containing information about the image stitching change points. Then, the standard deviation of the difference components for each single channel is calculated. The standard deviation is a statistic that measures the dispersion of data; here, it helps identify which single-channel difference components contain more variation information. The single-channel difference component with the largest standard deviation is retained because it is most likely to contain the main change points of the image stitching.

[0155] As an optional embodiment, to more intuitively locate change points, a binarized image of the same size as the stitched image can be initialized. Then, all pixels in this binarized image are traversed, and the value of any single-channel component (usually the single-channel difference component corresponding to the previously retained maximum standard deviation) of each pixel is checked. If the value of this single-channel component of a pixel is greater than a preset empirical value N (N is usually set according to the specific image and processing requirements), the pixel value of that pixel is set to 1 (indicating a change point); otherwise, it is set to 0 (indicating a non-change point). This yields a filtered image containing only the main change points of the image stitching. Specifically, a binarized image of the same size as the initial image is created, and the comparison value is determined according to the standard deviation std, where N is an integer, and in this embodiment, N is preferably 5.

[0156] refer to Figure 9 This is a schematic diagram of the filtered image. The white areas (pixel value of 1) in the binarized image represent pixels in the initial image that have significant fluctuations in any single-channel component. These pixels may be edges, textures, or noise in the image when stitching based on the initial stitching coefficients. The black areas (pixel value of 0) in the binarized image represent pixels in the image that have no fluctuations or negligible fluctuations in any single-channel component. These pixels are smooth areas or background in the image when stitching based on the initial stitching coefficients.

[0157] Furthermore, to obtain more accurate correlation coefficient values, the neighboring pixels of each pixel in the binarized image can be used as change points for calculation, or the pixels can be amplified in other ways. For example, as an optional embodiment, the filtered image can be subjected to at least one morphological dilation operation to obtain a mask image; wherein, the pixels with a pixel value of 1 in the dilated mask image are the change points of the image.

[0158] The filtered image is subjected to at least one morphological dilation operation. Morphological dilation is a non-linear operation typically used to enlarge bright areas or foreground objects in an image. It is achieved by sliding a structuring element (usually a small rectangle or disk) across the image and assigning the maximum value of the area covered by the structuring element to the pixel at the center of the structuring element. After morphological dilation, certain areas in the image may become brighter or more prominent. A mask image is then generated based on the dilated image. In this mask image, pixels with a value of 1 represent the points of change in the stitched image, i.e., those areas that have become more prominent due to morphological dilation.

[0159] Specifically, a circle with a structuring element radius of d (preferably 5) is used to morphologically dilate the binarized image, that is, the white area in the target binarized image is expanded by a radius of 5, resulting in the following: Figure 10 The image shown is an inflated mask image, where pixels in the mask image have a pixel value of 1.

[0160] According to an embodiment of the present invention, an image stitching coefficient determination method is provided. A base channel for image stitching is determined from multiple single channels pre-acquired from a line scan camera. The stitching scheme for the entire stitched image is determined based on the position of the base channel. The stitching coefficient of the single channel adjacent to the base channel is used as the unit stitching coefficient. All other single channels are stitched onto the base channel to obtain the corresponding stitched image. The correlation coefficient between the base channel and at least one other single channel in the stitched image is calculated. Multiple sets of unit stitching coefficients and multiple sets of correlation coefficients corresponding to these unit stitching coefficients are obtained. Polynomial fitting is performed based on these multiple sets of unit stitching coefficients and their corresponding correlation coefficients. The optimal unit stitching coefficient is determined based on the polynomial fitting result. This embodiment of the present invention, by calculating the correlation coefficient of the single channel of the line scan camera and performing polynomial fitting, can determine the optimal unit stitching coefficient for a single channel, thereby improving the accuracy and quality of image stitching and ensuring that the stitched image is visually more coherent and consistent.

[0161] It should be noted that the method of this embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this embodiment, and the multiple devices will interact with each other to complete the method described.

[0162] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] Based on the same inventive concept, corresponding to the methods provided in any of the above embodiments, the present invention also provides an image stitching coefficient determination device.

[0164] refer to Figure 11 This is a schematic diagram of an image stitching coefficient determination device provided in an embodiment of the present invention.

[0165] The image stitching coefficient determination device includes: an acquisition module 1110, a stitching module 1120, a calculation module 1130, and a determination module 1140.

[0166] The acquisition module 1111 is configured to determine a basic channel for image stitching from a plurality of single channels pre-acquired from a line scan camera;

[0167] The stitching module 1120 is configured to use the stitching coefficient of the single channel adjacent to the basic channel as the unit stitching coefficient to stitch all other single channels onto the basic channel to obtain the corresponding stitched image.

[0168] The calculation module 1130 is configured to calculate the correlation coefficient between the basic channel and at least one other single channel in the stitched image;

[0169] The determining module 1140 is configured to acquire multiple sets of unit splicing coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients, perform polynomial fitting based on the multiple sets of unit splicing coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients, and determine the optimal unit splicing coefficients based on the polynomial fitting results.

[0170] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this invention, the functions of each module can be implemented in one or more software and / or hardware components.

[0171] The image stitching coefficient determination device of the above embodiments is used to implement the corresponding image stitching coefficient determination method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0172] Based on the same inventive concept, corresponding to the image stitching coefficient determination method described in any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image stitching coefficient determination method described in any of the above embodiments.

[0173] Figure 12 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1210, a memory 1220, an input / output interface 1230, a communication interface 1240, and a bus 1250. The processor 1210, memory 1220, input / output interface 1230, and communication interface 1240 are interconnected internally via the bus 1250.

[0174] The processor 1210 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0175] The memory 1220 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1220 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1220 and is called and executed by the processor 1210.

[0176] Input / output interface 1230 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0177] The communication interface 1240 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0178] Bus 1250 includes a pathway for transmitting information between various components of the device, such as processor 1210, memory 1220, input / output interface 1230, and communication interface 1240.

[0179] It should be noted that although the above-described device only shows the processor 1210, memory 1220, input / output interface 1230, communication interface 1240, and bus 1250, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0180] The electronic devices described above are used to implement the corresponding image stitching coefficient determination method in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0181] Based on the same inventive concept, corresponding to the image stitching coefficient determination method described in any of the above embodiments, the present invention also provides a computer-readable storage medium storing computer instructions for causing the computer to execute the image stitching coefficient determination method as described in any of the above embodiments.

[0182] The aforementioned computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0183] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the image stitching coefficient determination method as described in any of the embodiments in the exemplary method section above, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0184] The following detailed description of the image stitching method based on a line scan camera according to the present invention will be provided through specific embodiments.

[0185] refer to Figure 13 This is a flowchart of an image stitching method based on a line scan camera provided in an embodiment of the present invention.

[0186] Obtain the single-channel parameters corresponding to the target object captured by the linear scan camera;

[0187] Determine the base channels used for image stitching;

[0188] Determine the splicing coefficients for splicing other single channels to the basic channel;

[0189] The stitched image of the target object is obtained by stitching the images together according to the stitching coefficients of each single channel; wherein the resolution of the stitched image is higher than that of the initial image captured by the line scan camera.

[0190] This embodiment determines the stitching scheme of each channel by selecting a basic channel, and then determines the specific stitching coefficient of each channel, thereby completing the pattern stitching of the line scan camera. This results in a clear stitched image, which is beneficial for product identification and sorting based on the image.

[0191] Furthermore, the image stitching method based on a line scan camera provided in this embodiment includes:

[0192] Step S1301: Obtain the single-channel parameters corresponding to the target object captured by the line scan camera.

[0193] First, a line scan camera can be used to photograph the target object and acquire the resulting single-channel images or data. These single channels may represent different color channels (such as red, green, and blue) or other specific image feature channels.

[0194] Step S1302: Determine the base channel used for image stitching.

[0195] First, one of the multiple single channels acquired from the line scan camera can be selected as the base for image stitching. This base channel will serve as the reference point for stitching, and the other channels will be stitched around it. For example, in this embodiment, the R channel is used as the base channel, and the data from the G and B channels are processed separately to be stitched onto the R channel to obtain the stitched image.

[0196] Step S1303: Determine the splicing coefficients for splicing other single channels to the basic channel.

[0197] For details, please refer to the appendix. Figure 14 This is a schematic diagram of the process for determining the splicing coefficients corresponding to splicing other single channels to the basic channel, provided by an embodiment of the present invention.

[0198] The step of determining the splicing coefficients corresponding to splicing other single channels to the basic channel includes:

[0199] Step S1401: Use the single-channel splicing coefficient adjacent to the basic channel as the unit splicing coefficient, and determine the channel splicing coefficients of the other single channels splicing to the basic channel;

[0200] Step S1402: Under multiple sets of different unit splicing coefficients, perform image splicing respectively;

[0201] Step S1403: Select at least one of the single channels as the selected single channel;

[0202] Step S1404: Calculate the correlation coefficient between the base channel and the selected single channel for each group of stitched images;

[0203] Step S1405: Based on the numerical relationship between the multiple sets of unit splicing coefficients and their corresponding correlation coefficients, find the optimal unit splicing coefficient;

[0204] Step S1406: Based on the optimal unit splicing coefficient, determine the channel splicing coefficient for splicing other single channels to the basic channel.

[0205] This embodiment stitches images together under different stitching coefficients and obtains the corresponding correlation coefficients, thereby quantifying the quality of the stitched images. Based on the numerical relationship between the stitching coefficients and the correlation coefficients, the optimal stitching coefficient is determined. This allows for the precise determination of suitable stitching coefficients through data processing, reducing reliance on the experience of on-site debugging personnel, meeting the requirements for data switching in different material scenarios, and improving the flexibility of production line products.

[0206] Specifically, the scheme for image stitching and correlation coefficient calculation using unit stitching coefficients in this embodiment is the same as in the previous embodiments. This embodiment also includes the purpose of highlighting and locating change points during stitching to reduce data processing volume and improve the accuracy of correlation coefficients; the specific scheme will not be elaborated further.

[0207] Step S1304: The target object is stitched together according to the stitching coefficients of each single channel to obtain a stitched image; wherein the resolution of the stitched image is higher than that of the initial image captured by the line scan camera.

[0208] Based on the calculated stitching coefficient offset, the R channel components remain unchanged (R channel is the base channel), the B channel components are shifted by the offset, and the G channel components are shifted by 2 × offset, and then the images are stitched together again to form a single image. Figure 15 As shown. Figure 15 This is the image obtained after stitching according to the optimal stitching coefficients. Compared to... Figure 2 , Figure 15 The material edges are clear and the resolution is high.

[0209] If offset is a decimal, the concatenation formula is as follows:

[0210]

[0211] Where img'(i,j) represents the image to be stitched together at position (i,j), img(i,j) represents the image to be stitched together at position (i,j), and {*} indicates taking the decimal part. This indicates rounding down. This indicates rounding up to the nearest integer.

[0212] According to an embodiment of the present invention, an image stitching method based on a line scan camera is provided. First, the single-channel parameters corresponding to the target object captured by the line scan camera are obtained. Further, a base channel for image stitching is determined. Then, stitching coefficients corresponding to stitching other single channels to the base channel are determined. Finally, the stitched image of the target object is obtained by stitching according to the stitching coefficients of each single channel. The resolution of the stitched image is higher than that of the initial image captured by the line scan camera. This embodiment of the present invention optimizes the unit stitching coefficient of the single channel of the line scan camera and stitches the image according to the optimized unit stitching coefficient to obtain a higher resolution image, thereby effectively improving the accuracy of material identification by the color sorter and increasing the accuracy and efficiency of the color sorter.

[0213] According to an embodiment of the present invention, an image stitching method based on a line scan camera is provided. First, the single-channel parameters corresponding to the target object captured by the line scan camera are obtained. Further, a base channel for image stitching is determined. Then, stitching coefficients corresponding to stitching other single channels to the base channel are determined. Finally, the stitched image of the target object is obtained by stitching according to the stitching coefficients of each single channel. The resolution of the stitched image is higher than that of the initial image captured by the line scan camera. This embodiment of the present invention optimizes the unit stitching coefficient of the single channel of the line scan camera and stitches the image according to the optimized unit stitching coefficient to obtain a higher resolution image, thereby effectively improving the accuracy of material identification by the color sorter and increasing the accuracy and efficiency of the color sorter.

[0214] It should be noted that the image stitching method based on a line scan camera in this embodiment of the invention can be executed by a single device, such as a computer or server. This image stitching method can also be applied in distributed scenarios, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the image stitching method based on a line scan camera in this embodiment of the invention, and these multiple devices will interact with each other to complete the image stitching method based on a line scan camera.

[0215] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0216] Based on the same inventive concept, corresponding to the image stitching method based on a line scan camera provided in any of the above embodiments, the present invention also provides an image stitching coefficient determination device.

[0217] refer to Figure 16 This is a schematic diagram of an image stitching device based on a line scan camera provided in an embodiment of the present invention.

[0218] The image stitching device based on a line scan camera includes: an acquisition module 1610, a first determination module 1620, a second determination module 1630, and a stitching module 1640.

[0219] The acquisition module 1610 is configured to acquire the single-channel parameters corresponding to the target object captured by the line scan camera;

[0220] The first determining module 1620 is configured to determine the base channels used for image stitching;

[0221] The second determining module 1630 is configured to determine the splicing coefficients corresponding to splicing other single channels to the basic channel;

[0222] The stitching module 1640 is configured to stitch together the target object according to the stitching coefficients of each single channel to obtain a stitched image; wherein the resolution of the stitched image is higher than that of the initial image captured by the line scan camera.

[0223] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this invention, the functions of each module can be implemented in one or more software and / or hardware components.

[0224] The apparatus described above is used to implement the corresponding image stitching method based on a line scan camera in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0225] Based on the same inventive concept, corresponding to the image stitching method based on a line scan camera described in any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image stitching method based on a line scan camera described in any of the above embodiments.

[0226] Figure 17This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1710, a memory 1720, an input / output interface 1730, a communication interface 1740, and a bus 1750. The processor 1710, memory 1720, input / output interface 1730, and communication interface 1740 are interconnected internally via the bus 1750.

[0227] The processor 1710 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0228] The memory 1720 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1720 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1720 and is called and executed by the processor 1710.

[0229] The input / output interface 1730 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0230] The communication interface 1740 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0231] Bus 1750 includes a pathway for transmitting information between various components of the device, such as processor 1710, memory 1720, input / output interface 1730, and communication interface 1740.

[0232] It should be noted that although the above-described device only shows the processor 1710, memory 1720, input / output interface 1730, communication interface 1740, and bus 1750, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0233] The electronic devices described above are used to implement the corresponding image stitching methods based on line scan cameras in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0234] Based on the same inventive concept, corresponding to the image stitching method based on a line scan camera described in any of the above embodiments, the present invention also provides a computer-readable storage medium storing computer instructions for causing the computer to execute the image stitching method based on a line scan camera as described in any of the above embodiments.

[0235] The aforementioned computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0236] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the image stitching method based on a line scan camera as described in any of the embodiments in the exemplary method section above, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0237] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowchart may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0238] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0239] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this invention should have the ordinary meaning understood by those skilled in the art. The terms "first," "second," and similar terms used in the embodiments of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0240] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A method for determining image stitching coefficients, characterized in that, include: The base channel for image stitching is determined from multiple single channels pre-acquired from the line scan camera; Using the splicing coefficient of the single channel adjacent to the base channel as the unit splicing coefficient, all other single channels are spliced ​​onto the base channel to obtain the corresponding spliced ​​image; Calculating the correlation coefficient between the basic channel and at least one other single channel in the stitched image further includes: Based on the original components of the multiple single channels, the change points of image stitching are located, and the correlation coefficient is calculated only for the change points, including: The original components of the multiple single channels are subjected to median filtering to obtain the median-filtered single-channel components of the multiple single channels. Calculate the difference between the original components of the plurality of single channels and the single channel components to obtain the difference component of each single channel; Calculate the standard deviation of the difference components for each single channel, and retain the largest standard deviation. ; Initialize a binarized image of the same size as the stitched image. Iterate through all pixels in the binarized image. If the value of any single-channel component of a pixel is greater than a certain threshold, then the pixel is considered a binarized image. If the pixel value is 1, then the pixel value of that pixel is set to 1; otherwise, the pixel value is 0, thus obtaining the filtered image. Here, N is a preset empirical value, and the change point of the stitched image is the pixel with a pixel value of 1 in the mask image. Obtain multiple sets of unit splicing coefficients and multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients. Perform polynomial fitting based on the multiple sets of unit splicing coefficients and the multiple sets of correlation coefficients corresponding to the multiple sets of unit splicing coefficients. Determine the optimal unit splicing coefficient based on the polynomial fitting result.

2. The image stitching coefficient determination method according to claim 1, characterized in that, The step of using the stitching coefficient of a single channel adjacent to the base channel as the unit stitching coefficient to stitch all other single channels onto the base channel to obtain the corresponding stitched image includes: Determine the position of the basic channel and the relative positions of all other single channels relative to the basic channel; Based on the relative position and the unit splicing coefficient, determine the channel splicing coefficient between other single channels and the basic channel; All other single channels are stitched onto the base channel according to the channel stitching coefficient to obtain the stitched image.

3. The method for determining image stitching coefficients according to claim 2, characterized in that, The step of splicing all other single channels onto the base channel according to the channel splicing coefficient includes: Each single channel is moved N unit pixels in the direction of the base channel according to its corresponding channel splicing coefficient; wherein, the unit pixel distance is the moving pixel distance corresponding to the unit splicing coefficient, and the single channel is spaced (N-1) other single channels apart from the base channel, where N is a positive integer greater than or equal to 1.

4. The image stitching coefficient determination method according to claim 1, characterized in that, The method further includes: The filtered image is subjected to at least one morphological dilation operation to obtain the mask image.

5. The image stitching coefficient determination method according to any one of claims 1, characterized in that, The calculation of the correlation coefficient between the basic channel and at least one other single channel in the stitched image includes: Based on the stitched image, the corresponding basic channel component x and the selected single channel component y for which the correlation coefficient is to be calculated are obtained; The correlation coefficient between the base channel and the selected single channel is calculated using the correlation coefficient calculation formula, under a unit splicing coefficient. The formula is as follows: in, , These are the average values ​​of the base channel components x and the average values ​​of the selected single channel components y for all pixels that need to be calculated in the current stitched image, respectively.

6. The image stitching coefficient determination method according to claim 5, characterized in that, The step of obtaining multiple sets of unit splicing coefficients and corresponding correlation coefficients, and performing polynomial fitting based on the values ​​of the multiple sets of unit splicing coefficients and corresponding correlation coefficients, includes: Calculate the correlation coefficients of the corresponding base channel and the selected single channel under different unit splicing coefficients; The quadratic polynomial relationship between the unit splicing coefficient 'a' and the correlation coefficient 'b' is obtained by fitting using the least squares method, namely: 。 7. The method for determining image stitching coefficients according to claim 6, characterized in that, The determination of the optimal splicing coefficients based on the polynomial fitting results includes: Find the extreme value of the fitted binomial to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is taken as the optimal unit splicing coefficient.

8. An image stitching method based on a linear scan camera, characterized in that, include: Obtain the single-channel parameters corresponding to the target object captured by the linear scan camera; Determine the base channels used for image stitching; Determine the splicing coefficients for splicing other single channels to the base channel, including: The splicing coefficient of the single channel adjacent to the basic channel is used as the unit splicing coefficient, and the splicing coefficient of the other single channels splicing to the basic channel is determined. Image stitching was performed under multiple sets of different unit stitching coefficients; Select at least one of the single channels as the selected single channel; Calculating the correlation coefficient between the base channel and the selected single channel for each set of stitched images further includes: Based on the original components of multiple single channels, the change points of image stitching are located, and the correlation coefficient is calculated only for the change points; The change points in the location image stitching include: The original components of the multiple single channels are subjected to median filtering to obtain the median-filtered single-channel components of the multiple single channels. Calculate the difference between the original components of the plurality of single channels and the single channel components to obtain the single channel difference components; Calculate the standard deviation of each single-channel difference component and retain the largest standard deviation. ; Initialize a binarized image of the same size as the stitched image. Iterate through all pixels in the binarized image. If the value of any single-channel component of a pixel is greater than a certain threshold, then the pixel is considered a binarized image. If the pixel value is 1, then the pixel value of that pixel is set to 1; otherwise, the pixel value is 0, thus obtaining the filtered image. Here, N is a preset empirical value, and the change point of the stitched image is the pixel with a pixel value of 1 in the mask image. Based on the numerical relationship between the unit splicing coefficients and their corresponding correlation coefficients, the optimal unit splicing coefficient is found. Based on the optimal unit splicing coefficient, determine the channel splicing coefficients for splicing other single channels to the base channel; The stitched image of the target object is obtained by stitching the images together according to the stitching coefficients of each single channel; wherein the resolution of the stitched image is higher than that of the initial image captured by the line scan camera.

9. The image stitching method based on a linear scan camera according to claim 8, characterized in that, The step of using the single-channel splicing coefficient adjacent to the basic channel as the unit splicing coefficient, and determining the channel splicing coefficients of other single channels spliced ​​to the basic channel, includes: Determine the position of the basic channel and the relative positions of all other single channels relative to the basic channel; Based on the relative position and the unit splicing coefficient, determine the channel splicing coefficient for splicing other single channels to the base channel; Each single channel is moved N unit pixels in the direction of the base channel according to its corresponding channel splicing coefficient; wherein, the unit pixel distance is the moving pixel distance corresponding to the unit splicing coefficient, and the single channel is spaced (N-1) other single channels apart from the base channel, where N is a positive integer greater than or equal to 1.

10. The image stitching method based on a line scan camera according to claim 8, characterized in that, The step of calculating the correlation coefficient between the base channel and the selected single channel in each group of stitched images further includes: The filtered image is subjected to at least one morphological dilation operation to obtain the mask image.

11. The image stitching method based on a linear scan camera according to any one of claims 9, characterized in that, The step of calculating the correlation coefficient between the base channel and the selected single channel in each group of stitched images further includes: Based on the stitched image, the corresponding basic channel component x and the selected single channel component y for which the correlation coefficient is to be calculated are obtained; The correlation coefficient between the base channel and the selected single channel is calculated using the correlation coefficient calculation formula, under a unit splicing coefficient. The formula is as follows: in, , These are the average values ​​of the base channel components x and the average values ​​of the selected single channel components y for all pixels that need to be calculated in the current stitched image, respectively.

12. The image stitching method based on a line scan camera according to claim 11, characterized in that, The process of finding the optimal unit splicing coefficient based on the numerical relationship between multiple sets of unit splicing coefficients and their corresponding correlation coefficients includes: Calculate the correlation coefficients of the corresponding base channel and the selected single channel under different unit splicing coefficients; The quadratic polynomial relationship between the unit splicing coefficient 'a' and the correlation coefficient 'b' is obtained by fitting using the least squares method, namely: Find the extreme value of the fitted binomial to obtain the unit splicing coefficient a corresponding to the maximum value of the correlation coefficient b. If the correlation coefficients of multiple selected single channels and the basic channel are calculated, the average value of the multiple unit splicing coefficients a is taken as the optimal unit splicing coefficient.

Images