Image processing method and related device

Abstract

The embodiment of the present application provides an image processing method and related devices. The embodiment of the present application obtains the original image of the target to be detected through a color line scan camera. The original image is a color line scan image in Bayer format, including bright field data and dark field data. Based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, the original image is converted into a target image with the same width and height, so that each row of pixels in the target image contains an RG value or a GB value. Furthermore, by forming a row number pair by forming each two adjacent rows of image data in the target image, a row number pair set is constructed, and then a color bright field image and a dark field image can be obtained by extracting the row number pair set. In this way, the situation in which subtle color defects of the product cannot be effectively detected due to the limited amount of information that can be presented by the black and white line scan image is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method and related devices. Background Art

[0002] With the intensification of industrial automation, machine vision technology has become an important choice for quality inspection in the industrial production process. In industrial inspection, relevant personnel control black and white line scan cameras to capture images of the target to be inspected under multiple light sources, and divide the line scan images into images under different light paths corresponding to the number of light sources based on the channel separation algorithm, and determine the quality of the target to be inspected through the image data of the target to be inspected in the images of different light paths. Since the existing technology only supports channel separation of line scan images of black and white line scan cameras, the amount of information that can be presented by black and white line scan images is limited, and it is impossible to effectively detect subtle color defects on products. Summary of the invention

[0003] The embodiments of the present application provide an image processing method and related devices, which detect a target to be detected by using a color line scan camera, and use a channel separation algorithm for the color line scan image to obtain a bright field image and a dark field image of the target to be detected, thereby reducing the situation where subtle color defects of the product cannot be effectively detected due to the limited amount of information that can be presented by the black and white line scan image.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Acquire image data of an original image; wherein the image format of the original image is a Bayer format, and the image data includes bright field data and dark field data;

[0006] Based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, the original image is converted into a target image so that in the target image, each row of pixels contains RG values ​​or GB values; wherein the width and height of the target image are the same as those of the original image;

[0007] For the target image, the image data of each two adjacent rows are grouped into row pairs to construct a row pair set, and the bright field image and the dark field image corresponding to the original image are determined based on the row pair set; wherein the image data in each row pair are different.

[0008] The embodiment of the present application obtains the original image of the target to be detected through a color line scan camera, and the original image is a color line scan image in Bayer format, including bright field data and dark field data. Based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, the original image is converted into a target image with the same width and height, so that each row of pixels in the target image contains an RG value or a GB value. Furthermore, by forming a row number pair by forming each two adjacent rows of image data in the target image, a row number pair set is constructed, and then the color bright field image and dark field image corresponding to the target to be detected can be obtained by extracting the row number pair set.

[0009] In some possible embodiments, the image data includes row and column coordinates and RGB values ​​of each pixel, and the converting the original image into a target image based on an arrangement rule of the RGB values ​​of the pixels in the Bayer array includes:

[0010] Reorganizing the arrangement of the image data in the original image so that the width of the reorganized image is twice that of the original image and the length is half that of the original image;

[0011] Construct an image memory of the target image, and store the pixel points in the image memory based on the modulo division result of the row and column coordinates of each pixel point and a preset parameter; wherein the preset parameter is determined based on the arrangement rule of the RGB values ​​of the pixel points in the Bayer array.

[0012] The embodiment of the present application reorganizes the arrangement of the image data of the original image, and arranges the bright field / dark field data originally in two adjacent rows into one row. The preset parameters are determined based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, and the storage position of each pixel in the image memory of the target image is determined according to the modulus division result of the row and column coordinates of each pixel in the reorganized image and the preset parameters, so that each row of pixels in the target image contains RG values ​​or GB values.

[0013] In some possible embodiments, reorganizing the arrangement of the image data in the original image includes:

[0014] The truncation marks corresponding to the odd-numbered lines or even-numbered lines in the image data are removed, and the truncation marks are used to determine the width of the original image.

[0015] The embodiment of the present application can expand two adjacent rows of pixels in the original image into one row by removing the truncation mark after the odd rows or the even rows.

[0016] In some possible embodiments, storing the pixel point in the image memory based on the modulo division result of the row and column coordinates of each pixel point and a preset parameter includes:

[0017] For each of the pixel points, a corresponding row and column position of the pixel point in the image memory is determined according to a modulo result of the pixel point column coordinates and the preset parameter.

[0018] The embodiment of the present application determines the preset parameters based on the arrangement rule of the Bayer array, and determines the storage position of the pixel in the image memory by dividing the column coordinates of each pixel in the target image by the preset parameters, so that each row of pixels in the target image contains an RG value or a GB value.

[0019] In some possible embodiments, for the target image, grouping image data of every two adjacent rows into row pairs to construct a row pair set includes:

[0020] Acquire image data of the target image, wherein the image data includes row and column coordinates and RGB values ​​of each pixel;

[0021] Pixel points of every two adjacent rows in the image data form the row number pairs, and determine the arrangement sequence numbers of the row number pairs according to the row coordinates of the pixels in the row number pairs, wherein the arrangement sequence numbers represent the arrangement order of the row number pairs in the row number pair set;

[0022] The row number pair set is determined based on the row number pairs and the arrangement sequence number.

[0023] In the embodiment of the present application, pixel points of two adjacent rows in the target image are grouped into row pairs, and the row pairs are labeled with serial numbers based on the row coordinates of the pixels in the row pairs, and the arrangement order of the row pairs in the row pair set is determined according to the size of the serial numbers.

[0024] In some possible embodiments, determining the bright field image and the dark field image corresponding to the original image based on the set of row number pairs includes:

[0025] Constructing a first image memory and a second image memory, wherein the width of the first image memory and the second image memory is the same as the original image, and the height is half of the original image;

[0026] According to the arrangement sequence number of each row number pair in the row number pair set, the row number pairs with odd arrangement sequence numbers are stored in the first image memory to generate a first image; and the row number pairs with even arrangement sequence numbers are stored in the second image memory to generate a second image; wherein, if the first image is the bright field image, the second image is the dark field image, and if the first image is the dark field image, the second image is the bright field image.

[0027] The embodiment of the present application constructs a first image memory and a second image memory respectively, and the width of the two image memories is the same as the original image, and the height is half of the original image. By storing the row number pairs with odd arrangement numbers in the row number pair set into the first image memory and storing the row number pairs with even arrangement numbers into the second image memory, the color bright field image and dark field image corresponding to the target to be detected can be obtained.

[0028] In some possible embodiments, after determining the bright field image and the dark field image corresponding to the original image based on the set of row number pairs, the method further includes:

[0029] An interpolation algorithm is used to convert the formats of the bright field image and the dark field image respectively.

[0030] In the embodiment of the present application, the image format of the bright field image and the dark field image obtained based on the original image is the same as the original image, both of which are in Bayer format. Since the data volume of the Bayer sensor is smaller than that of the RGB sensor, it is necessary to interpolate the bright field image and the dark field image to obtain a color image with accurate color.

[0031] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0032] An image data acquisition module is configured to acquire image data of an original image; wherein the image format of the original image is a Bayer format, and the image data includes bright field data and dark field data;

[0033] A target image acquisition module is configured to convert the original image into a target image based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, so that each row of pixels in the target image contains RG values ​​or GB values; wherein the width and height of the target image are the same as those of the original image;

[0034] The image extraction module is configured to execute, for the target image, grouping the image data of each two adjacent rows into row pairs, constructing a row pair set, and determining the bright field image and the dark field image corresponding to the original image based on the row pair set; wherein the image data in each of the row pairs are different.

[0035] In some possible embodiments, the image data includes row and column coordinates and RGB values ​​of each pixel, and the original image is converted into a target image based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, and the target image acquisition module is configured as follows:

[0036] Reorganizing the arrangement of the image data in the original image so that the width of the reorganized image is twice that of the original image and the length is half that of the original image;

[0037] Construct an image memory of the target image, and store the pixel points in the image memory based on the modulo division result of the row and column coordinates of each pixel point and a preset parameter; wherein the preset parameter is determined based on the arrangement rule of the RGB values ​​of the pixel points in the Bayer array.

[0038] In some possible embodiments, to perform the reorganization of the arrangement of the image data in the original image, the target image acquisition module is configured as follows:

[0039] The truncation marks corresponding to the odd-numbered lines or even-numbered lines in the image data are removed, and the truncation marks are used to determine the width of the original image.

[0040] In some possible embodiments, the result of the modulo division based on the row and column coordinates of each pixel and the preset parameters is executed to store the pixel in the image memory, and the target image acquisition module is configured as follows:

[0041] For each of the pixel points, a corresponding row and column position of the pixel point in the image memory is determined according to a modulo result of the pixel point column coordinates and the preset parameter.

[0042] In some possible embodiments, the image extraction module is configured to:

[0043] Acquire image data of the target image, wherein the image data includes row and column coordinates and RGB values ​​of each pixel;

[0044] Pixel points of every two adjacent rows in the image data form the row number pairs, and determine the arrangement sequence numbers of the row number pairs according to the row coordinates of the pixels in the row number pairs, wherein the arrangement sequence numbers represent the arrangement order of the row number pairs in the row number pair set;

[0045] The row number pair set is determined based on the row number pairs and the arrangement sequence number.

[0046] In some possible embodiments, determining the bright field image and the dark field image corresponding to the original image based on the set of row number pairs is performed, and the image extraction module is configured as follows:

[0047] Constructing a first image memory and a second image memory, wherein the width of the first image memory and the second image memory is the same as the original image, and the height is half of the original image;

[0048] According to the arrangement sequence number of each row number pair in the row number pair set, the row number pairs with odd arrangement sequence numbers are stored in the first image memory to generate a first image; and the row number pairs with even arrangement sequence numbers are stored in the second image memory to generate a second image; wherein, if the first image is the bright field image, the second image is the dark field image, and if the first image is the dark field image, the second image is the bright field image.

[0049] In some possible embodiments, after determining the bright field image and the dark field image corresponding to the original image based on the set of row number pairs, the image extraction module is further configured to:

[0050] An interpolation algorithm is used to convert the formats of the bright field image and the dark field image respectively.

[0051] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0052] processor;

[0053] a memory for storing instructions executable by the processor;

[0054] The processor is configured to execute the instructions to implement any one of the methods provided in the first aspect of the present application.

[0055] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any of the methods provided in the first aspect of the present application.

[0056] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 A schematic diagram of product color defects shown in an embodiment of the present application;

[0059] Figure 2a A schematic diagram of an application environment shown in an embodiment of the present application;

[0060] Figure 2b A schematic diagram of a light source triggering timing sequence shown in an embodiment of the present application;

[0061] Figure 3a This is an overall flow chart of the image processing method shown in the embodiment of the present application;

[0062] Figure 3b A schematic diagram of a Bayer array shown in an embodiment of the present application;

[0063] Figure 3c A schematic diagram of the pixel arrangement of an original image shown in an embodiment of the present application;

[0064] Figure 3d This is a schematic diagram of the pixel arrangement of the reconstructed image shown in the embodiment of the present application;

[0065] Figure 3e A schematic diagram of the use of a truncation mark shown in an embodiment of the present application;

[0066] Figure 3f A schematic diagram of a process of storing image data into a target image shown in an embodiment of the present application;

[0067] Figure 3g A schematic diagram of a first image memory and a second image memory shown in an embodiment of the present application;

[0068] Figure 3h This is a schematic diagram of interpolating a first image and a second image according to an embodiment of the present application;

[0069] Figure 4 A block diagram of an image processing device 400 shown in an embodiment of the present application;

[0070] Figure 5 This is a schematic diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0072] In the description of the embodiments of the present application, unless otherwise specified, the term "multiple" refers to two or more, and other quantifiers are similar and should be understood. The preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments of the present application and the features therein may be combined with each other if there is no conflict.

[0073] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. In the actual processing process or when the control device is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or drawings.

[0074] Line scan cameras inspect industrial products by detecting images of the target under multiple light paths to determine product quality. If a product has color defects, it may indicate that the product has quality risks, such as Figure 1 The left side shows the pad oxidation and Figure 1 The problem of different colors in screen printing of glass products is shown on the right. The related technology only supports channel separation of line scan images of black and white line scan cameras, so black and white line scan cameras are often used for industrial inspection. Channel separation is to separate the image of the target to be inspected under multiple light paths from the line scan image. Since the line scan images collected by black and white line scan cameras are black and white, the amount of information that can be presented is limited. It is impossible to effectively detect subtle color defects on the product.

[0075] In order to solve the above problems, the invention concept of the present application is as follows: the original image of the target to be detected is obtained by a color line scan camera, and the original image is a color line scan image in Bayer format, including bright field data and dark field data. Based on the arrangement rule of the RGB values ​​of pixels in the Bayer array, the original image is converted into a target image with the same width and height, so that each row of pixels in the target image contains an RG value or a GB value. Furthermore, by forming a row number pair by combining each two adjacent rows of image data in the target image, a row number pair set is constructed, and then a color bright field image and dark field image can be obtained by extracting the row number pair set. In this way, the situation in which subtle color defects of the product cannot be effectively detected due to the limited amount of information that can be presented by the black and white line scan image is reduced.

[0076] An image processing method and related devices provided in an embodiment of the present application are described in detail below with reference to the accompanying drawings.

[0077] See also Figure 2a, is a schematic diagram of an application environment according to an embodiment of the present application. Figure 2a As shown, the application environment may include, for example, a line scan camera 10, a light source switching system 20, a roller conveying platform 30, and a target to be detected 40. The light source switching system 20 may control the light source A and the light source B to be turned on and off.

[0078] When a line scan camera is used to perform industrial quality inspection on an object to be inspected, the object to be inspected 40 is first placed on the roller conveyor platform 30. The position of the line scan camera 10 is adjusted so that the CCD (Charge-coupled Device) direction of the line scan camera 10 is perpendicular to the conveying direction of the roller conveyor platform 30. Along the conveying direction of the roller conveyor platform 30, light sources A and B are placed in front of and behind the line scan camera 10, respectively, and the angles of light sources A and B are adjusted so that the light paths of the two light sources are located directly below the line scan camera 10.

[0079] Furthermore, the trigger mode of the line scan camera 10 is set to internal trigger, and the trigger signal of the line scan camera 10 is connected to the trigger signal of the light source switching system 20, so that the light source system 20 can control the state of the light source A and the light source B based on the number of scans of the line scan camera 10. By activating the rolling conveying platform 20 at a uniform speed, the line scan camera 10 can collect images of the target 40 to be inspected. Among them, the internal trigger is to control the frame rate of the camera to collect images through software settings, so that the light source switching system 20 can be set to synchronously control the lighting of the light source when the camera triggers the scan.

[0080] In some possible embodiments, a channel separation algorithm is used to separate the line scan image of the target to be detected 40 into a first image of the target to be detected under optical path A and a second image of the target to be detected 40 under optical path B.

[0081] The trigger logic of the light source switching system is that when the line scan camera 10 is triggered for an odd number of times, light source A is controlled to light up and light source B is turned off. When the line scan camera is triggered for an even number of times, light source A is controlled to turn off and light source B is controlled to light up. The specific implementation can be as follows Figure 2b As shown, every even-numbered scanning trigger of the electrical signal of the line scan camera will synchronously trigger the lighting of light source A, and every odd-numbered scanning trigger of the electrical signal will synchronously trigger the lighting of light source B.

[0082] It should be noted that the above Figure 2a The structure shown is only an example and is not limited to this structure in the embodiments of the present application.

[0083] The embodiment of the present application is based on the color line scan camera using the Bayer sensor to collect images of the target to be detected to obtain the original image of the target to be detected. And based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, a corresponding image processing method is set to achieve channel separation, which is used to extract the color image of the target to be detected under different light paths from the original image.

[0084] For the specific implementation process, please refer to Figure 2a In the industrial inspection process, it is necessary to fine-tune the angles of light source A and light source B so that the light path of light source A can directly enter the line scan camera through the mirror reflection of the target to be inspected to form a bright field image. Correspondingly, the light path of light source B cannot enter the line scan camera through the mirror reflection of the target to be inspected to form a dark field image. When the roller conveyor platform is started at a uniform speed to allow the color line scan camera to capture images of the target to be inspected, the image processing method provided in the embodiment of the present application can be used to perform channel separation on the captured line scan images, so as to extract the bright field image and dark field image of the target to be inspected from the original image scanned by the color line scan camera.

[0085] The image processing method in the embodiment of the present application is described in detail below with reference to the accompanying drawings. Figure 3a As shown, the following steps are included:

[0086] Step 301: Acquire image data of an original image; wherein the image format of the original image is a Bayer format, and the image data includes bright field data and dark field data.

[0087] The original image is the line scan image of the target to be detected captured by the color line scan camera of the Bayer sensor. The original image is obtained by collecting the original data stream of multiple channels by the color line scan camera, so the original image contains the bright field image and dark field image of the target to be detected. The image data is the data of each pixel point contained in the image, such as the row and column coordinates of the pixel point, RGB value and other information. Divided by the bright field and dark field angles, the pixel point data belonging to the bright field image in the image data is the bright field data, and the pixel point data belonging to the dark field image is the dark field data.

[0088] Step 302: based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, convert the original image into a target image, so that in the target image, each row of pixels contains RG values ​​or GB values; wherein the width and height of the target image are the same as those of the original image;

[0089] The image data contains the row and column coordinates of each pixel in the original image and the RGB value of each pixel, where R represents red, G represents green, and B represents blue. The Bayer array is the main technology for sensors to capture color images. It converts grayscale information into color information by simulating the sensitivity of the human eye to color to form a color image. The Bayer array is a 4×4 matrix that can be used as Figure 3b As shown, it consists of 4 B, 8 G and 4 R. In this array, all pixels in a row are B or G, and all pixels in adjacent rows are G or R. The Bayer array converts the grayscale image into a color image by performing 9 operations in a 2×2 matrix.

[0090] The embodiment of the present application aims to reorganize and extract the arrangement order of pixels in the original image to form a pixel arrangement that conforms to the Bayer array.

[0091] The following is an example of using a color line scan camera to scan odd-numbered lines as a bright field image of the target to be detected and even-numbered lines as a dark field image of the target to be detected. That is, the pixel arrangement in the original image can be as follows: Figure 3c As shown, the pixels in every two adjacent rows belong to images corresponding to different light paths, for example Figure 3c The 0th and 1st rows are dark field image pixels, and the 2nd and 3rd rows are bright field image pixels. It should be understood that the optical paths corresponding to the odd and even rows of the line scan camera are set by relevant personnel, and this application does not limit this.

[0092] When executing step 302, the arrangement of the image data in the original image must first be reorganized so that the width of the reorganized image is twice the original image and the length is half the original image. For example, if a 4×4 original image is reorganized, the reorganized image should be 8×2. The arrangement of the pixels in the reorganized image can be as follows: Figure 3d As shown, the original different light paths corresponding to the pixels in each two adjacent rows are updated to the same light paths corresponding to the pixels in each row, and different light paths corresponding to adjacent rows.

[0093] During implementation, the original image can be reconstructed by removing the truncation marks corresponding to the odd or even lines in the image data. Figure 3e As shown, the truncation mark is a mark used to determine the width of the original image. The data stream transmitted in the camera is one-dimensional. By adding a truncation mark after a certain amount of data stream, the one-dimensional data stream can be divided into lines. Then, by removing the truncation mark after the odd or even lines in the original image, two adjacent lines can be merged into one line.

[0094] Furthermore, an image memory of the target image is constructed, and the pixel points are stored in the image memory based on the modulo division result of the row and column coordinates of each pixel point and the preset parameters. The preset parameters are determined based on the arrangement rule of the RGB values ​​of the pixel points in the Bayer array, and the purpose is that after the pixel points are stored in the image memory to generate the target image, the arrangement rule of the pixel points in the target image conforms to the Bayer array.

[0095] During implementation, for each pixel, the corresponding row and column position of the pixel in the image memory is determined according to the modulus division result of the pixel column coordinates and the preset parameter. Specifically, since the Bayer array converts the grayscale graphics into a color image, it will perform operations with a 2×2 matrix. Therefore, the preset parameter value can be set to 4. Taking the Nth pixel in the 0th row of the image data as an example, if the result of N%4 (that is, the result of N modulo 4) is 0, the pixel is stored in the position of "N / 2" in the 0th row of the image memory. If the result of N%4 is 1, the pixel is stored in the position of "(N+1) / 2" in the 0th row of the image memory. If the result of N%4 is 2, the pixel is stored in the position of "N / 2-1" in the 1st row of the image memory. If the result of N%4 is 3, the pixel is stored in the position of "(N-1) / 2" in the 1st row of the image memory. Thus, the operation mode of the Mth row in the image data can correspond to the "M*2"th row and the "M*2+1"th row in the newly opened image memory.

[0096] To facilitate understanding of the above judgment process, Figure 3f As shown, for example, the first two rows in the original image are pixels of the bright field image, the pixels in the first row are "R1, G1, B1, G2, R2, G3, B2, G4", and the pixels in the second row are "R3, G5, B3, G6, R4, G7, B4, G8". Then after rearrangement, the pixels in the first row should be "R1, G1, B1, G1, G2, R2, G3, B2, G4, R3, G5, B3, G6, R4, G7, B4, G8". Through the above judgment of the row and column coordinates of the pixel points, the position of each pixel in the first row in the target image is determined. The pixels stored in the first row of the target image should be "R1, G1, R2, G3, R3, G5, R4, G7". The pixels stored in the second row should be "G2, B1, G4, B2, G6, B3, G8, B4". Therefore, in the target image, starting from the first row where all pixels contain R or G values, the pixels in every two adjacent rows are RG and GB alternately. Therefore, in the target image, the light paths of the pixels in every two adjacent rows are the same and follow the arrangement of the Bayer array.

[0097] Step 303: for the target image, group the image data of every two adjacent rows into row pairs, construct a row pair set, and determine the bright field image and dark field image corresponding to the original image based on the row pair set; wherein the image data in each row pair are different.

[0098] As mentioned above, in the target image, the light paths of two adjacent rows of pixels are the same and follow the Bayer array arrangement. Therefore, by constructing a row number pair set by forming a row number pair with two adjacent rows, the bright field image and dark field image corresponding to the original image can be determined based on the row number pair set.

[0099] When constructing a row number pair set, the image data of the target image can be obtained, and the pixels of each two adjacent rows in the image data can be grouped into row number pairs, and the arrangement sequence number of the row number pair can be determined according to the row coordinates of the pixels in the row number pair. The arrangement sequence number represents the arrangement order of the row number pair in the row number pair set, and the row number pair set can be determined based on the row number pairs and the arrangement sequence number.

[0100] During implementation, the minimum row coordinate in the row number pair can be selected as the arrangement number of the row number pair. Specifically, for example, in the target image, the 0th row and the 1st row form the first row number pair, and the 2nd row and the 3rd row form the second row number pair. Since the minimum row coordinate of the first row number pair is 0, the minimum row coordinate of the second row number pair is 1. Then the arrangement number of the first row number pair can be set to 1, and the arrangement number of the second row number pair can be set to 2.

[0101] Further, such as Figure 3g As shown, by constructing a first image memory and a second image memory, the width of the first image memory and the second image memory are the same as the original image, and the height is half of the original image. According to the arrangement sequence of each row number pair in the row number pair set, the row number pairs with odd arrangement sequences are stored in the first image memory to generate the first image; and the row number pairs with even arrangement sequences are stored in the second image memory to generate the second image. The row number pairs with odd arrangement sequences in the row number pair set are all pixels corresponding to the same optical path, and the row number pairs with even arrangement sequences are all pixels corresponding to the same optical path, and the row number pairs with odd arrangement sequences and the row number pairs with even arrangement sequences belong to different optical paths. Taking the example of first acquiring bright field image pixels by a color line scan camera, the first image should be a bright field image of the target to be detected, and the second image should be a dark field image.

[0102] In the embodiment of the present application, the image format of the bright field image and the dark field image obtained based on the original image is the same as the original image, both of which are in Bayer format. Since the data volume of the Bayer sensor is smaller than that of the RGB sensor, it is necessary to interpolate the bright field image and the dark field image to obtain a color image with accurate color. Figure 3hAs shown, during implementation, an interpolation algorithm can be used to convert the formats of the bright field image and the dark field image respectively, and the bright field image and the dark field image can be converted from the Bayer format to the BGR format.

[0103] Through the above process, the embodiment of the present application performs image processing on the line scan image in the Bayer format to obtain a color image of the target to be detected under different optical paths in the line scan image, thereby reducing the situation where subtle color defects of the product cannot be effectively detected due to the limited amount of information that can be presented by the black and white line scan image.

[0104] Based on the same inventive concept, the present application embodiment further provides an image processing device 400, specifically, Figure 4 As shown, including:

[0105] The image data acquisition module 401 is configured to acquire image data of an original image; wherein the image format of the original image is a Bayer format, and the image data is bright field data and dark field data;

[0106] The target image acquisition module 402 is configured to convert the original image into a target image based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, so that each row of pixels in the target image contains RG values ​​or GB values; wherein the width and height of the target image are the same as those of the original image;

[0107] The image extraction module 403 is configured to execute, for the target image, grouping the image data of each two adjacent rows into row pairs, constructing a row pair set, and determining the bright field image and the dark field image corresponding to the original image based on the row pair set; wherein the image data in each of the row pairs are different.

[0108] In some possible embodiments, the image data includes row and column coordinates and RGB values ​​of each pixel, and the original image is converted into a target image based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array. The target image acquisition module 402 is configured as follows:

[0109] Reorganizing the arrangement of the image data in the original image so that the width of the reorganized image is twice that of the original image and the length is half that of the original image;

[0110] Construct an image memory of the target image, and store the pixel points in the image memory based on the modulo division result of the row and column coordinates of each pixel point and a preset parameter; wherein the preset parameter is determined based on the arrangement rule of the RGB values ​​of the pixel points in the Bayer array.

[0111] In some possible embodiments, to perform the reorganization of the arrangement of the image data in the original image, the target image acquisition module 402 is configured as follows:

[0112] The truncation marks corresponding to the odd-numbered lines or even-numbered lines in the image data are removed, and the truncation marks are used to determine the width of the original image.

[0113] In some possible embodiments, the target image acquisition module 402 is configured to perform the modulo division result based on the row and column coordinates of each pixel point and the preset parameters, and store the pixel point in the image memory:

[0114] For each of the pixel points, a corresponding row and column position of the pixel point in the image memory is determined according to a modulo result of the pixel point column coordinates and the preset parameter.

[0115] In some possible embodiments, the image extraction module 403 is configured to:

[0116] Acquire image data of the target image, wherein the image data includes row and column coordinates and RGB values ​​of each pixel;

[0117] Pixel points of every two adjacent rows in the image data form the row number pairs, and determine the arrangement sequence numbers of the row number pairs according to the row coordinates of the pixels in the row number pairs, wherein the arrangement sequence numbers represent the arrangement order of the row number pairs in the row number pair set;

[0118] The row number pair set is determined based on the row number pairs and the arrangement sequence number.

[0119] In some possible embodiments, to determine the bright field image and the dark field image corresponding to the original image based on the set of line number pairs, the image extraction module 403 is configured as follows:

[0120] Constructing a first image memory and a second image memory, wherein the width of the first image memory and the second image memory is the same as the original image, and the height is half of the original image;

[0121] According to the arrangement sequence number of each row number pair in the row number pair set, the row number pairs with odd arrangement sequence numbers are stored in the first image memory to generate a first image; and the row number pairs with even arrangement sequence numbers are stored in the second image memory to generate a second image; wherein, if the first image is the bright field image, the second image is the dark field image, and if the first image is the dark field image, the second image is the bright field image.

[0122] In some possible embodiments, after determining the bright field image and the dark field image corresponding to the original image based on the set of line number pairs, the image extraction module 403 is further configured to:

[0123] An interpolation algorithm is used to convert the formats of the bright field image and the dark field image respectively.

[0124] Refer to the following Figure 5 The electronic device 130 according to this embodiment of the present application is described. Figure 5 The electronic device 130 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0125] like Figure 5 As shown, the electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0126] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0127] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0128] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0129] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), one or more devices that enable a user to interact with the electronic device 130, and / or any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 136. Figure 5As shown, the network adapter 136 communicates with other modules for the electronic device 130 via the bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0130] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 132 including instructions, and the instructions can be executed by the processor 131 of the device 400 to complete the above method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0131] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, which implements any of the image processing methods provided in the present application when executed by the processor 131 .

[0132] In an exemplary embodiment, various aspects of an image processing method provided by the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of an image processing method according to various exemplary embodiments of the present application described above in this specification.

[0133] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0134] The program product for image processing of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present application is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0135] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0136] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0137] The program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "like" or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device may be connected to the user electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0138] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0139] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0140] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0141] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image scaling device to produce a machine, so that the instructions executed by the processor of the computer or other programmable image scaling device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable image scaling device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, the instruction device being implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable image scaling device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0144] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0145] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire image data of an original image; wherein the image format of the original image is a Bayer format, and the image data includes bright field data and dark field data; Based on the arrangement rule of the three primary colors RGB values ​​of the pixel points in the Bayer array, the original image is converted into a target image, so that in the target image, each row of pixel points contains RG values ​​or GB values; wherein the width and height of the target image are the same as those of the original image; For the target image, the image data of each two adjacent rows are grouped into row pairs to construct a row pair set, and the bright field image and the dark field image corresponding to the original image are determined based on the row pair set; wherein the image data in each row pair are different.

2. The method according to claim 1, characterized in that The image data includes row and column coordinates and RGB values ​​of each pixel point, and the original image is converted into a target image based on the arrangement rule of the RGB values ​​of the pixels in the Bayer array, including: Reorganizing the arrangement of the image data in the original image so that the width of the reorganized image is twice that of the original image and the length is half that of the original image; Construct an image memory of the target image, and store the pixel points in the image memory based on the modulo division result of the row and column coordinates of each pixel point and a preset parameter; wherein the preset parameter is determined based on the arrangement rule of the RGB values ​​of the pixel points in the Bayer array.

3. The method according to claim 2, characterized in that The reorganizing the arrangement of the image data in the original image includes: The truncation marks corresponding to the odd-numbered lines or even-numbered lines in the image data are removed, and the truncation marks are used to determine the width of the original image.

4. The method according to claim 2, characterized in that: The step of storing the pixel points in the image memory based on the modulo division result of the row and column coordinates of each pixel point and a preset parameter comprises: For each of the pixel points, a corresponding row and column position of the pixel point in the image memory is determined according to a modulo result of the pixel point column coordinates and the preset parameter.

5. The method according to claim 1, characterized in that For the target image, the image data of each two adjacent rows are grouped into row pairs to construct a row pair set, including: Acquire image data of the target image, wherein the image data includes row and column coordinates and RGB values ​​of each pixel; Pixel points of every two adjacent rows in the image data form the row number pairs, and determine the arrangement sequence numbers of the row number pairs according to the row coordinates of the pixels in the row number pairs, wherein the arrangement sequence numbers represent the arrangement order of the row number pairs in the row number pair set; The row number pair set is determined based on the row number pairs and the arrangement sequence number.

6. The method according to claim 5, characterized in that Determining a bright field image and a dark field image corresponding to the original image based on the set of row number pairs includes: Constructing a first image memory and a second image memory, wherein the width of the first image memory and the second image memory is the same as that of the original image, and the height is half of that of the original image; According to the arrangement sequence number of each row number pair in the row number pair set, the row number pairs with odd arrangement sequence numbers are stored in the first image memory to generate a first image; and the row number pairs with even arrangement sequence numbers are stored in the second image memory to generate a second image; wherein, if the first image is the bright field image, the second image is the dark field image, and if the first image is the dark field image, the second image is the bright field image.

7. The method according to any one of claims 1 to 6, characterized in that: After determining the bright field image and the dark field image corresponding to the original image based on the row number pair set, the method further includes: An interpolation algorithm is used to convert the formats of the bright field image and the dark field image respectively.

8. An image processing device, characterized in that: The device comprises: An image data acquisition module is configured to acquire image data of an original image; wherein the image format of the original image is a Bayer format, and the image data includes bright field data and dark field data; A target image acquisition module is configured to convert the original image into a target image based on the arrangement rule of the three primary colors RGB values ​​of the pixel points in the Bayer array, so that each row of pixels in the target image contains RG values ​​or GB values; wherein the width and height of the target image are the same as those of the original image; The image extraction module is configured to execute, for the target image, grouping the image data of each two adjacent rows into row pairs, constructing a row pair set, and determining the bright field image and the dark field image corresponding to the original image based on the row pair set; wherein the image data in each of the row pairs are different.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as claimed in any one of claims 1 to 7.

Images