Cigarette box strip two-dimensional code rendering model construction method for code reading camera response function estimation

Abstract

The invention discloses a cigarette box strip two-dimensional code rendering model construction method for code reading camera response function estimation, and the method comprises the steps: constructing an end-to-end neural network rendering algorithm through employing a cigarette box strip two-dimensional code image sample, recovering a radiation field with a high dynamic range from low-illumination images with different exposures, and estimating a response function of a camera; comprising the following steps: designing a neural radiation field module, and constructing two continuous implicit neural network functions which are respectively used for modeling the density and scene radiance of a radiation field so as to more accurately estimate a camera response function and reflect the real physical characteristics of a camera; a tone mapping module is designed, a high-dynamic-range image is converted into a low-dynamic-range image, the density of a radiation field and scene radiance are combined with tone mapping of image color brightness, comprehensive modeling of the physical imaging process of the camera is achieved, and the generated image is more authentic. The method can adapt to changes of different scenes and lighting conditions, and the practical effect in a specific scene is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of cigarette manufacturing, and particularly to a method for constructing a rendering model of cigarette carton and strip two-dimensional codes for estimating the response function of a code-reading camera. Background Art

[0002] During the production process, cigarette manufacturing enterprises use a "carton condition" association system, with two-dimensional codes as the carrier, to associate small cartons, strip cartons, and case cigarettes, thereby realizing quality traceability. Among them, the code-reading camera of cigarette equipment collects information by scanning the two-dimensional codes on cigarette packages. However, due to the obvious fluctuations in the imaging quality of the code-reading camera under real cigarette site environmental factors such as different exposure conditions, light changes, and equipment jitter, the recognition accuracy of the code-reading camera in such complex scenarios is affected. To ensure that the code-reading camera can stably read information in the above specific complex environment, it is crucial to accurately estimate the response function of the code-reading camera.

[0003] Through research and analysis, the existing methods for estimating the response function of a code-reading camera are mainly divided into direct measurement methods and image inversion methods. The direct measurement method requires the use of high-precision measurement equipment to measure the radiance of the scene and the pixel values of the images output by the camera simultaneously, and then obtains the camera response function curve by fitting these measurement data; the image inversion method does not require measuring the radiance of the scene, but instead uses the rules of scene radiance and scene images, and inversely deduces the camera response function using the pixel values of the images.

[0004] That is, the existing methods for estimating the response function of a code-reading camera mainly use different exposure image sequences as input, and based on the known exposure ratios between images, establish the mapping relationship between the brightness information of each channel of the image and the scene irradiance, thereby obtaining a high-dynamic-range image reflecting the shooting scene. However, the above methods usually require the shooting scene to be static and assume that the illuminance in the image sequence is consistent. However, during actual shooting in the cigarette manufacturing scene, the dynamic changes of the target and the shaking of the imaging system are frequent and often difficult to control; in addition, the existing camera response function estimation schemes have poor robustness and are easily affected by adverse factors such as noise, and the noise in the image will interfere with the true information of the pixel values, causing the estimation result to deviate from the actual scene radiance, and further resulting in inaccurate estimation results of the camera response function and being unable to be applied in the cigarette manufacturing site. Summary of the Invention

[0005] In view of the above, the present invention aims to provide a method for constructing a rendering model of cigarette carton and strip two-dimensional codes for estimating the response function of a code-reading camera to solve the problems existing in the existing methods for estimating the response function of a code-reading camera of cigarette equipment.

[0006] The technical solution adopted by the present invention is as follows:

[0007] The present invention provides a method for constructing a rendering model of cigarette carton strip two-dimensional codes for estimating the response function of a code-reading camera, which includes:

[0008] Collecting cigarette carton strip two-dimensional code image samples under different illuminations and perspectives in the cigarette manufacturing environment, and constructing a data set for estimating the response function of the code-reading camera;

[0009] Using the radiation field encoding mechanism and the tone mapping mechanism, building a rendering model of cigarette carton strip two-dimensional codes based on an end-to-end neural network;

[0010] Using the data set for estimating the response function of the code-reading camera to train and test the rendering model of cigarette carton strip two-dimensional codes. Among them, the training process includes: presetting a color reconstruction loss function, estimating the radiation field and the camera inverse response function by minimizing the mean square error between the predicted low-illumination image and the real low-illumination image; based on the gradient of the color reconstruction loss function, adjusting the parameters of the model through the backpropagation algorithm to optimize the radiation field encoding mechanism, the tone mapping mechanism, and the estimation of the camera response function.

[0011] In at least one possible implementation manner, the building of the rendering model of cigarette carton strip two-dimensional codes based on an end-to-end neural network by using the radiation field encoding mechanism and the tone mapping mechanism includes:

[0012] Presetting a radiation field encoding mechanism to model the scene radiation, obtaining a radiation field model for extracting and representing radiation information from the input image, and capturing the imaging details of the cigarette packaging, where the imaging details at least include the gloss of the two-dimensional code, the paper texture, and the light reflection characteristics of the printing ink;

[0013] Presetting a tone mapping mechanism and establishing a non-linear mapping relationship between radiance and low illumination for converting a high dynamic range image into a low dynamic range image to reduce noise and highlight parts in the two-dimensional code image;

[0014] Combining the radiation field encoding mechanism and the tone mapping mechanism for image rendering, simulating the physical imaging process of the code-reading camera and outputting the final cigarette carton strip two-dimensional code image. Among them, when simulating, the imaging model, optical characteristics of the code-reading camera, and illumination conditions of the shooting environment are taken into account.

[0015] In at least one possible implementation manner, the radiation field encoding mechanism specifically includes:

[0016] Taking the ray information formed by the ray starting point and the ray direction as the input of the radiation field encoding mechanism to eliminate the reflection and shadow of the two-dimensional code caused by different cigarette packaging materials under illumination change conditions;

[0017] A radiation field model is constructed using a multi-layer perceptron to model the spatial position and direction information of rays, and the density and radiance of the rays are output through a non-linear transformation.

[0018] In at least one possible implementation, the tone mapping mechanism specifically includes:

[0019] Taking the density and radiance output by the radiation field encoding mechanism as the input of the tone mapping mechanism, and taking the exposure time as an additional parameter;

[0020] Constructing a differentiable tone mapping representation: fusing the density, radiance, and exposure time, and converting the fusion result into color information through a multi-layer perceptron for fine-tuning the brightness and contrast in the QR code image;

[0021] Using three multi-layer perceptrons to process the color channels corresponding to RGB in the image respectively;

[0022] Estimating the camera response function and following the classical non-parametric camera response function calibration method to adjust the color information by simulating the imaging process of the camera;

[0023] After converting the tone mapping functions of all images into functions in the logarithmic radiance domain, representing the color information through the inverse camera response transformation.

[0024] In at least one possible implementation, the image rendering includes: constructing an expression of the expected color information of the ray between the boundaries by combining the radiation field model and the tone mapping function, and generating the final color of the current ray by weighted accumulation of the color and density of each point, where the final color represents the pixel color corresponding to the ray.

[0025] In at least one possible implementation, the method of collecting image samples includes: obtaining images with different exposure levels by the doubling method, and having a predetermined exposure ratio value between adjacent exposure levels.

[0026] Compared with the prior art, the main design concept of the present invention lies in starting from constructing a camera response function estimation dataset, building an end-to-end neural rendering algorithm, recovering a high-dynamic-range neural radiance field from a series of low-light images with different exposures, and estimating the camera's response function; specifically including designing a neural radiance field module, constructing two consecutive implicit neural network functions, which are respectively used to model the density of the radiance field and the scene radiance, and can more accurately estimate the camera response function, thereby reflecting the true physical characteristics of the camera; and designing a tone mapping module to convert the high-dynamic-range image into a low-dynamic-range image, combining the density of the radiance field and the scene radiance with the tone mapping of the image color brightness, realizing a comprehensive modeling of the camera's physical imaging process. This comprehensive modeling ability makes the generated images more realistic. In summary, the present invention comprehensively and realistically models the physical imaging process of the code-reading camera of the cigarette equipment through an end-to-end neural network rendering mechanism, and can adapt to changes in different scenarios and lighting conditions, enhancing the practical effect in specific industrial vision application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described below in conjunction with the accompanying drawings, where:

[0028] Figure 1 It is a schematic diagram of a method for constructing a rendering model of cigarette carton and strip two-dimensional codes for estimating the response function of a code-reading camera provided by an embodiment of the present invention;

[0029] Figure 2 It is a schematic diagram of the architecture of a rendering model of cigarette carton and strip two-dimensional codes provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The embodiments of the present invention are described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation of the present invention.

[0031] An embodiment of the present invention proposes a method for constructing a rendering model of cigarette carton and strip two-dimensional codes for estimating the response function of a code-reading camera. Specifically, as Figure 1 shown, which includes:

[0032] Step S1: Collect cigarette carton and strip two-dimensional code image samples with different illuminations and perspectives in the cigarette manufacturing environment, and construct a dataset for estimating the response function of the code-reading camera;

[0033] Specifically, an industrial camera can be used to capture the two-dimensional code on the cigarette carton strip in the production scenario to construct a data set for estimating the response function of the code-reading camera, ensuring that the data set includes the illumination changes and perspective changes in the cigarette manufacturing environment. In actual operation, this data set can be divided into a training set and a test set. After the subsequent model training is completed, the test set can be used for testing, which will not be elaborated and limited in this embodiment.

[0034] To expand, a tripod can be used to fix the camera, and the lens focal length is set to 5 mm. Since the exposure time setting of the camera is not accurate and there is a difference between the actual exposure and the set exposure parameters, in order to facilitate image sample collection, it is preferred to use the doubling method to obtain images with different exposure levels, and the exposure ratio between adjacent exposure levels is approximately 0.5.

[0035] Step S2: Use the radiation field encoding mechanism and the tone mapping mechanism to build a rendering model of the two-dimensional code on the cigarette carton strip based on an end-to-end neural network;

[0036] To expand, the following steps can be referred to:

[0037] Step S21: Predesigned a radiation field encoding mechanism to model the scene radiation to obtain a neural radiation field model , which is used to extract and represent the radiation information from the input image, capture the imaging details of the cigarette packaging (such as gloss, paper texture, and light reflection characteristics of printing ink), so as to simulate and reconstruct a high-quality image of the two-dimensional code on the cigarette carton strip;

[0038] Step S22: Predesigned a tone mapping mechanism to establish a non-linear mapping between radiance and low illuminance to convert a high-dynamic range image into a low-dynamic range image, thereby reducing the noise and highlight parts in the image and fully retaining the details of the two-dimensional code on the cigarette carton strip;

[0039] Step S23: Combine the radiation field encoding mechanism and the tone mapping mechanism for image rendering, simulate the physical imaging process of the code-reading camera, and output the final image of the two-dimensional code on the cigarette carton strip.

[0040] Here, combined with Figure 2The architecture of the schematic rendering model of the cigarette carton strip two-dimensional code is introduced in detail for the above process. First, a radiance field encoding mechanism can be designed. The main goal of this module is to model the radiance information in the scene, so as to efficiently extract and represent the radiance field information from the input image, capture the imaging details of cigarette packaging (luster, paper texture, and light reflection characteristics of printing ink), and simulate and reconstruct high-quality images of cigarette carton strip two-dimensional codes. This process includes encoding the light distribution, light intensity, and object surface reflection characteristics in the scene to ensure that the system can capture the complex changes in light and materials.

[0041] Then, a differentiable tone mapping mechanism is constructed, which is used to map the radiance to low illuminance and establish a non-linear mapping relationship. The tone mapping mechanism can process high dynamic range images, convert them into low dynamic range images suitable for display, reduce noise and highlights in the images, and fully retain the details of cigarette carton strip two-dimensional codes.

[0042] Finally, neural network-based rendering can generate the final image by simulating the physical imaging process of the camera. The imaging model of the camera, optical characteristics, and illumination conditions of the shooting environment are considered during the rendering process. The cigarette carton strip two-dimensional code rendering model provided in this embodiment can generate high-quality images according to the input scene and camera parameters, and capture specific two-dimensional code imaging details that appear in the real cigarette manufacturing scene.

[0043] Among them, the implementation details of the radiance field encoding mechanism specifically include:

[0044] S21.1: The ray formed by the ray origin and the ray direction is used as the input of the radiance field encoding mechanism. The origin and direction of the ray provide spatial position information and viewing angle information, which are crucial for the complete modeling of the scene and are used to solve the problems of two-dimensional code reflection and shadow caused by different cigarette packaging materials under changing lighting conditions;

[0045] S21.2: Use a multi-layer perceptron to construct a neural radiance field model to model the spatial position and direction information of the ray. Through non-linear transformation, the density of the ray and the radiance are output. This process can be expressed as:

[0046]

[0047] Among them, the radiance represents the light intensity information at this point, and the density Used to judge the geometric characteristics of this point, such as whether there is an object or a transparent area. The neural radiance field is used to model and process the strong light reflection problem to avoid overexposure and information loss, so as to ensure the clarity of the two-dimensional code on the cigarette carton strip under different lighting environments.

[0048] Among them, the implementation details of the tone mapping mechanism specifically include:

[0049] S22.1: Take the density and radiance output by the radiance field encoding mechanism as the input of the tone mapping mechanism, and take the exposure time as an additional parameter;

[0050] S22.2: Construct a differentiable tone mapping representation, fuse the radiance, density and exposure time, and convert the fused result into color information through a multi-layer perceptron to finely adjust the brightness and contrast in the image, so as to optimize the two-dimensional code image of the cigarette carton strip under different lighting conditions. This process can be expressed as:

[0051]

[0052] Among them, is the exposure time of the camera capture ray , is the camera response function.

[0053] S22.3: Since different camera responses are used for tone mapping in the RGB channels of the image, three multi-layer perceptrons are used to process each color channel respectively;

[0054] S22.4: Use a multi-layer perceptron to estimate the camera response function , and follow the classic non-parametric camera response function calibration method to adjust the color information by simulating the imaging process of the camera, so that the output result is more realistic and conforms to the characteristics of the physical camera;

[0055] S22.5: Convert all images to the logarithmic radiance domain to optimize the network, and the calculation formula is as follows:

[0056]

[0057] Thus, the inverse camera response function can be expressed as: .

[0058] S22.6: Convert the tone mapping function into a function in the logarithmic radiance domain , through the inverse camera response transformation, the color information can be expressed as:

[0059]

[0060] Regarding the links involved in the aforementioned multi-layer perceptron, they can be specifically expanded as follows:

[0061] (1) Construct a hidden layer and use the ReLU nonlinear activation function as the core part of the hidden layer;

[0062] (2) The logarithmic radiance data is input into the multi-layer perceptron. The logarithmic transformed data is smoother, which makes it easier for the multi-layer perceptron to accurately model the brightness and scene details during the learning process.

[0063] (3) Use the mean square error loss function to evaluate the estimated error of the camera response function;

[0064] (4) Use log-radiance data to train a multi-layer perceptron to optimize the estimation of the camera response function. In each iteration, the model uses the error information fed back by the mean square error loss to update the network weights, so that the estimation of the camera response function continues to approach the characteristics of the real physical imaging process.

[0065] Further, the implementation process of the image rendering may include:

[0066] S23.1: By combining the radiation field model and the tone mapping model, a From the border arrive Expected color information Expressed by the color of each point and density Perform weighted accumulation to generate the final color of the current ray. The calculation process is as follows:

[0067]

[0068] in, is the light intensity density integral, and .

[0069] S23.2: The color information of all rays is integrated into a complete image output. Since each point where a ray passes has radiosity and density information, the final accumulated result is the pixel color corresponding to the current ray. By weighted accumulation of this information, the imaging process of the barcode reader camera of the cigarette equipment is simulated, and then the light attenuation is simulated and the image brightness is optimized to obtain a high-quality cigarette box barcode QR code image.

[0070] Continuing from the above, step S3, the cigarette box barcode rendering model is trained using the data set.

[0071] Use the training set data obtained in the foregoing steps to train the cigarette carton strip two-dimensional code rendering model, and use model supervision optimization to improve the accuracy of the cigarette carton strip two-dimensional code rendering model, including continuously adjusting model parameters. The optimization objective includes minimizing the difference between the generated image and the real image, so as to ensure that the cigarette carton strip two-dimensional code rendering model can provide high-fidelity rendering results under different scenarios and lighting conditions. That is, the code reading camera of the cigarette equipment can accurately read the cigarette carton strip two-dimensional code under different lighting and environments.

[0072] Regarding this step, it can be specifically expanded into the following process:

[0073] S3.1: Preset and design a color reconstruction loss function. By minimizing the mean square error between the predicted low-illumination image and the real low-illumination image, the purpose of estimating the radiation field and the camera inverse response function is achieved; among them, the calculation formula of the color reconstruction loss function is referred to as follows:

[0074]

[0075] Among them, and are two different refined models corresponding to constructing the neural radiation field respectively.

[0076] S3.2: Based on the gradient of the color reconstruction loss function, adjust the parameters of the model through the backpropagation algorithm;

[0077] S3.3: Through training and updating parameters, optimize the radiation field encoding mechanism and the tone mapping mechanism to make the final rendering result closer to the real image, and at the same time, the estimation of the camera response function can also be optimized.

[0078] In summary, the main design concept of the present invention is to start from constructing a camera response function estimation dataset, build an end-to-end neural rendering algorithm, recover a high-dynamic-range neural radiance field from a series of low-light images with different exposures, and estimate the camera's response function; specifically, it includes designing a neural radiance field module, constructing two consecutive implicit neural network functions, which are respectively used to model the density of the radiance field and the scene radiance, and can more accurately estimate the camera response function, thereby reflecting the true physical characteristics of the camera; and designing a tone mapping module to convert the high-dynamic-range image into a low-dynamic-range image, combining the density of the radiance field and the scene radiance with the tone mapping of the image color brightness, realizing a comprehensive modeling of the camera's physical imaging process. This comprehensive modeling ability makes the generated images more realistic. The present invention comprehensively and realistically models the physical imaging process of the barcode reading camera of the cigarette equipment through an end-to-end neural network rendering mechanism, and can adapt to changes in different scenarios and lighting conditions, enhancing the practical effect in specific industrial vision application scenarios.

[0079] In the embodiments of the present invention, if there are any expressions of orientation, they are based on the relative concepts of the embodiments. In addition, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent the situation where A exists alone, A and B exist simultaneously, or B exists alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0080] The above has detailed the structure, features, and effects of the present invention according to the embodiments shown in the drawings. However, the above are only the preferred embodiments of the present invention. It should be noted that for the technical features involved in the above embodiments and their preferred methods, those skilled in the art can reasonably combine and match them into various equivalent solutions without departing from and without changing the design concept and technical effects of the present invention; therefore, the present invention is not limited by the scope shown in the drawings. Any changes made in accordance with the concept of the present invention, or modified into equivalent embodiments with equivalent changes, as long as they still do not exceed the spirit covered by the description and the drawings, should be within the protection scope of the present invention.

Claims

1. A method for constructing a cigarette box barcode rendering model for estimating the response function of a barcode reader camera, characterized in that: include: Collect cigarette box barcode image samples under different lighting and viewing angles in a cigarette manufacturing environment, and construct a barcode reader camera response function estimation dataset; Using the radiation field encoding mechanism and tone mapping mechanism, a cigarette box barcode rendering model based on an end-to-end neural network was built; The cigarette box barcode two-dimensional code rendering model is trained and tested using the code reading camera response function estimation dataset, wherein the training process includes: presetting a color reconstruction loss function, estimating the radiation field and the camera inverse response function by minimizing the mean square error between the predicted low-light image and the actual low-light image; and adjusting the parameters of the model through a back propagation algorithm based on the gradient of the color reconstruction loss function to optimize the radiation field encoding mechanism, the tone mapping mechanism, and the estimation of the camera response function.

2. The method for constructing a cigarette box barcode rendering model for estimating a barcode reader camera response function according to claim 1, characterized in that: The method of using the radiation field encoding mechanism and the tone mapping mechanism to build a cigarette box barcode rendering model based on an end-to-end neural network includes: The radiation field encoding mechanism is preset to model the scene radiation, and a radiation field model is obtained, which is used to extract and represent radiation information from the input image and capture the imaging details of the cigarette package, wherein the imaging details at least include the gloss of the QR code, the texture of the paper, and the light reflection characteristics of the printing ink; A tone mapping mechanism is preset and a nonlinear mapping relationship between radiance and low illumination is established to convert a high dynamic range image into a low dynamic range image to reduce noise and highlight parts in the QR code image; The radiation field encoding mechanism and the tone mapping mechanism are combined for image rendering to simulate the physical imaging process of the barcode reading camera and output the final cigarette box barcode QR code image. The imaging model, optical characteristics and lighting conditions of the barcode reading camera and the shooting environment are taken into account during the simulation.

3. The method for constructing a cigarette box barcode rendering model for estimating a barcode reader camera response function according to claim 2, characterized in that: The radiation field encoding mechanism specifically includes: The ray information formed by the ray starting point and ray direction is used as the input of the radiation field encoding mechanism to eliminate the reflection and shadow of the QR code caused by different cigarette packaging materials under the condition of changing lighting; A radiation field model is constructed using a multi-layer perceptron to model the spatial position and direction information of the ray, and the density and radiance of the ray are output through nonlinear transformation.

4. The method for constructing a cigarette box barcode rendering model for estimating a barcode reader camera response function according to claim 3, characterized in that: The tone mapping mechanism specifically includes: The density and radiance output by the radiation field encoding mechanism are used as inputs to the tone mapping mechanism, and the exposure time is used as an additional parameter; Construct a differentiable tone mapping representation: fuse density, radiance, and exposure time, and convert the fusion result into color information through a multi-layer perceptron to fine-tune the brightness and contrast in the QR code image; Use three multi-layer perceptrons to process the color channels corresponding to RGB in the image respectively; The camera response function is estimated and the color information is adjusted by simulating the imaging process of the camera following the classic non-parametric camera response function calibration method. After the tone mapping functions of all images are converted into functions in the log-radiance domain, the color information is expressed by the inverse camera response transformation.

5. The method for constructing a cigarette box barcode rendering model for estimating a barcode reader camera response function according to claim 4, characterized in that: The image rendering includes: constructing an expression of expected color information about the ray between boundaries by combining a radiation field model and a tone mapping function, generating a final color of the current ray by weighted accumulation of the color and density of each point, and the final color represents the pixel color corresponding to the ray.

6. The method for constructing a cigarette box barcode rendering model for estimating a barcode reader camera response function according to any one of claims 1 to 5, characterized in that: The method of collecting image samples includes: obtaining images of different exposure levels by multiplication, and having a predetermined exposure ratio value between adjacent exposure levels.

Images