Image shooting method and device, storage medium and electronic equipment

Abstract

The invention discloses an image shooting method and device, a storage medium and electronic equipment. The method comprises the following steps: determining a reference structure parameter of a heating element arranged in target glass; wherein the target glass is glass arranged on the light sensing side of the camera; determining feature information of the camera; wherein the feature information comprises at least one of specification information of a sensor included in the camera and imaging feature information of a lens included in the camera; determining a first image including a light source area; based on the reference structure parameters, the feature information and the first image, generating a second image used for representing the diffraction effect of the light source corresponding to the light source area; based on the second image, evaluating the reference structure parameter to obtain an evaluation result; and determining a target structure parameter of the heating element based on the reference structure parameter and the evaluation result, so that the camera performs shooting under the condition that the heating element conforms to the target structure parameter. According to the embodiment of the invention, the image quality of the image shot by the camera can be improved, so that the driving safety is improved.

Description

Technical Field

[0001] The present disclosure relates to driving technology, and in particular to an image shooting method, device, storage medium and electronic device. Background Art

[0002] When using autonomous driving technology, it is often necessary to capture images of the actual scene through a camera installed in the vehicle, and then make driving decisions through the autonomous driving algorithm.

[0003] In some cases, there is glass in front of the camera, and the camera needs to shoot through the glass. In order to avoid the presence of water vapor on the glass surface, which affects the image quality of the camera, a heating element can be set in the glass. However, if there is a light source in front of the glass, the light source will produce a diffraction beam due to the diffraction of the heating element. In this way, the image quality of the camera cannot be guaranteed, which will affect driving safety. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides an image capturing method, an apparatus, a storage medium and an electronic device.

[0005] According to one aspect of an embodiment of the present disclosure, there is provided an image capturing method, including:

[0006] Determining reference structural parameters of a heating element disposed in a target glass; wherein the target glass is a glass disposed on a photosensitive side of a camera;

[0007] Determine characteristic information of the camera; wherein the characteristic information includes at least one of the following: specification information of a sensor included in the camera, imaging characteristic information of a lens included in the camera;

[0008] determining a first image including a light source area;

[0009] Based on the reference structure parameters, the characteristic information and the first image, generating a second image for representing the diffraction effect of the light source corresponding to the light source area;

[0010] Based on the second image, evaluating the reference structure parameters to obtain an evaluation result;

[0011] Based on the reference structural parameters and the evaluation result, target structural parameters of the heating element are determined, so that the camera takes pictures when the heating element meets the target structural parameters.

[0012] According to another aspect of an embodiment of the present disclosure, there is provided an image capturing device, including:

[0013] A first determination module is used to determine reference structural parameters of a heating element provided in a target glass; wherein the target glass is glass provided on a photosensitive side of a camera;

[0014] A second determination module is used to determine characteristic information of the camera; wherein the characteristic information includes at least one of the following: specification information of a sensor included in the camera, imaging characteristic information of a lens included in the camera;

[0015] A third determining module, used to determine the first image including the light source area;

[0016] a generating module, configured to generate a second image representing a diffraction effect of the light source corresponding to the light source area based on the reference structure parameters determined by the first determining module, the characteristic information determined by the second determining module, and the first image determined by the third determining module;

[0017] an evaluation module, configured to evaluate the reference structure parameters determined by the first determination module based on the second image generated by the generation module to obtain an evaluation result;

[0018] The fourth determination module is used to determine the target structural parameters of the heating element based on the reference structural parameters determined by the first determination module and the evaluation result obtained by the evaluation module, so that the camera can take pictures when the heating element meets the target structural parameters.

[0019] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned image capturing method.

[0020] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0021] processor;

[0022] a memory for storing instructions executable by the processor;

[0023] The processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned image shooting method.

[0024] According to another aspect of the embodiments of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor, the above-mentioned image capturing method is executed.

[0025] Based on the image capture method, device, storage medium, electronic device and program product provided by the above-mentioned embodiments of the present disclosure, the glass that needs to be captured by the camera can be called the target glass, and a heating element can be provided in the target glass to remove water vapor. Based on the reference structural parameters of the heater, the characteristic information of the camera, and the first image including the light source area, a second image can be generated to represent the diffraction effect of the light source corresponding to the light source area. With reference to the diffraction effect reflected by the second image, the rationality of the reference structural parameters can be effectively evaluated to obtain an evaluation result. In combination with the reference structural parameters and the evaluation results, the target structural parameters of the heater can be reasonably determined. In this way, by searching for structural parameters (i.e., target structural parameters) that can minimize the diffraction effect for the heater, and applying the heater that meets the structural parameters to the actual vehicle scene, it is beneficial to improve the image quality of the image captured by the camera through the target glass, reduce the adverse effects that diffraction may have on the operation of the autonomous driving algorithm, and thus help improve driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of a diffraction light column that appears due to diffraction of a light source by a heating element in the related art.

[0027] Figure 2 is a flowchart of an image capturing method provided by some exemplary embodiments of the present disclosure.

[0028] Figure 3-1 is a schematic diagram of a heating element in some exemplary embodiments of the present disclosure.

[0029] Figure 3-2 is a schematic diagram of a heating element in some other exemplary embodiments of the present disclosure.

[0030] Figure 3-3 is a schematic diagram of a linear spread function curve of a lens in some exemplary embodiments of the present disclosure.

[0031] Figure 4-1 is a schematic diagram of a second image in some exemplary embodiments of the present disclosure.

[0032] Figure 4-2 is a schematic diagram of a second image in some other exemplary embodiments of the present disclosure.

[0033] Figure 4-3 is a schematic diagram of a second image in some further exemplary embodiments of the present disclosure.

[0034] Figure 4-4 is a schematic diagram of a second image in some further exemplary embodiments of the present disclosure.

[0035] Figure 4-5 is a schematic diagram of a second image in some further exemplary embodiments of the present disclosure.

[0036] Figure 5-1 is a schematic diagram of a heating element in some further exemplary embodiments of the present disclosure.

[0037] Figure 5-2 Schematic diagram of a heating element in some further exemplary embodiments of the present disclosure.

[0038] Figure 6 is a flowchart of a method for generating a second image provided by some exemplary embodiments of the present disclosure.

[0039] Figure 7-1 is a schematic diagram of a third image in some exemplary embodiments of the present disclosure.

[0040] Figure 7-2 is a schematic diagram of a third image in some other exemplary embodiments of the present disclosure.

[0041] Figure 7-3 is a schematic diagram of a third image in some further exemplary embodiments of the present disclosure.

[0042] Figure 7-4 is a schematic diagram of a third image in some further exemplary embodiments of the present disclosure.

[0043] Figure 7-5 is a schematic diagram of a third image in some further exemplary embodiments of the present disclosure.

[0044] Figure 8 It is a flowchart of a method for determining the diffraction light intensity corresponding to each pixel point in the first image based on the third image, the fourth image, and the preset first diffraction light intensity calculation parameters provided by some exemplary embodiments of the present disclosure.

[0045] Fig. 9 It is a flowchart of a method for determining the diffraction light intensity corresponding to each pixel point in the first image based on the third image, the fourth image, and the preset first diffraction light intensity calculation parameters provided by other exemplary embodiments of the present disclosure.

[0046] Fig.10 It is a flowchart of a method for generating a second image provided by some other exemplary embodiments of the present disclosure.

[0047] Fig.11 It is a flowchart diagram of a method for determining the diffraction light intensity corresponding to each pixel point in the first image based on the third image, the number of first photosensitive elements, the number of second photosensitive elements, preset second diffraction light intensity calculation parameters, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image, provided by some exemplary embodiments of the present disclosure.

[0048] Figure 12-1It is a flowchart of a parameter evaluation method provided by some exemplary embodiments of the present disclosure.

[0049] Figure 12-2 It is a schematic diagram of the diffraction light intensity energy level distribution diagram in some exemplary embodiments of the present disclosure.

[0050] Figure 12-3 It is a schematic diagram of the diffraction light intensity energy level distribution diagram in some other exemplary embodiments of the present disclosure.

[0051] Figure 12-4 It is a schematic diagram of the diffraction light intensity energy level distribution diagram in some further exemplary embodiments of the present disclosure.

[0052] Figure 12-5 Schematic diagram of the diffraction light intensity energy level distribution diagram in some further exemplary embodiments of the present disclosure.

[0053] Figure 12-6 Schematic diagram of the diffraction light intensity energy level distribution diagram in some further exemplary embodiments of the present disclosure.

[0054] Fig.13 It is a flowchart of parameter evaluation methods provided by other exemplary embodiments of the present disclosure.

[0055] Fig.14 It is a flowchart of a method for determining target structure parameters provided by some exemplary embodiments of the present disclosure.

[0056] Figure 15-1 is a flowchart of determining target structure parameters in some exemplary embodiments of the present disclosure.

[0057] Figure 15-2 is a schematic diagram of determining target structure parameters in some exemplary embodiments of the present disclosure.

[0058] Fig.16 is a schematic diagram of the structure of an image capturing device provided by some exemplary embodiments of the present disclosure.

[0059] Fig.17 It is a schematic diagram of the structure of the generation module in some exemplary embodiments of the present disclosure.

[0060] Fig.18 is a structural diagram of a second determination submodule in some exemplary embodiments of the present disclosure.

[0061] Fig.19 It is a structural schematic diagram of the second determination submodule in some other exemplary embodiments of the present disclosure.

[0062] Fig. 20 It is a schematic diagram of the structure of the generation module in some other exemplary embodiments of the present disclosure.

[0063] Fig.21is a schematic structural diagram of the fourth determination submodule in some exemplary embodiments of the present disclosure.

[0064] Fig. 22 is a schematic diagram of the structure of the evaluation module in some exemplary embodiments of the present disclosure.

[0065] Fig.23 It is a schematic diagram of the structure of the evaluation module in some other exemplary embodiments of the present disclosure.

[0066] Fig.24 is a structural diagram of the fourth determination module in some exemplary embodiments of the present disclosure.

[0067] Fig.25 It is a schematic diagram of the structure of an electronic device provided by some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0068] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0069] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0070] Application Overview

[0071] As the level of autonomous driving increases, the requirements for the image quality of images taken by cameras installed on the vehicle are getting higher and higher.

[0072] If there is glass in front of the camera, in certain weather conditions such as winter, the glass surface will form water vapor due to changes in humidity, temperature, etc., as well as condensation of water molecules in the air inside the vehicle, affecting the quality of the images taken by the camera. In order to prevent water vapor from affecting the quality of the images taken by the camera, the commonly used solutions are: (a) adjusting the temperature inside the vehicle through air conditioning to evaporate the water vapor; (b) using a fan to accelerate the air flow inside the vehicle to evaporate the water vapor; (c) setting a heating element in the glass to remove the water vapor through the heating element.

[0073] It should be noted that for the above solution (c), if there is a light source in front of the camera, the light source will produce a diffraction beam due to the diffraction of the heating element. For details, please refer to Figure 1 The effects presented in the middle area 110 and the area 120 still cannot guarantee the image quality of the image taken by the camera, which will affect driving safety.

[0074] Exemplary Systems

[0075] In the embodiments of the present disclosure, the glass that needs to be photographed by the camera can be called the target glass, and a heating element can be provided in the target glass to remove water vapor. Based on the characteristic information of the camera and the first image including the light source area, combined with the discrete Fresnel diffraction calculation method or other diffraction calculation methods, the structural parameters that can reduce the diffraction effect as much as possible can be searched for the heating element, so that the heating element that meets the structural parameters can be used in the actual vehicle scene. In this way, it is beneficial to improve the image quality of the image taken by the camera through the target glass, reduce the adverse effects that diffraction may have on the operation of the autonomous driving algorithm, and thus help improve driving safety.

[0076] Exemplary Methods

[0077] Figure 2 is a flowchart of an image capturing method provided by some exemplary embodiments of the present disclosure. Figure 2 The method shown may include step 210 , step 220 , step 230 , step 240 , step 250 , and step 260 .

[0078] Step 210, determining reference structural parameters of a heating element disposed in a target glass; wherein the target glass is a glass disposed on the photosensitive side of a camera.

[0079] Optionally, the target glass may include but is not limited to a vehicle windshield, privacy glass, etc. Since the target glass is a glass disposed on the photosensitive side of the camera, the camera can capture images through the target glass, so it can be considered that the optical path of the camera capturing images introduces the target glass.

[0080] In one example, the target glass may be a front windshield, and the camera may be a front-view camera, which may capture images through the front windshield.

[0081] Optionally, a heating element may be provided in the target glass. The heating element may be in the form of a wire, and accordingly, the heating element may also be referred to as a heating wire. The heating element may include a straight heating wire, such as Figure 3-1 Alternatively, the heating element may include a shaped heating wire, such as Figure 3-2 The reference structural parameters of the heating element can be understood as the structural parameters of the heating element that are initially selected to be evaluated for rationality. The reference structural parameters of the heating element can be used to characterize the structure of the heating element, for example, to characterize the macroscopic structure of the heating element (which can be clearly observed by the naked eye), the surface microstructure, etc.

[0082] Step 220, determining characteristic information of the camera; wherein the characteristic information includes at least one of the following: specification information of a sensor included in the camera, imaging characteristic information of a lens included in the camera.

[0083] Optionally, the sensor included in the camera may be, for example, a complementary metal oxide semiconductor (CMOS) sensor. The specification information of the sensor is information that can reflect the specifications of the sensor. The specification information of the sensor may also be referred to as the pattern of the sensor. The sensor may include a photosensitive array, and the specification information of the sensor may include but is not limited to the number of first photosensitive elements of the photosensitive array along a first preset direction, the number of second photosensitive elements of the photosensitive array along a second preset direction, the first size of a single photosensitive element along the first preset direction, and the second size of a single photosensitive element along the second preset direction. The first preset direction may be, for example, a horizontal direction. The second preset direction may be, for example, a vertical direction. Both the first size and the second size may be in micrometer order.

[0084] Optionally, the lens included in the camera may also be referred to as a lens. The imaging characteristic information of the lens refers to information that can reflect the influence of the lens on the imaging. The imaging characteristic information of the lens may include, for example, a Line Spread Function (LSF) curve of the lens. It is understood that the Line Spread Function curve is a mathematical description of the brightness distribution of light in the image of a line light source, for example, see Figure 3-3 Of course, the imaging characteristic information of the lens is not limited thereto, and may include, for example, a Modulation Transfer Function (MTF) curve, etc., which will not be listed here one by one.

[0085] Step 230: Determine a first image including a light source area.

[0086] Optionally, the first image can be obtained by performing image modeling on the light source. The first image can include a light source area to present information such as the shape, size, and intensity of the light source. In addition to the light source area, the first image can also include a background area; wherein the light source area can be white and the background area can be black. The light source area can be the central area of ​​the first image.

[0087] Step 240: Generate a second image for representing the diffraction effect of the light source corresponding to the light source area based on the reference structure parameters, the feature information and the first image.

[0088] Optionally, a discrete Fresnel diffraction calculation method or other diffraction calculation methods can be used for calculation. On this basis, when the heating element meets the reference structural parameters and the camera meets the characteristic information, the diffraction effect of the light source can be superimposed on the first image to obtain a second image for representing the diffraction effect of the light source. The size of the second image can be the same as that of the first image.

[0089] For example, the second image may be Figure 4-1 As another example, the second image can be Figure 4-2 As another example, the second image can be as shown Figure 4-3 As another example, the second image can be as shown Figure 4-4 As another example, the second image can be as shown Figure 4-5 shown.

[0090] Step 250: Evaluate the reference structure parameters based on the second image to obtain evaluation results.

[0091] Optionally, the rationality of the reference structure parameters can be evaluated based on the diffraction effect reflected by the second image to obtain an evaluation result. The evaluation result can be in a binary classification form, used to characterize whether the evaluation of the reference structure parameters is passed or not. Of course, the evaluation result can also be in other forms, such as in the form of an evaluation score, where the higher the evaluation score, the more reasonable the reference structure parameters are, and the lower the evaluation parameter, the more unreasonable the reference structure parameters are.

[0092] Step 260 , based on the reference structural parameters and the evaluation result, target structural parameters of the heating element are determined, so that the camera can take pictures when the heating element meets the target structural parameters.

[0093] Optionally, the target structural parameters of the heating element can be determined based on the reference structural parameters in a manner that matches the evaluation results. The target structural parameters of the heating element can be understood as structural parameters that meet the requirements for the rationality of the final selection of the heating element. In a real vehicle scenario, the heating element set in the target glass can be a heating element that meets the target structural parameters, and the camera can be used to capture images through the target glass.

[0094] In an embodiment of the present disclosure, the glass that needs to be photographed by a camera can be called a target glass, and a heating element can be provided in the target glass to remove water vapor. Based on the reference structural parameters of the heater, the characteristic information of the camera, and the first image including the light source area, a second image can be generated to represent the diffraction effect of the light source corresponding to the light source area. With reference to the diffraction effect reflected in the second image, the rationality of the reference structural parameters can be effectively evaluated to obtain an evaluation result. Combining the reference structural parameters and the evaluation results, the target structural parameters of the heater can be reasonably determined. In this way, by searching for structural parameters (i.e., target structural parameters) that can minimize the diffraction effect for the heater, and applying the heater that meets the structural parameters to the actual vehicle scene, it is beneficial to improve the image quality of the image taken by the camera through the target glass, reduce the adverse effects of diffraction on the operation of the autonomous driving algorithm, and thus help improve driving safety.

[0095] In some optional examples, the heating element may include a special-shaped heating wire, and the reference structural parameters may include at least one of the following: heating wire diameter, heating wire shape parameters, heating wire period, heating wire amplitude, heating wire surface burr width, and heating wire surface burr spacing.

[0096] Alternatively, the heating wire diameter can be expressed as Figure 5-1 The E in the figure. The heating wire shape parameter can be used to characterize the shape of the special-shaped heating wire. The shape of the special-shaped heating wire can include but is not limited to a straight line, an S shape (specifically the shape of a sin function), a sawtooth shape, etc. The heating wire period can be expressed as Figure 5-1 The heating wire period can be understood as the horizontal distance between adjacent crests and troughs on the special-shaped heating wire. The heating wire amplitude can be expressed as Figure 5-1 B in the figure. The heating wire amplitude can be understood as the vertical distance between adjacent wave crests and wave troughs on the special-shaped heating wire. The burr width on the heating wire surface can be expressed as Figure 5-2 The width of the burr on the heating wire surface can be understood as the width of a single burr among the burrs on the surface of the special-shaped heating wire. The spacing between the burrs on the heating wire surface can be expressed as Figure 5-2 D in the figure. The burr spacing on the heating wire surface can be understood as the distance between two adjacent burrs along the horizontal direction among the burrs existing on the surface of the special-shaped heating wire.

[0097] It should be noted that the heating wire diameter, heating wire shape parameters, heating wire period, heating wire amplitude, etc. can effectively reflect the macro structure of the heating element, and the heating wire surface burr width and heating wire surface burr spacing can effectively reflect the surface micro structure of the heating element. In this way, by referring to the structural parameters, the structure of the heating element can be more completely and effectively reflected, and the target structural parameters can be determined on this basis, so that the macro structure and surface micro structure of the heating element in the target glass ultimately used in the actual vehicle scene can be as reasonable as possible, which is beneficial to improve the image quality of the image taken by the camera through the target glass, thereby improving driving safety.

[0098] Figure 6 is a flowchart of a method for generating a second image provided by some exemplary embodiments of the present disclosure. Figure 6 The method shown may include step 610, step 620, step 630, step 640 and step 650. Optionally, a combination of step 610 to step 650 may be used as an optional implementation of step 240 of the present disclosure.

[0099] Step 610: Perform image modeling on the heating element based on the reference structural parameters to obtain a third image.

[0100] Optionally, a third image may be obtained by performing image modeling on the heating element, and the third image may include a region of the heating element, so as to present the heating element that meets the reference structural parameters.

[0101] In one example, the reference structural parameters include the heating wire diameter and the heating wire shape parameter. The heating wire diameter may be in millimeter order. The heating wire shape parameter is used to characterize that the shape of the special-shaped heating wire is a straight line. Then the third image may be as follows: Figure 7-1 or Figure 7-2 shown.

[0102] In another example, the reference structural parameters include the heating wire diameter, the heating wire shape parameter, the heating wire period and the heating wire amplitude. The heating wire diameter can be in the millimeter level. The heating wire shape parameter is used to characterize that the shape of the special-shaped heating wire is S-shaped. The heating wire period and the heating wire amplitude can both be in the millimeter level. Then the third image can be as follows: Figure 7-3 or Figure 7-4 shown.

[0103] In another example, the reference structural parameters include the heating wire diameter, the heating wire shape parameter, the heating wire period and the heating wire amplitude. The heating wire diameter can be in the millimeter level. The heating wire shape parameter is used to characterize that the shape of the special-shaped heating wire is a sawtooth shape. The heating wire period and the heating wire amplitude can both be in the millimeter level. Then the third image can be as follows: Figure 7-5 shown.

[0104] Step 620: Determine a line spread function curve of the lens based on the imaging characteristic information in the feature information.

[0105] Optionally, the imaging characteristic information may include a line spread function curve of the lens, and then the line spread function curve of the lens may be directly extracted from the imaging characteristic information.

[0106] Step 630: convert the line spread function curve in the one-dimensional representation form into a two-dimensional representation form to obtain a fourth image.

[0107] It should be noted that the line spread function curve is in a one-dimensional representation form, and the line spread function curve can be considered as one-dimensional data. In step 630, the line spread function curve belonging to the one-dimensional data can be processed by dimensionality upscaling to obtain two-dimensional data, which can be in an image format and can be used as a fourth image.

[0108] In one example, the fourth image includes U rows and V columns, a total of U*V pixels. Then, sampling can be performed at equal intervals from left to right on the line diffusion function curve to obtain V discrete points, and the ordinate of the jth (j is greater than or equal to 1 and less than or equal to V) point among the V discrete points can be used as the actual light intensity of each of the U pixels located in the jth column in the fourth image. After obtaining the actual light intensity of each of the U*V pixels, the actual light intensity of each of the U*V pixels can be converted into pixel brightness, and the complete fourth image can be generated based on the pixel brightness of each of the U*V pixels.

[0109] Step 640, based on the third image, the fourth image, and the preset first diffraction light intensity calculation parameters, determine the diffraction light intensity corresponding to each pixel point in the first image.

[0110] Optionally, the preset first diffraction light intensity calculation parameter may be a preset parameter for assisting the calculation of the diffraction light intensity. The first diffraction light intensity calculation parameter may include, but is not limited to, a preset wavelength, a preset distance between the light source and the diffraction surface, a sampling spacing along the first preset direction on the diffraction surface, a sampling spacing along the second preset direction on the diffraction surface, a sampling spacing along the first preset direction on the observation surface, a sampling spacing along the second preset direction on the observation surface, and the like. The preset wavelength may be expressed as λ. The preset distance may be expressed as z. The sampling spacing along the first preset direction on the diffraction surface may be expressed as Δx0. The sampling spacing along the second preset direction on the diffraction surface may be expressed as Δy0. The sampling spacing along the first preset direction on the observation surface may be expressed as Δx1. The sampling spacing along the second preset direction on the observation surface may be expressed as Δy1.

[0111] In some optional embodiments of the present disclosure, Figure 8 As shown, step 640 may include step 810 , step 820 , and step 830 .

[0112] Step 810: Perform Fourier transform on the third image to obtain a fifth image.

[0113] Optionally, the fifth image may be obtained by performing Discrete Fourier Transform (DFT) on the third image.

[0114] Step 820: Perform convolution calculation on the fifth image and the fourth image to obtain a sixth image.

[0115] Optionally, the fourth image can be regarded as a matrix, and the fifth image can be regarded as another matrix. By using the formula for convolution calculation between matrices in the relevant technology, the convolution calculation result of the fifth image and the fourth image can be obtained, that is, the sixth image can be obtained.

[0116] Step 830: Determine the diffraction light intensity corresponding to each pixel point in the first image based on the sixth image and the first diffraction light intensity calculation parameter.

[0117] Optionally, there may be a one-to-one correspondence between each photosensitive element in the photosensitive array included in the sensor and each pixel point in the first image. In this case, the following formula may be used to determine the diffracted light intensity corresponding to each pixel point in the first image:

[0118]

[0119] Among them, U1(p△x1,q△y1) represents the diffraction light intensity corresponding to the pixel point with abscissa p△x1 and ordinate q△y1 in the first image, and the pixel point is located in the qth row and pth column in the first image. The pixel point can correspond to the photosensitive element located in the qth row and pth column in the photosensitive array. e is a constant in mathematics, approximately equal to 2.71828. i represents an imaginary number. k is equal to 2π / λ, λ represents a preset wavelength. z represents a preset distance. U0(m△x0,n△y0) represents the actual light intensity of the pixel point with abscissa m△x0 and ordinate n△y0 in the sixth image, and the pixel point is located in the nth row and mth column in the sixth image. Optionally, the size of the sixth image can be the same as the size of the first image, then m and p can be the same, and n and q can be the same.

[0120] In some embodiments, the above formula may be deformed or fine-tuned according to actual needs, such as adding coefficients to certain items in the formula.

[0121] It can be seen that based on the third image and the fourth image, by sequentially performing Fourier transform and convolution calculation, the sixth image can be obtained efficiently and reliably. The sixth image can be considered as the result of fusing the third image and the fourth image, and the information it carries includes both reference structure parameters and line spread function curves. In this way, based on the sixth image and the first diffraction light intensity calculation parameters, it is possible to effectively predict the effect of the diffraction effect of the light source on the image quality when the heating element meets the reference structure parameters and the lens meets the line spread function curve through calculation, and on this basis, the diffraction light intensity corresponding to each pixel point in the first image is obtained, which is conducive to ensuring the rationality and reliability of the obtained diffraction light intensities.

[0122] In some other optional embodiments of the present disclosure, Fig. 9 As shown, step 640 may include step 910 , step 920 , and step 930 .

[0123] Step 910: Perform convolution calculation on the third image and the fourth image to obtain a seventh image.

[0124] Optionally, the third image can be regarded as a matrix, and the fourth image can be regarded as another matrix. By using the formula for convolution calculation between matrices in the relevant technology, the convolution calculation result of the third image and the fourth image can be obtained, that is, the seventh image is obtained.

[0125] Step 920: Perform Fourier transform on the seventh image to obtain an eighth image.

[0126] Optionally, the eighth image may be obtained by performing discrete Fourier transform on the seventh image.

[0127] Step 930: Determine the diffraction light intensity corresponding to each pixel point in the first image based on the eighth image and the first diffraction light intensity calculation parameter.

[0128] Optionally, in step 930, Figure 8 The formula introduced in the illustrated implementation manner determines the diffraction light intensity corresponding to each pixel point in the first image, and U0(m△x0,n△y0) in the formula represents the actual light intensity of the pixel point with abscissa m△x0 and ordinate n△y0 in the eighth image.

[0129] It can be seen that based on the third image and the fourth image, the eighth image can be obtained efficiently and reliably by performing convolution calculation and Fourier transform in sequence. The eighth image can be considered as the result of fusing the third image and the fourth image, and the information it carries includes both reference structure parameters and line spread function curves. In this way, based on the eighth image and the first diffraction light intensity calculation parameters, it is possible to effectively predict the effect of the diffraction effect of the light source on the image quality when the heating element meets the reference structure parameters and the lens meets the line spread function curve through calculation, and on this basis, the diffraction light intensity corresponding to each pixel point in the first image is obtained, which is conducive to ensuring the rationality and reliability of the obtained diffraction light intensities.

[0130] Regardless of which implementation method is adopted in step 640, step 650 may be performed after the diffraction light intensity corresponding to each pixel point in the first image is determined.

[0131] Step 650: Generate a second image based on the diffracted light intensity corresponding to each pixel point in the first image.

[0132] Optionally, the size of the second image can be the same as the size of the first image. Then, for the pixel point located in the i-th row and j-th column in the first image, the actual light intensity of the pixel point can be multiplied by the diffracted light intensity corresponding to the pixel point, and the obtained product can be used as the actual light intensity of the pixel point located in the i-th row and j-th column in the second image. Next, the actual light intensity of the pixel point located in the i-th row and j-th column in the second image can be converted into pixel brightness, and a complete second image can be generated based on the pixel brightness of all the pixels in the second image.

[0133] In the embodiments of the present disclosure, by modeling the image of the heating element, a third image carrying reference structural parameters can be obtained efficiently and reliably. Based on the line spread function curve extracted from the imaging characteristic information, a fourth image carrying imaging characteristic information can be obtained efficiently and reliably. In combination with the third image, the fourth image and the preset first diffraction light intensity calculation parameters, it is possible to effectively predict through calculation the influence of the diffraction effect of the light source on the image quality when the heating element meets the reference structural parameters and the lens meets the imaging characteristic information, and on this basis, the diffraction light intensity corresponding to each pixel point in the first image is obtained, which is conducive to ensuring the rationality and reliability of these diffraction light intensities obtained. In this way, an effective reference can be provided for the generation of the second image, which is conducive to ensuring the rationality and reliability of the generated second image.

[0134] Fig.10 It is a flowchart of a method for generating a second image provided by some other exemplary embodiments of the present disclosure. Fig.10 The method shown may include step 1010, step 1020, step 1030 and step 1040. Optionally, a combination of step 1010 to step 1040 may be used as an optional implementation of step 240 of the present disclosure.

[0135] Step 1010: Perform image modeling on the heating element based on the reference structural parameters to obtain a third image.

[0136] Optionally, the specific implementation of step 1010 may refer to the above description of step 610 and will not be repeated here.

[0137] Step 1020: Determine the number of first photosensitive elements along a first preset direction and the number of second photosensitive elements along a second preset direction in a photosensitive array included in the sensor based on the specification information in the characteristic information.

[0138] Optionally, the specification information may include the number of first photosensitive elements and the number of second photosensitive elements. Then, the number of first photosensitive elements and the number of second photosensitive elements may be directly extracted from the specification information.

[0139] Step 1030, based on the third image, the number of first photosensitive elements, the number of second photosensitive elements, the preset second diffraction light intensity calculation parameters, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image, determine the diffraction light intensity corresponding to each pixel point in the first image.

[0140] Optionally, the preset second diffraction light intensity calculation parameter may be a preset parameter for assisting the diffraction light intensity calculation. Different from the first diffraction light intensity calculation parameter, the second diffraction light intensity calculation parameter may not include a sampling interval, but include a preset wavelength and a preset distance between the light source and the diffraction surface.

[0141] Optionally, there may be a one-to-one correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image. For example, the photosensitive element located in the i-th row and j-th column in the photosensitive array corresponds to the pixel point located in the i-th row and j-th column in the first image.

[0142] In some optional embodiments of the present disclosure, Fig.11 As shown, step 1030 may include step 1110 , step 1120 , step 1130 , step 1140 and step 1150 .

[0143] Step 1110, performing Fourier transform on the third image to obtain a fifth image.

[0144] Optionally, the fifth image may be obtained by performing discrete Fourier transform on the third image.

[0145] Step 1120, based on the second diffraction light intensity calculation parameter, determine the preset wavelength and the preset distance between the light source and the diffraction surface.

[0146] Since the second diffraction light intensity calculation parameters may include a preset wavelength and a preset distance, the preset wavelength and the preset distance may be directly extracted from the second diffraction light intensity calculation parameters.

[0147] Step 1130: Determine a first sampling interval corresponding to a first preset direction based on a preset wavelength, a preset distance, and the number of first photosensitive elements.

[0148] Step 1140, determining a second sampling interval corresponding to a second preset direction based on a preset wavelength, a preset distance, and a second number of photosensitive elements.

[0149] In an example, the preset wavelength can be expressed as λ, the preset distance can be expressed as z, the number of first photosensitive elements can be expressed as M, the first sampling interval corresponding to the first preset direction can be expressed as Δx, the number of second photosensitive elements can be expressed as N, and the second sampling interval corresponding to the second preset direction can be expressed as Δy, then:

[0150]

[0151] Of course, the calculation formula for the first sampling interval and the second sampling interval is not limited thereto. For example, the right side of the equal sign in the above formula for obtaining Δx may be multiplied by a coefficient.

[0152] Step 1150, based on the fifth image, the second diffraction light intensity calculation parameters, the first sampling interval, the second sampling interval, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image, determine the diffraction light intensity corresponding to each pixel point in the first image.

[0153] Optionally, in step 1150, Figure 8 The formula introduced in the illustrated embodiment determines the diffraction light intensity corresponding to each pixel point in the first image, and lets U0(m△x0,n△y0) in the formula represent the actual light intensity of the pixel point with abscissa m△x0 and ordinate n△y0 in the fifth image, and lets Δx0 and Δx1 be equal to the first sampling spacing, and Δy0 and Δy1 be equal to the second sampling spacing.

[0154] It can be seen that based on the third image, the fifth image can be obtained efficiently and reliably through Fourier transform, and the information carried by the fifth image can include reference structure parameters. Based on the second diffraction light intensity calculation parameter, the preset wavelength and preset distance can be obtained efficiently and reliably, and combined with the first photosensitive element number and the second photosensitive element number extracted from the specification information, the first sampling distance and the second sampling distance can be determined in real time. Combined with the fifth image, the second diffraction light intensity calculation parameter, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image, it is possible to effectively predict through calculation the impact of the diffraction effect of the light source on the image quality when the heating element meets the reference structure parameters and the sensor meets the specification information, and on this basis, the diffraction light intensity corresponding to each pixel point in the first image is obtained, which is conducive to ensuring the rationality and reliability of these diffraction light intensities obtained.

[0155] Of course, the implementation method of step 1030 is not limited to this. For example, after obtaining the third image, the third image can be Fourier transformed to obtain a fifth image, and the fifth image and the fourth image can be convolved to obtain a sixth image. Based on the sixth image, the number of first photosensitive elements, the number of second photosensitive elements, the preset second diffraction light intensity calculation parameters, and the correspondence between each photosensitive element in the photosensitive array and each pixel in the first image, the diffraction light intensity corresponding to each pixel in the first image is determined.

[0156] Step 1040: Generate a second image based on the diffracted light intensity corresponding to each pixel point in the first image.

[0157] Optionally, the specific implementation of step 1040 may refer to the above description of step 650 and will not be repeated here.

[0158] In the embodiments of the present disclosure, by modeling the image of the heating element, a third image carrying the reference structural parameters can be obtained efficiently and reliably. Combined with the third image, the number of first photosensitive elements and the number of second photosensitive elements extracted from the specification information, the preset second diffraction light intensity calculation parameters, and the correspondence between each photosensitive element in the photosensitive array and each pixel in the first image, it is possible to effectively predict through calculation the impact of the diffraction effect of the light source on the image quality when the heating element meets the reference structural parameters and the lens meets the imaging characteristic information, and on this basis, obtain the diffraction light intensity corresponding to each pixel in the first image, which is conducive to ensuring the rationality and reliability of these diffraction light intensities. In this way, an effective reference can be provided for the generation of the second image, which is conducive to ensuring the rationality and reliability of the generated second image.

[0159] Figure 12-1 It is a flowchart of a parameter evaluation method provided by some exemplary embodiments of the present disclosure. Figure 12-1 The method shown may include step 1210 and step 1220. Optionally, the combination of step 1210 and step 1220 may be used as an optional implementation of step 250 of the present disclosure.

[0160] Step 1210: Generate a diffraction light intensity energy level distribution diagram based on the second image.

[0161] Optionally, the imaging characteristic information of the lens may include multiple line spread function curves. For example, the imaging characteristic information of the lens may include the line spread function curve of the lens along the meridian direction at the first wavelength, the line spread function curve along the meridian direction at the second wavelength, the line spread function curve along the meridian direction at the third wavelength, the line spread function curve along the sagittal direction at the first wavelength, the line spread function curve along the sagittal direction at the second wavelength, and the line spread function curve along the sagittal direction at the third wavelength, etc. Then, the multiple line spread function curves may specifically be 6 line spread function curves. Here, the first wavelength may be a wavelength within the wavelength range corresponding to red light. The second wavelength may be a wavelength within the wavelength range corresponding to green light. The third wavelength may be a wavelength within the wavelength range corresponding to blue light.

[0162] For each of the six line spread function curves, it can be converted from a one-dimensional representation form to a two-dimensional representation form to obtain a corresponding fourth image. In this way, a total of six fourth images can be obtained.

[0163] For each of the six fourth images, the above steps 640 to 650 may be performed to generate a corresponding second image. In this way, a total of six second images may be obtained.

[0164] It should be noted that among the 6 second images, there are 3 second images corresponding to the meridian direction and 3 second images corresponding to the sagittal direction, and all the 6 second images are single-channel images. Then, the 3 second images corresponding to the meridian direction can be superimposed along the channel direction to obtain an RGB image. For the RGB image, the diffraction light intensity energy level distribution diagram can be determined by analyzing and counting the light intensity in different regions. Similarly, the 3 second images corresponding to the sagittal direction can be superimposed along the channel direction to obtain another RGB image. For the RGB image, the diffraction light intensity energy level distribution diagram can be determined by analyzing and counting the light intensity in different regions. In the diffraction light intensity energy level distribution diagram, the horizontal axis can represent the distance from the upper edge of the image (or the lower edge of the image), and the vertical axis can represent the diffraction light intensity.

[0165] In one example, the diffracted light intensity level distribution diagram can be as follows: Figure 12-2 or Figure 12-3 As shown, it may correspond to reference structural parameters including heating wire diameter and heating wire shape parameters, the heating wire diameter is in millimeter order, and the heating wire shape parameters are used to characterize the situation where the shape of the special-shaped heating wire is a straight line.

[0166] In another example, the diffracted light intensity level distribution diagram can be as follows Figure 12-4 or Figure 12-5 As shown, it can correspond to the reference structural parameters including the heating wire diameter, the heating wire shape parameters, the heating wire period and the heating wire amplitude. The heating wire diameter is in the millimeter level. The heating wire shape parameters are used to characterize that the shape of the special-shaped heating wire is S-shaped. The heating wire period and the heating wire amplitude are both in the millimeter level.

[0167] In another example, the diffracted light intensity level distribution diagram can be as follows Figure 12-6 As shown, it can correspond to the reference structural parameters including the heating wire diameter, the heating wire shape parameters, the heating wire period and the heating wire amplitude. The heating wire diameter is in the millimeter level. The heating wire shape parameters are used to characterize that the shape of the special-shaped heating wire is sawtooth-shaped. The heating wire period and the heating wire amplitude are both in the millimeter level.

[0168] By observation Figure 12-2 to Figure 12-6 It can be seen that the diffraction light intensity energy level distribution diagram may include a left region where the diffraction light intensity changes frequently, a middle region where the diffraction light intensity is basically kept at zero, and a right region where the diffraction light intensity changes frequently.

[0169] Step 1220, based on the diffraction light intensity energy level distribution diagram, evaluate the reference structure parameters to obtain evaluation results.

[0170] Assume that there are two diffraction light intensity energy level distribution diagrams, namely the diffraction light intensity energy level distribution diagram corresponding to the meridional direction and the diffraction light intensity energy level distribution diagram corresponding to the sagittal direction. For the convenience of description, the diffraction light intensity energy level distribution diagram corresponding to the meridional direction can be called the first diffraction light intensity energy level distribution diagram, and the diffraction light intensity energy level distribution diagram corresponding to the sagittal direction can be called the second diffraction light intensity energy level distribution diagram.

[0171] For the first diffraction light intensity energy level distribution diagram, the number of peaks per unit length and the diffraction light intensity corresponding to each peak in the left area or the right area included therein can be determined, and based on this, it can be judged whether the rationality of the reference structure parameters corresponding to the meridian direction meets the requirements. For example, if the number of peaks per unit length is less than the preset number, and the diffraction light intensity corresponding to most of the peaks is less than the preset light intensity, it can be judged that the rationality of the reference structure parameters corresponding to the meridian direction meets the requirements. If the number of peaks per unit length is greater than or equal to the preset number, and / or the diffraction light intensity corresponding to most of the peaks is greater than the preset light intensity, it can be judged that the rationality of the reference structure parameters corresponding to the meridian direction does not meet the requirements.

[0172] Similarly, for the second diffraction light intensity energy level distribution diagram, the number of peaks per unit length and the diffraction light intensity corresponding to each peak in the left area or the right area included therein can be determined, and based on this, it can be judged whether the rationality of the reference structure parameters corresponding to the sagittal arc direction meets the requirements.

[0173] If the rationality of the reference structural parameters meets the requirements for both the meridian direction and the sagittal arc direction, the evaluation result can be used to indicate that the evaluation of the reference structural parameters has passed. If the rationality of the reference structural parameters does not meet the requirements for at least one of the meridian direction and the sagittal arc direction, the evaluation result can be used to indicate that the evaluation of the reference structural parameters has not passed.

[0174] In some embodiments, the number of line spread function curves is not limited to 6. For example, multiple wavelengths can be selected from the wavelength range corresponding to red light, and each wavelength in the multiple wavelengths can be used as a first wavelength. Similarly, multiple wavelengths can be selected from the wavelength range corresponding to green light, and each wavelength in the multiple wavelengths can be used as a second wavelength. Similarly, multiple wavelengths can also be selected from the wavelength range corresponding to blue light, and each wavelength in the multiple wavelengths can be used as a third wavelength. In this way, the number of line spread function curves can be an integer multiple of 6 (which can be 1), and these line spread functions can all be used to obtain the corresponding fourth image, and further obtain the corresponding second image based on this.

[0175] In an embodiment of the present disclosure, a diffraction light intensity energy level distribution map can be generated based on the second image. The diffraction light intensity energy level distribution map can effectively characterize the distribution of the diffraction light intensity. In this way, it can provide an effective reference for the evaluation of the reference structure parameters, which is conducive to ensuring the accuracy and reliability of the evaluation results.

[0176] Fig.13 It is a flowchart of parameter evaluation methods provided by other exemplary embodiments of the present disclosure. Fig.13 The method shown may include step 1310 and step 1320. Optionally, the combination of step 1310 and step 1320 may be used as an optional implementation of step 250 of the present disclosure.

[0177] Step 1310: Perform a machine vision task on the second image through a neural network model to obtain a task execution result.

[0178] Optionally, the neural network model may be, for example, a target detection model for detecting a light source region in an image, and accordingly, the machine vision task may be a target detection task. Thus, the task execution result obtained by executing the machine vision task on the second image may be target detection information for the light source region. The target detection information may be represented, for example, by a one- or two-dimensional detection frame (for framing the light source region) in the second image.

[0179] Step 1320, based on the task execution result, evaluate the reference structure parameters to obtain an evaluation result.

[0180] Optionally, the light source area in the second image can be manually framed to form annotation information corresponding to the second image. The annotation information corresponding to the second image can be represented by a labeling box in the second image. After obtaining the annotation information and the target detection information, the error between the annotation information and the target detection information can be calculated. If the calculated error is within the allowable range, the evaluation result can be used to characterize that the evaluation of the reference structure parameters has passed. If the calculated error is outside the allowable range, the evaluation result can be used to characterize that the evaluation of the reference structure parameters has failed.

[0181] Generally speaking, during the actual operation of the autonomous driving algorithm, several neural network models are often used for calculation. In the embodiments of the present disclosure, by using the neural network model for calculation during the parameter evaluation process, it is possible to ensure that the evaluation results are consistent with the actual vehicle scene, so that the rationality of the target structural parameters is better guaranteed. Using a heating element that meets the target structural parameters in an actual vehicle scene is conducive to improving the image quality of the image taken by the camera through the target glass, thereby helping to improve driving safety.

[0182] It should be noted that Fig.13The neural network model involved in the illustrated embodiment is not limited to the target detection model, but may also be other models required to be used in the autonomous driving algorithm, and the present disclosure does not limit this.

[0183] Fig.14 It is a flowchart of a method for determining target structure parameters provided by some exemplary embodiments of the present disclosure. Fig.14 The method shown may include step 1410, step 1420 and step 1430. Optionally, the combination of step 1420 and step 1430 may be an optional implementation of step 260 of the present disclosure.

[0184] Step 1410: Evaluate the reference structure parameters based on the second image to obtain evaluation results.

[0185] Optionally, the specific implementation of step 1410 may refer to the above description of step 250 and will not be repeated here.

[0186] Step 1420 , in response to the evaluation result indicating that the evaluation of the reference structural parameters is passed, the reference structural parameters are determined as target structural parameters.

[0187] Step 1430 , in response to the evaluation result indicating that the evaluation of the reference structural parameters has failed, the reference structural parameters are updated, and based on the updated reference structural parameters and the characteristic information, target structural parameters of the heating element are determined.

[0188] Optionally, if the evaluation result indicates that the evaluation of the reference structure parameters is passed, which means that the reference structure parameters are applicable to the actual vehicle scene, then the reference structure parameters can be directly determined as the target structure parameters. If the evaluation result indicates that the evaluation of the reference structure parameters is not passed, which means that the reference structure parameters are not applicable to the actual vehicle scene, then the reference structure parameters can be updated. For example, the reference structure parameters include the heating wire diameter, heating wire shape parameters, heating wire period, heating wire amplitude, heating wire surface burr width, and heating wire surface burr spacing. At least one of the six parameters can be changed to obtain the updated reference structure parameters. Based on the updated reference structure parameters, feature information, and the first image, an image for representing the diffraction effect of the light source corresponding to the light source area can be generated in a manner similar to the method of generating the second image described above. Next, the reference structure parameters can be evaluated based on the generated image, and the target structure parameters can be determined based on the updated reference structure parameters and the obtained evaluation results.

[0189] In an embodiment of the present disclosure, the reference structural parameters are evaluated based on the second image. After the evaluation results are obtained, the target structural parameters of the heating element can be determined in a determination method that is compatible with the evaluation results, which is conducive to ensuring the rationality and reliability of the target structural parameters finally determined.

[0190] In some optional examples, such as Figure 15-1 , Figure 15-2 As shown, the linear spread function curve of the lens included in the camera, the specification information of the sensor included in the camera, and the reference structural parameters of the heating element set in the target glass can be determined. In addition, some simulation parameters need to be set, including but not limited to object distance, saturation, light source intensity, light source size, etc. Based on the above information, a discrete Fresnel diffraction calculation method can be used to calculate the diffracted light intensity so as to generate a second image based on the calculated diffracted light intensity. Optionally, the process of obtaining the diffracted light intensity based on the above information and the process of generating the second image can be implemented by computer simulation software.

[0191] like Figure 15-1 As shown, the preset neural network model can be used to perform the machine vision task on the second image to obtain the task execution result, and judge whether the rationality of the reference structure parameters meets the requirements. Or, as Figure 15-2 As shown, the diffraction light intensity energy level distribution diagram can be determined based on the second image, so as to judge whether the rationality of the reference structure parameters meets the requirements according to the diffraction light intensity energy level distribution diagram.

[0192] If the rationality of the reference structure parameters meets the requirements, the reference structure parameters can be used as the target structure parameters. If the rationality of the reference structure parameters does not meet the requirements, the reference structure parameters can be modified until they meet the requirements. In this way, more accurate and reliable target structure parameters can be obtained. In the actual vehicle scene, a heating element that meets the target structure parameters can be used to improve the image quality of the image taken by the camera through the target glass and improve driving safety.

[0193] Exemplary Devices

[0194] Fig.16 is a schematic diagram of the structure of an image capturing device provided by some exemplary embodiments of the present disclosure. Fig.16 The apparatus shown may include:

[0195] The first determination module 1610 is used to determine the reference structural parameters of the heating element provided in the target glass; wherein the target glass is the glass provided on the photosensitive side of the camera;

[0196] The second determination module 1620 is used to determine the characteristic information of the camera; wherein the characteristic information includes at least one of the following: specification information of the sensor included in the camera, imaging characteristic information of the lens included in the camera;

[0197] A third determining module 1630 is used to determine a first image including a light source area;

[0198] A generating module 1640, configured to generate a second image representing a diffraction effect of a light source corresponding to the light source region based on the reference structure parameters determined by the first determining module 1610, the feature information determined by the second determining module 1620, and the first image determined by the third determining module 1630;

[0199] An evaluation module 1650 is used to evaluate the reference structure parameters determined by the first determination module 1610 based on the second image generated by the generation module 1640 to obtain an evaluation result;

[0200] The fourth determination module 1660 is used to determine the target structural parameters of the heating element based on the reference structural parameters determined by the first determination module 1610 and the evaluation result obtained by the evaluation module 1650, so that the camera can take pictures when the heating element meets the target structural parameters.

[0201] In some optional examples, such as Fig.17 As shown, the generating module 1640 includes:

[0202] A first modeling submodule 1710 is used to perform image modeling on the heating element based on the reference structure parameters determined by the first determining module 1610 to obtain a third image;

[0203] A first determining submodule 1720, configured to determine a line spread function curve of the lens based on the imaging characteristic information in the feature information determined by the second determining module 1620;

[0204] A conversion submodule 1730 is used to convert the line spread function curve in the one-dimensional representation form determined by the first determination submodule 1720 into a two-dimensional representation form to obtain a fourth image;

[0205] A second determination submodule 1740 is used to determine the diffraction light intensity corresponding to each pixel point in the first image based on the third image obtained by the first modeling submodule 1710, the fourth image obtained by the conversion submodule 1730, and the preset first diffraction light intensity calculation parameters;

[0206] The first generating submodule 1750 is used to generate a second image based on the diffracted light intensity corresponding to each pixel point in the first image determined by the second determining submodule 1740.

[0207] In some optional examples, such as Fig.18 As shown, the second determining submodule 1740 includes:

[0208] A first transform unit 1810 is used to perform Fourier transform on the third image obtained by the first modeling submodule 1710 to obtain a fifth image;

[0209] a first convolution unit 1820, configured to perform convolution calculation on the fifth image obtained by the first transformation unit 1810 and the fourth image obtained by the conversion submodule 1730 to obtain a sixth image;

[0210] The first determination unit 1830 is used to determine the diffraction light intensity corresponding to each pixel point in the first image based on the sixth image obtained by the first convolution unit 1820 and the first diffraction light intensity calculation parameter.

[0211] In some optional examples, such as Fig.19 As shown, the second determining submodule 1740 includes:

[0212] a second convolution unit 1910, configured to perform convolution calculation on the third image obtained by the first modeling submodule 1710 and the fourth image obtained by the conversion submodule 1730 to obtain a seventh image;

[0213] A second transform unit 1920 is used to perform Fourier transform on the seventh image obtained by the second convolution unit 1910 to obtain an eighth image;

[0214] The second determination unit 1930 is used to determine the diffraction light intensity corresponding to each pixel point in the first image based on the eighth image obtained by the second transformation unit 1920 and the first diffraction light intensity calculation parameter.

[0215] In some optional examples, such as Fig. 20 As shown, the generating module 1640 includes:

[0216] A second modeling submodule 2010 is used to perform image modeling on the heating element based on the reference structural parameters determined by the first determining module 1610 to obtain a third image;

[0217] The third determination submodule 2020 is used to determine the number of first photosensitive elements along the first preset direction and the number of second photosensitive elements along the second preset direction in the photosensitive array included in the sensor based on the specification information in the feature information determined by the second determination module 1620;

[0218] The fourth determination submodule 2030 is used to determine the diffraction light intensity corresponding to each pixel point in the first image based on the third image obtained by the second modeling submodule 2010, the number of first photosensitive elements determined by the third determination submodule 2020, the number of second photosensitive elements determined by the third determination submodule 2020, the preset second diffraction light intensity calculation parameter, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image.

[0219] The second generating submodule 2040 is used to generate a second image based on the diffracted light intensity corresponding to each pixel point in the first image determined by the fourth determining submodule 2030.

[0220] In some optional examples, such as Fig.21 As shown, the fourth determining submodule 2030 includes:

[0221] A third transform unit 2110 is used to perform Fourier transform on the third image obtained by the second modeling submodule 2010 to obtain a fifth image;

[0222] A third determining unit 2120, configured to determine a preset wavelength and a preset distance between the light source and the diffraction surface based on the second diffraction light intensity calculation parameter;

[0223] The fourth determining unit 2130 is used to determine a first sampling interval corresponding to the first preset direction based on the preset wavelength and the preset distance of the third determining unit 2120 and the number of first photosensitive elements determined by the third determining submodule 2020;

[0224] A fifth determining unit 2140 is configured to determine a second sampling interval corresponding to a second preset direction based on the preset wavelength and the preset distance of the third determining unit 2120 and the number of second photosensitive elements determined by the third determining submodule 2020;

[0225] The sixth determination unit 2150 is used to determine the diffraction light intensity corresponding to each pixel point in the first image based on the fifth image obtained by the third transformation unit 2110, the second diffraction light intensity calculation parameter, the first sampling spacing determined by the fourth determination unit 2130, the second sampling spacing determined by the fifth determination unit 2140, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image.

[0226] In some optional examples, such as Fig. 22 As shown, the evaluation module 1650 includes:

[0227] The third generating submodule 2210 is used to generate a diffraction light intensity energy level distribution diagram based on the second image generated by the generating module 1640;

[0228] The first evaluation submodule 2220 is used to evaluate the reference structure parameters determined by the first determination module 1610 based on the diffraction light intensity energy level distribution diagram generated by the third generation submodule 2210 to obtain an evaluation result.

[0229] In some optional examples, such as Fig.23 As shown, the evaluation module 1650 includes:

[0230] The task execution submodule 2310 is used to execute the machine vision task on the second image generated by the generation module 1640 through the neural network model to obtain a task execution result;

[0231] The second evaluation submodule 2320 is used to evaluate the reference structure parameters determined by the first determination module 1610 based on the task execution result obtained by the task execution submodule 2310 to obtain an evaluation result.

[0232] In some optional examples, such as Fig.24 As shown, the fourth determination module 1660 includes:

[0233] A fifth determining submodule 2410 is configured to determine the reference structural parameters as target structural parameters in response to the evaluation result obtained by the evaluation module 1650 indicating that the evaluation of the reference structural parameters is successful;

[0234] The sixth determination submodule 2420 is used to update the reference structural parameters in response to the evaluation result obtained by the evaluation module 1650 indicating that the evaluation of the reference structural parameters has failed, and determine the target structural parameters of the heating element based on the updated reference structural parameters and characteristic information.

[0235] In some optional examples, the heating element includes a special-shaped heating wire, and the reference structural parameters include at least one of the following: heating wire diameter, heating wire shape parameters, heating wire period, heating wire amplitude, heating wire surface burr width, and heating wire surface burr spacing.

[0236] In the device of the present disclosure, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve corresponding functions and effects, and the present disclosure does not list them one by one.

[0237] Exemplary Electronic Devices

[0238] Fig.25 The block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. The electronic device 2500 includes one or more processors 2510 and a memory 2520 .

[0239] The processor 2510 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 2500 to perform desired functions.

[0240] The memory 2520 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 2510 may execute one or more computer program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0241] In one example, the electronic device 2500 may further include: an input device 2530 and an output device 2540 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0242] The input device 2530 may also include, for example, a keyboard, a mouse, etc.

[0243] The output device 2540 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0244] Of course, to simplify, Fig.25 Only some of the components related to the present disclosure in the electronic device 2500 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 2500 may also include any other appropriate components.

[0245] Exemplary computer program products and computer-readable storage media

[0246] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0247] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0248] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0249] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0250] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. The specific details disclosed above are only for the purpose of illustration and ease of understanding, not limitation. The above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0251] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. An image capturing method, comprising: Determining reference structural parameters of a heating element disposed in a target glass; wherein the target glass is a glass disposed on a photosensitive side of a camera; Determine characteristic information of the camera; wherein the characteristic information includes at least one of the following: specification information of a sensor included in the camera, imaging characteristic information of a lens included in the camera; determining a first image including a light source area; Based on the reference structure parameters, the characteristic information and the first image, generating a second image for representing the diffraction effect of the light source corresponding to the light source area; Based on the second image, evaluating the reference structure parameters to obtain an evaluation result; Based on the reference structural parameters and the evaluation result, target structural parameters of the heating element are determined, so that the camera takes pictures when the heating element meets the target structural parameters.

2. The method according to claim 1, wherein: The step of generating a second image for representing a diffraction effect of a light source corresponding to the light source area based on the reference structure parameter, the characteristic information and the first image comprises: Based on the reference structural parameters, image modeling is performed on the heating element to obtain a third image; Determining a line spread function curve of the lens based on the imaging characteristic information in the feature information; Convert the line spread function curve in a one-dimensional representation form into a two-dimensional representation form to obtain a fourth image; Determine the diffracted light intensity corresponding to each pixel point in the first image based on the third image, the fourth image, and the preset first diffracted light intensity calculation parameter; The second image is generated based on the diffracted light intensity corresponding to each pixel point in the first image.

3. The method according to claim 2, wherein: The determining, based on the third image, the fourth image, and the preset first diffraction light intensity calculation parameter, the diffraction light intensity corresponding to each pixel point in the first image includes: Performing Fourier transform on the third image to obtain a fifth image; Performing convolution calculation on the fifth image and the fourth image to obtain a sixth image; Based on the sixth image and the first diffraction light intensity calculation parameter, the diffraction light intensity corresponding to each pixel point in the first image is determined.

4. The method according to claim 2, wherein: The determining, based on the third image, the fourth image, and the preset first diffraction light intensity calculation parameter, the diffraction light intensity corresponding to each pixel point in the first image includes: Performing convolution calculation on the third image and the fourth image to obtain a seventh image; Performing Fourier transform on the seventh image to obtain an eighth image; Based on the eighth image and the first diffraction light intensity calculation parameter, the diffraction light intensity corresponding to each pixel point in the first image is determined.

5. The method according to claim 1, wherein: The step of generating a second image for representing a diffraction effect of a light source corresponding to the light source area based on the reference structure parameter, the characteristic information and the first image comprises: Based on the reference structural parameters, image modeling is performed on the heating element to obtain a third image; Based on the specification information in the characteristic information, determining the number of first photosensitive elements along a first preset direction and the number of second photosensitive elements along a second preset direction in the photosensitive array included in the sensor; Determine the diffraction light intensity corresponding to each pixel point in the first image based on the third image, the number of the first photosensitive elements, the number of the second photosensitive elements, a preset second diffraction light intensity calculation parameter, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image; The second image is generated based on the diffracted light intensity corresponding to each pixel point in the first image.

6. The method according to claim 5, wherein: The determining of the diffraction light intensity corresponding to each pixel point in the first image based on the third image, the number of the first photosensitive elements, the number of the second photosensitive elements, the preset second diffraction light intensity calculation parameter, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image includes: Performing Fourier transform on the third image to obtain a fifth image; Determine a preset wavelength and a preset distance between the light source and the diffraction surface based on the second diffraction light intensity calculation parameter; Determine a first sampling interval corresponding to the first preset direction based on the preset wavelength, the preset distance and the number of the first photosensitive elements; Determine a second sampling interval corresponding to the second preset direction based on the preset wavelength, the preset distance and the number of the second photosensitive elements; Based on the fifth image, the second diffraction light intensity calculation parameters, the first sampling spacing, the second sampling spacing, and the correspondence between each photosensitive element in the photosensitive array and each pixel point in the first image, the diffraction light intensity corresponding to each pixel point in the first image is determined.

7. The method according to any one of claims 1 to 6, wherein: The step of evaluating the reference structure parameter based on the second image to obtain an evaluation result includes: Based on the second image, generating a diffraction light intensity energy level distribution diagram; Based on the diffraction light intensity energy level distribution diagram, evaluating the reference structure parameters to obtain an evaluation result; or, The step of evaluating the reference structure parameter based on the second image to obtain an evaluation result includes: Performing a machine vision task on the second image through a neural network model to obtain a task execution result; Based on the task execution result, the reference structure parameters are evaluated to obtain an evaluation result.

8. The method according to any one of claims 1 to 6, wherein: The step of determining target structural parameters of the heating element based on the reference structural parameters and the evaluation result comprises: In response to the evaluation result indicating that the evaluation of the reference structural parameters is passed, determining the reference structural parameters as the target structural parameters; In response to the evaluation result indicating that the evaluation of the reference structural parameters fails, the reference structural parameters are updated, and target structural parameters of the heating element are determined based on the updated reference structural parameters and the characteristic information.

9. The method according to any one of claims 1 to 6, wherein: The heating element includes a special-shaped heating wire, and the reference structural parameters include at least one of the following: heating wire diameter, heating wire shape parameter, heating wire period, heating wire amplitude, heating wire surface burr width, and heating wire surface burr spacing.

10. An image capturing device, comprising: A first determination module is used to determine reference structural parameters of a heating element provided in a target glass; wherein the target glass is glass provided on a photosensitive side of a camera; A second determination module is used to determine characteristic information of the camera; wherein the characteristic information includes at least one of the following: specification information of a sensor included in the camera, imaging characteristic information of a lens included in the camera; A third determining module, used to determine a first image including a light source area; a generating module, configured to generate a second image representing a diffraction effect of the light source corresponding to the light source area based on the reference structure parameters determined by the first determining module, the characteristic information determined by the second determining module, and the first image determined by the third determining module; an evaluation module, configured to evaluate the reference structure parameters determined by the first determination module based on the second image generated by the generation module to obtain an evaluation result; The fourth determination module is used to determine the target structural parameters of the heating element based on the reference structural parameters determined by the first determination module and the evaluation result obtained by the evaluation module, so that the camera can take pictures when the heating element meets the target structural parameters.

11. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image capturing method according to any one of claims 1 to 9.

12. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the image capturing method described in any one of claims 1 to 9.

Images