Image data mapping and neural network inference methods, devices and electronic equipment

Abstract

The application relates to the technical field of image processing, and particularly provides an image data mapping and neural network inference method and device and electronic equipment. A Fourier transform result corresponding to an initial image is determined; the initial image is high dynamic range image data; image phase data corresponding to the initial image is determined according to the Fourier transform result; and the initial image is mapped to a target dynamic range according to the image phase data to obtain a target mapping image. Based on the image phase data, the phase information of the initial image can be maintained in the image data mapping process, and the detail information of the initial image is better preserved to obtain a target mapping image suitable for a machine vision perception scene.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to image data mapping and neural network inference methods, apparatuses, and electronic devices. Background Technology

[0002] High-Dynamic Range (HDR) images contain far more detail than ordinary images and have wide applications in fields such as autonomous driving and medical imaging. However, most neural networks are trained on Low-Dynamic Range (LDR) input images, and their performance drops significantly when faced with HDR input images.

[0003] To adapt current neural network engines to HDR images, image data mapping is typically used to map the acquired HDR type input image, and then the mapped LDR image is used as the input image for the corresponding neural network to ensure the image processing performance of the corresponding neural network.

[0004] Existing data mapping methods for HDR images can be broadly categorized into two types: methods that equalize overall brightness by decomposing the image's luminance and reflectance components to achieve HDR image mapping; and methods that utilize luminance variation curves designed based on human visual mechanisms to achieve HDR image mapping. Both methods focus on human visual perception, adjusting the image's brightness distribution to achieve HDR image mapping. Therefore, these mapping methods exhibit some blurring when processing certain scene colors and are not suitable for machine vision perception methods such as neural networks. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide an image data mapping and neural network inference method, apparatus and electronic device to solve the technical problem that the existing HDR image mapping method is not applicable to machine vision perception scenarios.

[0006] In a first aspect, embodiments of this application provide an image data mapping method, which includes:

[0007] Determine the Fourier transform result corresponding to the initial image; wherein, the initial image is high dynamic range image data;

[0008] Based on the Fourier transform result, determine the image phase data corresponding to the initial image;

[0009] Based on the image phase data, the initial image is mapped to the target dynamic range to obtain the target mapped image.

[0010] In the above implementation process, this image data mapping method determines the Fourier transform result corresponding to the initial image with high dynamic range; determines the image phase data corresponding to the initial image based on the Fourier transform result; and maps the initial image to the target dynamic range based on the image phase data to obtain the target mapped image. Based on the image phase data, the phase information of the initial image can be preserved during the image data mapping process, thereby better retaining the detail information of the initial image to obtain a target mapped image suitable for machine vision perception scenarios. This solves the technical problem that "existing HDR image mapping methods are not suitable for machine vision perception scenarios."

[0011] Optionally, in this embodiment, the Fourier transform result includes: real part data and imaginary part data corresponding to the initial image; determining the Fourier transform result corresponding to the initial image includes: determining the Fourier transform result corresponding to the initial image based on a preset optical mapping model; wherein the preset optical mapping model includes: the correspondence between the pixel value of the target pixel in the initial image and the local brightness parameter corresponding to the target pixel and the imaginary part data, and the correspondence between the pixel value of the target pixel and the global brightness parameter corresponding to the target pixel and the real part data.

[0012] In the above implementation process, since the preset optical mapping model includes the correspondence between the pixel value of the target pixel in the initial image and the local brightness parameter and imaginary data corresponding to the target pixel, as well as the correspondence between the pixel value of the target pixel and the global brightness parameter and real data corresponding to the target pixel, the image phase data corresponding to the initial image can be quickly determined based on the correspondence.

[0013] Optionally, in this embodiment of the application, determining the Fourier transform result corresponding to the initial image based on a preset optical mapping model includes:

[0014] Based on the pixel value I of the k-th pixel in the initial image k (h, w), the local brightness parameter α corresponding to the k-th pixel. k Determine the imaginary data i′ corresponding to the k-th pixel. k (h, w) = I k (h, w) + α k According to the pixel value I of the k-th pixel in the initial image k (h, w), the global brightness parameter β of the initial image, and the real part data I′ corresponding to the k-th pixel are determined. k (h, w) = Ik (h, w) + β; The step of determining the image phase data corresponding to the initial image based on the Fourier transform result includes: determining the image phase data corresponding to the k-th pixel based on the imaginary part data and the real part data corresponding to the k-th pixel.

[0015] In the above implementation process, according to i′ k (h, w) = I k (h, w) + α k It can quickly determine the pixel value I based on the k-th pixel in the initial image. k (h, w), the local brightness parameter α corresponding to the k-th pixel. k ", determine the imaginary part data corresponding to the k-th pixel. According to I′ k (h, w) = I k (h, w) + β can be used to quickly determine the pixel value I of the k-th pixel in the initial image. k (h, w) and the global brightness parameter β” of the initial image are used to determine the real part data corresponding to the k-th pixel. Then, based on the real part data and imaginary part data corresponding to the initial image, the image phase data corresponding to the initial image can be quickly determined.

[0016] Furthermore, the functional expression for the aforementioned image phase data is: It satisfies differentiability. It can be combined with a "neural network model for image processing" and its internal parameters can be updated using the backpropagation algorithm; it can also be used to update the local brightness parameter α corresponding to the k-th pixel based on the backpropagation algorithm. k Parameters such as the global brightness parameter β of the initial image are used to determine the image phase data and are optimized to obtain better image processing results.

[0017] Secondly, embodiments of this application provide a neural network inference method, which includes:

[0018] Label the target of interest in the initial image to obtain image verification data; wherein, the initial image is high dynamic range image data of the target neural network model to be input;

[0019] The initial image is processed by an image mapping method to obtain a target-mapped image; wherein the target mapping method includes any of the image data mapping methods described in the first aspect above.

[0020] The target mapped image is input into the target neural network model to obtain the image processing result output by the target neural network model; wherein, the target neural network model can realize at least one image processing method among image classification, object detection, semantic segmentation and instance segmentation;

[0021] Based on the image verification data and the image processing results, determine the current loss function value;

[0022] The internal parameters of the target neural network model are optimized based on the current loss function value to obtain an optimized neural network model.

[0023] In the above implementation process, the image data mapping method achieves image data mapping based on image phase data. This allows for the preservation of the initial image's phase information during the mapping process, thus better retaining the initial image's detailed information. Applying this image data mapping method to neural network inference, a target mapped image suitable for machine vision perception scenarios is obtained. Using this target mapped image as input data for the target neural network model allows for better optimization of the target neural network model's internal parameters, resulting in an "optimized neural network model" that better suits the machine vision perception scenario.

[0024] Optionally, in this embodiment of the application, optimizing the intrinsic parameters of the target neural network model based on the current loss function value to obtain an optimized neural network model includes: obtaining the intrinsic parameter gradient of each intrinsic parameter relative to the loss function value; and optimizing the intrinsic parameters according to the intrinsic parameter gradient and the current loss function value to obtain the optimized neural network model.

[0025] In the above implementation process, since the neural network model satisfies differentiability, the backpropagation algorithm can be used to calculate the gradient of the internal parameters. Based on the gradient of the internal parameters, the internal parameters of the target neural network model are updated to obtain better image processing results.

[0026] Optionally, in this embodiment of the application, the imaginary part data i′ corresponding to the k-th pixel in the initial image is used. k (h, w), real part data I′ k (h, w), determine the image phase data corresponding to the k-th pixel. In the case of [missing information], the method for determining the local brightness parameter and the global brightness parameter in the target mapping method includes: obtaining the local brightness parameter gradient relative to the loss function value and the global brightness parameter gradient relative to the loss function value; optimizing the current local brightness parameter and the current global brightness parameter based on the local brightness parameter gradient, the global brightness parameter gradient, and the current loss function value to obtain optimized local brightness parameters and optimized global brightness parameters.

[0027] In the above implementation process, due to image phase data Since differentiability is satisfied, the gradients of the local and global brightness parameters can be calculated using the backpropagation algorithm. Based on these gradients, the local brightness parameter α corresponding to the k-th pixel can be calculated. k The parameters used to determine the image phase data, such as the global brightness parameter β of the initial image, are optimized. Based on the optimized local brightness parameters and the optimized global brightness parameters, better image processing results can be obtained.

[0028] Optionally, in embodiments of this application, wherein, based on Determine the initial values ​​of the local brightness parameter and the global brightness parameter; α k0 β0 represents the initial value of the local brightness parameter corresponding to the k-th pixel in the initial image, ε represents the initial value of the global brightness parameter, N represents the number of pixels in the initial image.

[0029] In the above implementation process, through Determining the initial values ​​of the local brightness parameter and the global brightness parameter corresponding to the k-th pixel can simplify the subsequent optimization process for the local and global brightness parameter model parameters.

[0030] Thirdly, embodiments of this application provide an image data mapping apparatus, which includes:

[0031] The parameter determination module is used to determine the Fourier transform result corresponding to the initial image; wherein, the initial image is high dynamic range image data;

[0032] The phase data determination module is used to determine the image phase data corresponding to the initial image based on the Fourier transform result;

[0033] The image mapping acquisition module is used to map the initial image to the target dynamic range based on the image phase data to obtain the target mapped image.

[0034] Optionally, in this embodiment, the Fourier transform result includes: real part data and imaginary part data corresponding to the initial image; the parameter determination module is specifically used to: determine the Fourier transform result corresponding to the initial image based on a preset optical mapping model; wherein the preset optical mapping model includes: the correspondence between the pixel value of the target pixel in the initial image and the local brightness parameter corresponding to the target pixel and the imaginary part data, and the correspondence between the pixel value of the target pixel and the global brightness parameter corresponding to the target pixel and the real part data.

[0035] Optionally, in this embodiment of the application, the parameter determination module may further be used to: determine the pixel value I of the k-th pixel in the initial image. k (h, w), the local brightness parameter α corresponding to the k-th pixel. k Determine the imaginary data i′ corresponding to the k-th pixel. k (h, w) = I k (h, w) + α k According to the pixel value I of the k-th pixel in the initial image k (h, w), the global brightness parameter β of the initial image, and the real part data I′ corresponding to the k-th pixel are determined. k (h, w) = I k (h, w) + β; The step of determining the image phase data corresponding to the initial image based on the Fourier transform result includes: determining the image phase data corresponding to the k-th pixel based on the imaginary part data and the real part data corresponding to the k-th pixel.

[0036] Fourthly, embodiments of this application provide a neural network inference device, which includes:

[0037] An image annotation module is used to annotate targets of interest in an initial image to obtain image verification data; wherein, the initial image is high dynamic range image data of the target neural network model to be input;

[0038] A mapping processing module is used to perform image mapping processing on the initial image based on a target mapping method to obtain a target mapped image; wherein, the target mapping method includes the image data mapping method described in the first aspect above;

[0039] The processing result acquisition module is used to input the target mapping image into the target neural network model and acquire the image processing result output by the target neural network model; wherein, the target neural network model can realize at least one image processing method among image classification, object detection, semantic segmentation and instance segmentation;

[0040] The loss value determination module is used to determine the current loss function value based on the image verification data and the image processing result;

[0041] The parameter optimization module is used to optimize the internal parameters of the target neural network model based on the current loss function value, so as to obtain an optimized neural network model.

[0042] Optionally, in this embodiment, the parameter optimization module described above can be specifically used to: obtain the gradient of each internal parameter relative to the loss function value; optimize the internal parameters according to the internal parameter gradient and the current loss function value to obtain the optimized neural network model.

[0043] Optionally, in this embodiment of the application, the imaginary part data i′ corresponding to the k-th pixel in the initial image is used. k (h, w), real part data I′ k (h, w), determine the image phase data corresponding to the k-th pixel. In this case, the neural network inference device may further include: a local gradient acquisition module, used to acquire the local brightness parameter gradient of the local brightness parameter relative to the loss function value, and the global brightness parameter gradient of the global brightness parameter relative to the loss function value; and a global gradient acquisition module, used to optimize the current local brightness parameter and the current global brightness parameter based on the local brightness parameter gradient, the global brightness parameter gradient, and the current loss function value, so as to obtain optimized local brightness parameters and optimized global brightness parameters.

[0044] Optionally, in this embodiment of the application, the neural network inference device may further include: an initial value determination module, used for determining initial values ​​based on... Determine the initial values ​​of the local brightness parameter and the global brightness parameter; α k0 β0 represents the initial value of the local brightness parameter corresponding to the k-th pixel in the initial image, ε represents the initial value of the global brightness parameter, N represents the stability parameter, and N represents the number of pixels in the initial image.

[0045] Fifthly, embodiments of this application also provide an electronic device, including: a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the computer program is executed by the processor, it performs the image data mapping method as described in the first aspect above or the neural network inference method as described in the second aspect above.

[0046] Sixthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions that, when executed by a processor, perform the image data mapping method as described in the first aspect above or the neural network inference method as described in the second aspect above.

[0047] The beneficial effects of this application are as follows: By determining the Fourier transform result corresponding to the initial image with a high dynamic range; based on the Fourier transform result, determining the image phase data corresponding to the initial image; and based on the image phase data, mapping the initial image to the target dynamic range to obtain a target mapped image. Based on the image phase data, the phase information of the initial image can be maintained during the image data mapping process, thereby better preserving the detail information of the initial image to obtain a target mapped image suitable for machine vision perception scenarios. This solves the technical problem that "existing HDR image mapping methods are not suitable for machine vision perception scenarios." Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an image data mapping method provided in an embodiment of this application;

[0050] Figure 2 A flowchart illustrating a neural network inference method provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of the structure of an image data mapping device provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a neural network inference device provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application.

[0056] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0057] Please see Figure 1 The illustration shows a flowchart of an image data mapping method provided in an embodiment of this application. This image data mapping method may include the following steps:

[0058] Step 101: Determine the Fourier transform result corresponding to the initial image; wherein, the initial image is high dynamic range image data;

[0059] Step 102: Determine the image phase data corresponding to the initial image based on the Fourier transform result;

[0060] Step 103: Based on the image phase data, map the initial image to the target dynamic range to obtain the target mapped image.

[0061] In step 101, the Fourier transform result refers to the Fourier transform data including the initial image. Specifically, it can include the real and imaginary parts of the image data after performing a Fourier transform on the initial image, or it can include "the real and imaginary parts determined based on the pixel value of the target pixel in the initial image, the local brightness parameter corresponding to the target pixel, and the global brightness parameter of the initial image." The initial image can be image data acquired based on an HDR camera (or in the camera's HDR mode). Specifically, when the Fourier transform result includes "the real and imaginary parts of the image data after performing a Fourier transform on the initial image," the Fourier transform result corresponding to the initial image can be determined by performing a Fourier transform on the initial image. When the Fourier transform result includes "the real and imaginary parts determined based on the pixel value of the target pixel in the initial image, the local brightness parameter corresponding to the target pixel, and the global brightness parameter of the initial image," the real and imaginary parts can be determined based on the pixel value I of the k-th pixel in the initial image. k (h, w), the local brightness parameter α corresponding to the k-th pixel. k Determine the imaginary part data i′ corresponding to the k-th pixel. k (h, w) = I k (h, w) + α k ; and, based on the pixel value I of the k-th pixel in the initial image k (h, w), the global brightness parameter β of the initial image, and the real part data I′ corresponding to the k-th pixel are determined. k (h, w) = I k (h, w) + β.

[0062] In step 102, if the Fourier transform result includes "the real part and imaginary part of the image data after Fourier transform of the initial image", the image phase data corresponding to the initial image can be determined based on the ratio of the imaginary part to the real part. If the Fourier transform result includes "the real part and imaginary part determined based on the pixel value of the target pixel in the initial image, the local brightness parameter corresponding to the target pixel, and the global brightness parameter of the initial image", the image phase data corresponding to the k-th pixel can be determined based on the imaginary and real part data corresponding to the k-th pixel.

[0063] In step 103, the target dynamic range can be determined based on actual application requirements; specifically, it can be a low dynamic range or other reasonable range. Taking a target dynamic range of [0, 255] as an example, it can be determined based on I″... k (h, w) = quantize(P) k(h, w) * 255), mapping the image phase data to the target dynamic range, I″ k (h, w) represents the pixel value corresponding to the k-th pixel in the target mapped image, and quantize() represents the rounding operation symbol. The specific rounding rules can be adjusted according to the actual application.

[0064] Therefore, the image data mapping method provided in this application determines the Fourier transform result corresponding to the initial image with a high dynamic range; determines the image phase data corresponding to the initial image based on the Fourier transform result; and maps the initial image to the target dynamic range based on the image phase data to obtain a target mapped image. Based on the image phase data, the phase information of the initial image can be maintained during the image data mapping process, thereby better preserving the detail information of the initial image to obtain a target mapped image suitable for machine vision perception scenarios. This solves the technical problem that "existing HDR image mapping methods are not suitable for machine vision perception scenarios."

[0065] The image data mapping method provided in this application achieves adaptive equalization of the image brightness domain by inverting the image phase. Compared with existing HDR image mapping methods, this application restores and enhances the brightness distribution and details of the initial image during the image data mapping process.

[0066] In some optional embodiments, the Fourier transform result includes: real part data and imaginary part data corresponding to the initial image; step 101, determining the Fourier transform result corresponding to the initial image, includes: determining the Fourier transform result corresponding to the initial image based on a preset optical mapping model; wherein, the preset optical mapping model includes: the correspondence between the pixel value of the target pixel in the initial image and the local brightness parameter corresponding to the target pixel and the imaginary part data, and the correspondence between the pixel value of the target pixel and the global brightness parameter corresponding to the target pixel and the real part data.

[0067] The target pixel can be any pixel in the initial image. Specifically, the imaginary data corresponding to the target pixel can be determined based on its pixel value, the corresponding local brightness parameter, and the correspondence between the pixel value, the corresponding local brightness parameter, and the imaginary data. Similarly, the real data corresponding to the target pixel can be determined based on its pixel value, the global brightness parameter, and the correspondence between the pixel value, the corresponding global brightness parameter, and the real data. For example, if the target pixel is the k-th pixel in the initial image, the imaginary data corresponding to the k-th pixel can be determined based on its pixel value and the corresponding local brightness parameter; the real data corresponding to the k-th pixel can be determined based on its pixel value and the global brightness parameter.

[0068] In some optional embodiments, determining the Fourier transform result corresponding to the initial image based on a preset optical mapping model includes: determining the pixel value I of the k-th pixel in the initial image. k (h, w), the local brightness parameter α corresponding to the k-th pixel. k Determine the imaginary data i′ corresponding to the k-th pixel. k (h, w) = I k (h, w) + α k According to the pixel value I of the k-th pixel in the initial image k (h, w), the global brightness parameter β of the initial image, and the real part data I′ corresponding to the k-th pixel are determined. k (h, w) = I k (h, w) + β; The step of determining the image phase data corresponding to the initial image based on the Fourier transform result includes: determining the image phase data corresponding to the k-th pixel based on the imaginary part data and the real part data corresponding to the k-th pixel.

[0069]

[0070] Among them, the local brightness parameter α k The value of the global brightness parameter β can be determined based on α. k = Determined. Based on i′ k (h, w) = I k (h, w) + α k It can quickly determine the pixel value I based on the k-th pixel in the initial image. k(h, w), the local brightness parameter α corresponding to the k-th pixel. k ", determine the imaginary part data corresponding to the k-th pixel. According to I′ k (h, w) = I k (h, w) + β can be used to quickly determine the pixel value I of the k-th pixel in the initial image. k (h, w) and the global brightness parameter β” of the initial image are used to determine the real part data corresponding to the k-th pixel. Then, based on the real part data and imaginary part data corresponding to the initial image, the image phase data corresponding to the initial image is quickly determined.

[0071] Furthermore, the functional expression for the aforementioned image phase data is: It satisfies differentiability. It can be combined with a "neural network model for image processing" and its internal parameters can be updated using the backpropagation algorithm; it can also be used to update the local brightness parameter α corresponding to the k-th pixel based on the backpropagation algorithm. k Parameters used to determine image phase data, such as the global brightness parameter β of the initial image, are optimized to achieve better image processing results. When the above image data mapping method is combined with a neural network model for image processing, it is possible to... The initial values ​​of the local and global brightness parameters are determined, and then the current local and global brightness parameters are optimized using the gradients of the local and global brightness parameters and the current loss function value to obtain the optimized local and global brightness parameters.

[0072] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a neural network inference method provided in an embodiment of this application. The neural network inference method may include the following steps:

[0073] Step 201: Label the target of interest in the initial image to obtain image verification data; wherein, the initial image is high dynamic range image data of the target neural network model to be input;

[0074] Step 202: Perform image mapping processing on the initial image based on the target mapping method to obtain a target mapped image; wherein, the target mapping method includes any of the image data mapping methods described in the first aspect above;

[0075] Step 203: Input the target mapping image into the target neural network model and obtain the image processing result output by the target neural network model; wherein, the target neural network model can realize at least one image processing method among image classification, target detection, semantic segmentation and instance segmentation;

[0076] Step 204: Determine the current loss function value based on the image verification data and the image processing results;

[0077] Step 205: Optimize the internal parameters of the target neural network model based on the current loss function value to obtain an optimized neural network model.

[0078] The labeled target of interest corresponds to the image processing method that the target neural network model can achieve. For example, if the target neural network model can perform image classification, the target of interest can be the image classification result of the initial image; if the target neural network model can simultaneously perform image classification and object detection, the target of interest can be the image classification result of the initial image and the object detection result in the initial image; if the target neural network model can perform semantic segmentation, the target of interest can be the semantic segmentation result of the initial image; if the target neural network model can perform instance segmentation, the target of interest can be the instance segmentation result of the initial image. The image processing methods that the target neural network model can achieve can be adjusted according to actual application requirements; specifically, the image processing methods that the target neural network model can achieve can include only one of image classification, object detection, semantic segmentation, or instance segmentation, or can include any two or more of these methods simultaneously.

[0079] The target neural network model can include convolutional layers and pooling layers, the number of which can be adjusted according to actual application requirements. Convolutional layers extract image features from the input initial image through convolution operations to obtain a feature image, while pooling layers refine the extracted image features. The specific implementation of step 202 can refer to the specific implementation of steps 101-103 above.

[0080] The loss function value can be determined based on the image processing capabilities of the target neural network model. For example, if the target neural network model can perform image classification, the loss function value can be determined based on the accuracy index. If the target neural network model can perform semantic segmentation, the loss function value can be determined based on the Mean Intersection over Union (MIoU) index.

[0081] It should be noted that, based on a preset optical mapping model, when the Fourier transform result corresponding to the initial image is determined: Since the preset optical mapping model includes the correspondence between the pixel values ​​of the target pixels in the initial image and the local brightness parameters and imaginary data corresponding to the target pixels, as well as the correspondence between the pixel values ​​of the target pixels and the global brightness parameters and real data corresponding to the target pixels, the image phase data corresponding to the initial image can be quickly determined based on these correspondences. Based on the image phase data, the phase information of the initial image can be preserved during the image data mapping process, thereby better retaining the detail information of the initial image to obtain a target mapped image suitable for machine vision perception scenarios. Therefore, based on the preset optical mapping model, a target mapped image suitable for machine vision perception scenarios corresponding to the initial image can be quickly and accurately determined.

[0082] Furthermore, since the target mapping image is adaptable to machine vision perception scenarios, the target mapping image obtained based on a preset optical mapping model can be directly applied to the neural network inference process, thereby avoiding the need for additional computing resources to be consumed by the target mapping image obtained based on image mapping processing, thus improving computational efficiency. In some optional embodiments, step 205, optimizing the internal parameters of the target neural network model based on the current loss function value to obtain an optimized neural network model, includes: obtaining the internal parameter gradient of each internal parameter relative to the loss function value; optimizing the internal parameters according to the internal parameter gradient and the current loss function value to obtain the optimized neural network model.

[0083] This can be achieved by using backpropagation to calculate the gradient of each intrinsic parameter relative to the loss function value (i.e., the rate of change of each intrinsic parameter relative to the output of the target neural network model), and then optimizing the intrinsic parameters based on these gradients to obtain a loss function value that satisfies the preset loss function conditions. Optimization of the intrinsic parameters can be based on algorithms such as gradient descent.

[0084] Among them, the local brightness parameter α k The value of the global brightness parameter β can be determined according to... Determined; or based on The initial values ​​of the local and global brightness parameters are determined, and then the current local and global brightness parameters are optimized using the gradients of the local and global brightness parameters and the current loss function value to obtain the optimized local and global brightness parameters.

[0085] Specifically, the target neural network model can be a road image recognition and classification model for autonomous driving, or a target region recognition model for CT images for medical imaging. Correspondingly, in the case where the target neural network model is a road image recognition and classification model for autonomous driving, the initial image can be image data of a road scene acquired by an HDR camera; in the case where the target neural network model is a target region recognition model for CT images for medical imaging, the initial image can be image data of a CT imaging scene acquired by an image acquisition device in HDR mode. By acquiring HDR images of the target scene (the road scene or CT imaging scene as described above), an initial image of the corresponding scene is obtained. Then, the target neural network model is optimized based on the initial image to obtain an "optimized neural network model" that better suits the corresponding scene.

[0086] In some alternative embodiments, based on the imaginary part data i′ corresponding to the k-th pixel in the initial image... k (h, w), real part data I ' k (h, w), determine the image phase data corresponding to the k-th pixel. In the case of [missing information], the method for determining the local brightness parameter and the global brightness parameter in the target mapping method includes: obtaining the local brightness parameter gradient relative to the loss function value and the global brightness parameter gradient relative to the loss function value; optimizing the current local brightness parameter and the current global brightness parameter based on the local brightness parameter gradient, the global brightness parameter gradient, and the current loss function value to obtain optimized local brightness parameters and optimized global brightness parameters.

[0087] Wherein, due to the functional expression of image phase data: Both the optical mapping model and the target neural network model satisfy differentiability, enabling end-to-end joint training between them to improve the accuracy of the trained model. Specifically, the backpropagation algorithm can be used to calculate the gradient of the local or global brightness parameters relative to the loss function value (i.e., the rate of change of the local brightness parameters relative to the output of the target neural network model) or the gradient of the global brightness parameters (i.e., the rate of change of the global brightness parameters relative to the output of the target neural network model). Based on these gradients, the current local or global brightness parameters are optimized to obtain optimized local or global brightness parameters that satisfy the preset loss function conditions. Specifically, optimization algorithms such as gradient descent can be used to optimize the current local and global brightness parameters.

[0088] In some alternative embodiments, based on Determine the initial values ​​of the local brightness parameter and the global brightness parameter; α k0 β0 represents the initial value of the local brightness parameter corresponding to the k-th pixel in the initial image, ε represents the initial value of the global brightness parameter, N represents the stability parameter, and N represents the number of pixels in the initial image.

[0089] Among them, through Determining the initial values ​​of the local brightness parameter and the global brightness parameter corresponding to the k-th pixel can simplify the subsequent optimization process for the local and global brightness parameter model parameters.

[0090] Please refer to Figure 3 , Figure 3 This is a schematic diagram of an image data mapping device provided in an embodiment of this application. The image data mapping device includes:

[0091] The parameter determination module 301 is used to determine the Fourier transform result corresponding to the initial image; wherein, the initial image is high dynamic range image data;

[0092] The phase data determination module 302 is used to determine the image phase data corresponding to the initial image based on the Fourier transform result;

[0093] The image mapping acquisition module 303 is used to map the initial image to the target dynamic range based on the image phase data to obtain the target mapped image.

[0094] Please refer to Figure 4 , Figure 4This is a schematic diagram of a neural network inference device provided in an embodiment of this application. The neural network inference device includes:

[0095] The image annotation module 401 is used to annotate the target of interest in the initial image to obtain image verification data; wherein, the initial image is high dynamic range image data of the target neural network model to be input;

[0096] The mapping processing module 402 is used to perform image mapping processing on the initial image based on the target mapping method to obtain a target mapped image; wherein, the target mapping method includes any of the image data mapping methods described in the first aspect above;

[0097] The processing result acquisition module 403 is used to input the target mapping image into the target neural network model and acquire the image processing result output by the target neural network model; wherein, the target neural network model can realize at least one image processing method among image classification, target detection, semantic segmentation and instance segmentation;

[0098] The loss value determination module 404 is used to determine the current loss function value based on the image verification data and the image processing result;

[0099] The parameter optimization module 405 is used to optimize the internal parameters of the target neural network model based on the current loss function value, so as to obtain an optimized neural network model.

[0100] It should be understood that this image data mapping / neural network inference device corresponds to the image data mapping / neural network inference method embodiment described above, and is capable of performing the various steps involved in the above method embodiment. The specific functions of this image data mapping / neural network inference device can be found in the description above; to avoid repetition, detailed descriptions are appropriately omitted here. This image data mapping / neural network inference device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0101] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 500 provided in this application includes a processor 501 and a memory 502. These components are interconnected and communicate with each other via a communication bus 503 and / or other forms of connection mechanisms (not shown). The memory 502 stores a computer program executable by the processor 501. When executed by the processor 501, the computer program performs the image data mapping method described in the first aspect above or the neural network inference method described in the second aspect.

[0102] This application also provides a computer-readable storage medium storing computer program instructions, which, when executed by processor 501, perform the image data mapping method described in the first aspect or the neural network inference method described in the second aspect.

[0103] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0104] It should be understood that the disclosed apparatus / systems and methods can also be implemented in other ways, as provided in the embodiments of this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0105] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0106] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. An image data mapping method, characterized by, The method comprises: determining a Fourier transform result corresponding to an initial image; wherein the initial image is high dynamic range image data; determining image phase data corresponding to the initial image according to the Fourier transform result; mapping the initial image to a target dynamic range according to the image phase data to obtain a target mapping image; wherein the Fourier transform result comprises real part data and imaginary part data corresponding to the initial image; the determination of the Fourier transform result corresponding to the initial image comprises determining the Fourier transform result corresponding to the initial image based on a preset optical mapping model; wherein the preset optical mapping model comprises a correspondence between a pixel value of a target pixel point in the initial image and a local brightness parameter corresponding to the target pixel point and the imaginary part data, and a correspondence between the pixel value of the target pixel point and a global brightness parameter corresponding to the target pixel point and the real part data; the determination of the Fourier transform result corresponding to the initial image based on the preset optical mapping model comprises: According to the pixel value of the kth pixel point in the initial image , the local brightness parameter corresponding to the kth pixel point , determine the imaginary part data corresponding to the kth pixel point ; According to the pixel value of the kth pixel point in the initial image , the global brightness parameter of the initial image , determine the real part data corresponding to the kth pixel point ; The method further comprises: determining image phase data corresponding to the initial image according to the Fourier transform result, wherein the determining the image phase data corresponding to the initial image according to the Fourier transform result comprises: determining the image phase data corresponding to the kth pixel point according to the imaginary part data and the real part data corresponding to the kth pixel point .

2. A neural network inference method, comprising: The method comprises: annotating a target of interest in an initial image to obtain image verification data; wherein the initial image is high dynamic range image data to be input into a target neural network model; performing image mapping processing on the initial image based on a target mapping method to obtain a target mapping image; wherein the target mapping method comprises the image data mapping method of the above claim 1; inputting the target mapping image into the target neural network model to obtain an image processing result output by the target neural network model; wherein the target neural network model can implement at least one of image processing modes of image classification, target detection, semantic segmentation and instance segmentation; determining a current loss function value according to the image verification data and the image processing result; optimizing internal parameters of the target neural network model based on the current loss function value to obtain an optimized neural network model.

3. The method of claim 2, wherein, the optimization of the internal parameters of the target neural network model based on the current loss function value to obtain the optimized neural network model comprises: obtaining an internal parameter gradient of each internal parameter with respect to a loss function value; optimizing the internal parameters according to the internal parameter gradient and the current loss function value to obtain the optimized neural network model.

4. The method of claim 3, wherein, wherein In the case of determining the image phase data corresponding to the kth pixel point in the initial image according to the imaginary part data , the real part data corresponding to the kth pixel point , the determination method of the local brightness parameter and the global brightness parameter in the target mapping method comprises: obtaining a local brightness parameter gradient of the local brightness parameter with respect to the loss function value and a global brightness parameter gradient of the global brightness parameter with respect to the loss function value; optimizing the current local brightness parameter and the current global brightness parameter according to the local brightness parameter gradient, the global brightness parameter gradient and the current loss function value to obtain an optimized local brightness parameter and an optimized global brightness parameter.

5. The method of claim 4, wherein, wherein based on , , determine the initial value of the local brightness parameter and the global brightness parameter; represents the initial value of the local brightness parameter corresponding to the kth pixel point in the initial image, represents the initial value of the global brightness parameter, represents a stability parameter, represents the number of pixel points in the initial image.

6. An image data mapping apparatus characterized by comprising: the device comprises: The parameter determination module is configured to determine a Fourier transform result corresponding to the initial image; the initial image is high dynamic range image data; and the Fourier transform result includes real part data and imaginary part data corresponding to the initial image. The phase data determination module is configured to determine image phase data corresponding to the initial image based on the Fourier transform result. The mapping image acquisition module is configured to map the initial image to a target dynamic range based on the image phase data to obtain a target mapping image. The parameter determination module is specifically configured to: determine the Fourier transform result corresponding to the initial image based on a preset optical mapping model. The preset optical mapping model includes a correspondence between a pixel value of a target pixel point in the initial image, a local brightness parameter corresponding to the target pixel point, and the imaginary part data, and a correspondence between the pixel value of the target pixel point, a global brightness parameter corresponding to the target pixel point, and the real part data. The determining, based on a preset optical mapping model, of the Fourier transform result corresponding to the initial image comprises: determining, according to a pixel value of a kth pixel point in the initial image , a local brightness parameter corresponding to the kth pixel point , and determining the imaginary part data corresponding to the kth pixel point . According to the pixel value of the kth pixel point in the initial image , the global brightness parameter of the initial image , determining the real part data corresponding to the kth pixel point ; The phase data determination module is specifically configured to determine the image phase data corresponding to the kth pixel point according to the imaginary part data and the real part data corresponding to the kth pixel point. .

7. A neural network inference apparatus, comprising: The device includes: An image labeling module is configured to label a target of interest in an initial image to obtain image verification data; the initial image is high dynamic range image data to be input into a target neural network model. A mapping processing module is configured to perform image mapping processing on the initial image based on a target mapping method to obtain a target mapping image; the target mapping method includes the image data mapping method of claim 1. A processing result acquisition module is configured to input the target mapping image into the target neural network model to obtain an image processing result output by the target neural network model; the target neural network model can implement at least one of image processing modes including image classification, target detection, semantic segmentation, and instance segmentation. A loss value determination module is configured to determine a current loss function value based on the image verification data and the image processing result. A parameter optimization module is configured to optimize internal parameters of the target neural network model based on the current loss function value to obtain an optimized neural network model.

8. An electronic device, comprising: The electronic device includes: a memory; a processor; The memory has stored thereon a computer program executable by the processor, and the computer program, when executed by the processor, performs the method of any one of claims 1-5.

Images