A camera raw image data processing matching method and system

Abstract

The application discloses a camera raw image data processing matching method and system, and relates to the field of intelligent parking management. The method comprises the following steps: adapting a visual model pre-trained based on an RGB image to camera raw image data, adjusting raw input data by using a learnable ISP process through constructing an input data adapter, and obtaining input data more suitable for the input of a visual algorithm; in addition, a model-level intermediate stage feature extraction adapter is constructed between an ISP stage and a subsequent visual algorithm network, intermediate stage features from the input-level adapter are extracted, the features are embedded into the subsequent visual algorithm network, the network contains prior knowledge from the ISP stage, and the capability of the visual algorithm is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent parking management, and in particular to a camera raw image data processing and matching method and system. BACKGROUND

[0002] In recent years, with the development of social economy, the demand for convenience in daily travel has been increasing, and the number of cars has been increasing, leading to parking difficulties and chaos, which has become one of the main problems to be solved in the development of urban traffic in China. In recent years, with the development of intelligent technology, in order to effectively alleviate this problem, traffic management departments in various places have tried and constructed various intelligent parking management technologies. Among them, setting up parking spaces on the roadside, using intelligent sensing devices to collect video image data, and processing and analyzing video image data through computer vision algorithms to realize intelligent parking management is a mainstream solution at present.

[0003] However, in the night, rain, fog and other special scenes, the clarity of the RGB image captured by the camera is low, and the image noise is large, so that the visual algorithm is difficult to achieve accurate recognition results on such images, and in many cases, the human eye is also difficult to distinguish the content in the image, which further affects the accurate scene analysis of high-bit video scene images; for such low-light images, currently, mainly based on the RGB image generated by the camera, through the use of generative adversarial network or diffusion model algorithm, but in many cases, the light condition is very bad, and it is difficult to realize the recovery of low-light images, and some noise information is easily introduced, affecting the application of visual algorithm; however, the raw data collected by the camera has more rich physical information under these various and challenging different light conditions; the raw image is usually 12-bit or 14-bit data, containing a large amount of original image information, in the camera imaging process, the image signal processing (ISP) will receive the original raw image, after a series of image signal processing, the usually more human eye visual perception friendly RGB image is obtained, which is usually an 8-bit image.

[0004] At present, for computer vision tasks based directly on raw data, the ISP process is usually combined with the network of the rear-end vision task, but the interaction ability between the ISP process and the subsequent visual algorithm network is often ignored, resulting in poor matching degree of the existing camera raw image data and the visual model. SUMMARY

[0005] To solve the above technical problems, the present application provides a camera raw image data processing and matching method and system, which can solve the problem of poor matching degree of the existing camera raw image data and the visual model.

[0006] To achieve the above object, the application provides a camera raw image data processing matching method, which comprises the following steps:

[0007] The input camera raw image data is filtered and corrected by an input data adapter to generate input data of a pre-trained visual model;

[0008] Intermediate stage feature information is extracted from intermediate data in the filtering and correction process by a preset intermediate stage feature extraction adapter;

[0009] The input data of the pre-trained visual model is taken as input of the pre-trained visual model to generate output of the pre-trained visual model;

[0010] The output of the pre-trained visual model is updated and embedded according to the intermediate stage feature information and output.

[0011] Further, the step of filtering and correcting the input camera raw image data by the input data adapter to generate the input data of the pre-trained visual model comprises:

[0012] The input camera raw image data is gain and noise reduced, white balanced, color corrected and tone mapped by the input data adapter to generate the input data of the pre-trained visual model.

[0013] Further, the step of filtering and correcting the input camera raw image data by the input data adapter to generate the input data of the pre-trained visual model comprises:

[0014] According to formula I2=(g*I1)*k{r1, r2, θ}, I2=I2+(g*I1-I2)*σ,

[0015] I4=I3*

[0016] E ccm , I5(R ’ , G ’ , B ’ )=MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, wherein the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the long axis r1, the short axis r2, the kernel angle θ, the filtering parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, the color correction matrix is E ccm , MLP represents a multi-layer perceptron, D i∈(r,g,b) represents the white balance parameter of each color space, i.e. RGB three colors, Wherein, I1 represents the input data of the pre-training visual model, I2 represents the image after the gain and noise reduction process, I3 represents the image after the white balance, and I4 represents the image after the color correction.

[0017] Further, the step of extracting the intermediate stage feature information from the intermediate data in the filtering correction process by the preset intermediate stage feature extraction adapter includes:

[0018] extracting the intermediate stage feature information from the intermediate data in the filtering correction process by the convolution model in the preset intermediate stage feature extraction adapter and performing feature splicing;

[0019] processing the spliced intermediate stage feature information by the residual convolution model in the preset intermediate stage feature extraction adapter to generate the adaptive feature.

[0020] Further, the step of updating and embedding the output of the pre-training visual model according to the intermediate stage feature information includes:

[0021] performing feature fusion on the adaptive feature and the output result of each stage in the pre-training visual model.

[0022] Further, the present application provides a camera raw image data processing matching system, which comprises:

[0023] a generating module configured to perform filtering correction processing on the input camera raw image data by an input data adapter to generate input data of a pre-training visual model;

[0024] an extracting module configured to extract intermediate stage feature information from the intermediate data in the filtering correction process by a preset intermediate stage feature extraction adapter;

[0025] the generating module is further configured to take the input data of the pre-training visual model as the input of the pre-training visual model to generate the output of the pre-training visual model;

[0026] an embedding output module configured to update and embed the output of the pre-training visual model according to the intermediate stage feature information and output.

[0027] Further, the generating module is specifically configured to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data by the input data adapter to generate the input data of the pre-training visual model.

[0028] Further, the generating module is specifically configured to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data by the input data adapter to generate the input data of the pre-training visual model.

[0029] I2 = I2 + (g*I1 - I2)*sigma,

[0030] I4 = I3*E ccm , I5(R ’ , G ’ , B ’ ) = MLP(I4(R, G, B)) to generate a pre-trained visual model

[0031] The input data of the pre-trained visual model, wherein the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the long axis is r1, the short axis is r2, the kernel angle is theta, the filtering parameter is sigma, the input data of the pre-trained visual model is I5, the white balance parameter is p, and the color correction matrix is E ccm MLP represents a multi-layer perception, and D i∈(r,g,b) represents the white balance parameter of each color space, that is, the white balance parameter of three colors of RGB, represents a white balance matrix, I2 represents an image after a gain and noise reduction process, I3 is an image after white balance, and I4 represents an image after color correction.

[0032] Further, the extraction module is specifically configured to extract intermediate stage feature information from intermediate data in the filtering and correction process by a convolution model in a pre-set intermediate stage feature extraction adapter and perform feature splicing, and process the intermediate stage feature information after the feature splicing by a residual convolution model in the pre-set intermediate stage feature extraction adapter to generate an adaptive feature.

[0033] Further, the embedding output module is specifically configured to perform feature fusion on the adaptive feature and output results of each stage in the pre-trained visual model.

[0034] The present application provides a camera raw image data processing matching method and system, which adapt a visual model pre-trained based on an RGB image to camera raw image data, adjust raw input data by using a learnable ISP process to obtain input data more suitable for visual algorithm input by constructing an input data adapter, and further construct a model-level intermediate stage feature extraction adapter between an ISP stage and a subsequent visual algorithm network, embed the features from the input-level adapter into the subsequent visual algorithm network by extracting intermediate stage features, let the network contain prior knowledge from the ISP stage, and improve the capability of the visual algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flowchart of a camera raw image data processing matching method provided by the present application;

[0036] Figure 2 is a schematic diagram of a camera raw image data processing matching system provided by the present application. DETAILED DESCRIPTION

[0037] The device structure and implementation of the present application are described in further detail below with reference to the accompanying drawings and examples.

[0038] The present application provides a camera raw image data processing matching method, as shown in the accompanying drawings, specifically comprising the following steps: Figure 1

[0039] 101. Filter and correct the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model.

[0040] Specifically, the step of filtering and correcting the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model comprises: gain and noise reduction, white balance processing, color correction processing, and tone mapping processing of the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model.

[0041] Further, the step of gain and noise reduction, white balance processing, color correction processing, and tone mapping processing of the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model comprises: generating input data for the pre-trained visual model according to the formula I2' = (g·I1)*k{r1, r2, θ}, I2 = I2' + (g·I1-I2')·σ, I4 = I3*E ccm , I5(R', G', B') = MLP(I4(R, G, B)) where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the long axis r1, the short axis r2, the kernel angle θ, the filter parameter is σ, the input data for the pre-trained visual model is I5, the white balance parameter is ρ, the color correction matrix is E ccm , MLP represents a multi-layer perceptron, D i∈(r,g,b) represents the white balance parameter of each color space, i.e. RGB three colors, represents the white balance matrix, I2 represents the image after gain and noise reduction, I3 is the image after white balance; I4 represents the image after color correction.

[0042] ​For example, the input raw data is defined as I1, and the designed input data adapter obtains the input data I5 facing the visual algorithm, which means the input data more suitable for the visual algorithm; corresponding to the ISP processing process, the input data adapter includes the following processes: gain and noise reduction, white balance, color correction, tone mapping process;

[0043] Specifically, before the ISP processing process, the input data I1 is first de-mosaicked. Since the sensor can only capture photons and cannot perceive color light, it cannot perceive chroma information, and mosaic phenomenon will occur. Usually, interpolation is used to remove the mosaic; for parameter learning in the gain and noise reduction, white balance, color correction, tone mapping process in the input data adapter, the self-attention mechanism method is used to calculate, and the key and value values of the attention module are generated by using multiple linear layers, and the query is a learnable parameter, which is specifically represented as:

[0044] Wherein, FFN represents a forward propagation calculation network, including two linear layers and an activation function layer; for the gain and noise reduction process, first define the gain ratio g, which is used to adapt the input image I1 to different light scenes, then for noise reduction, an adaptive anisotropic Gaussian kernel is used to suppress noise under various noise conditions, and the Gaussian kernel parameters k are defined, which specifically include the long axis r1, the short axis r2, and the kernel angle θ, and θ is simplified to 0, in addition, the filtering parameter σ is defined to maintain the clarity and detail information of the generated image;

[0045] Specifically, the process of I1 to I2 image is represented as: I2' = (g·I1)*k{r1, r2, θ}; I2 = I2' + (g·I1-I2')·σ, for the white balance and color correction process, the input is the image I2 after the gain and noise reduction process, since the filtered sensor data is affected by the color temperature of the environment light, the camera applies white balance to generate a color image that is satisfactory to the human eye under normal lighting, for white balance, define the parameter ρ, and calculate the white balance parameter ρ by calculating the Minkowski distance; for the color correction process, a 3x3 color correction matrix CCM is used to convert the color space of the camera to the output color space, i.e. the RGB space, and the parameter E ccm is defined, which represents the color correction matrix obtained from the raw data of the RAW file; the calculation process of white balance and color correction is represented as: I4 = I3*E ccm .

[0046] Wherein, D i∈(r,g,b) represents the white balance parameter of each color space, i.e. RGB three colors, which specifically includes D r , D g , Db ; avg() represents calculating average value, represents white balance matrix, I2 represents image after gain and noise reduction process, I3 represents image after white balance, E ccm represents color correction matrix, I4 represents image after color correction.

[0047] For tone mapping process, usually look-up table method is used to realize, in order to realize end-to-end learning process of algorithm, by using neural network representation method, input pixel intensity RGB is mapped to continuous coordinate space, by using a multilayer perceptron, specific representation is as follows: I5(R ’ , G ’ , B ’ ) = MLP (I4(R, G, B)), MLP represents multilayer perceptron, finally, input adapter output result I5 is obtained.

[0048] For the embodiment of the application, before step 101, the raw data set of different visual tasks can also be constructed; specifically, the visual tasks include but are not limited to commonly used visual algorithms such as target detection and semantic segmentation; wherein the data set can be obtained by two ways, one is real data acquisition, and the other is data synthesis by using related algorithms to obtain corresponding raw images; for the way of acquiring real data, by using the camera used by high-bit video monitoring equipment or video pile monitoring equipment, the camera is fixed on a stable tripod, so as to ensure that the photographed image is aligned at pixel level, different scenes are photographed, the exposure time of the camera can be adjusted to simulate images under different light conditions, and RGB images and raw data RAW images are obtained; since the cost of obtaining the above image by manual shooting is high, the shooting process is very time-consuming, and a large amount of time and manpower cost is needed to obtain a large amount of data for model training, therefore, the simulated raw image is obtained by using the synthesis method; for raw synthesis data, a large number of RGB images under different light conditions are used, a series of data inverse processing process and noise simulation and injection are used to obtain the RAW image of the corresponding scene;

[0049] 102, the intermediate stage feature information is extracted from the intermediate data in the filtering correction process by the preset intermediate stage feature extraction adapter.

[0050] Specifically, the intermediate stage feature information is extracted from the intermediate data in the filtering correction process by the convolution model in the preset intermediate stage feature extraction adapter, and the feature splicing is performed; the intermediate stage feature information after feature splicing is processed by the residual convolution model in the preset intermediate stage feature extraction adapter, and the adaptive feature is generated.

[0051] For example, the above-mentioned input-level adapter ensures that the input raw image is converted into data for vision algorithms. However, the intermediate results during the ISP data conversion process are often not fully utilized, including I1 to I4. The pre-set intermediate stage feature extraction adapter integrates the intermediate information in the ISP process into the backbone network of the model, realizing a tight connection between the ISP process and the subsequent vision model network, thus promoting the capabilities of the vision algorithm. Specifically, the pre-trained vision model backbone network represents the feature extraction network part of the vision model, which is then connected to the detection or segmentation network part of the vision model. Specifically, the pre-trained vision model backbone network is divided into 4 stages. The output I5 of the above steps will pass through the 4 stages of the backbone network one by one, connecting to the detection or segmentation network part of the vision model. For I1 to I4, the features of I1 to I4 are first extracted using a convolutional module, which includes multiple convolutional layers, activation function layers, normalization layers, etc. Then, the extracted features are concatenated, specifically represented as: C(I1 to I4) = C(conv(I1), conv(I2), conv(I3), conv(I4)). The concatenated features C (I1~I4) are fed into the residual convolution module to obtain the adapted features f. The residual convolution module contains connections of multiple residual convolutional neural networks. The normalization layer includes, but is not limited to, normalization operations such as instance normalization layer and adaptive instance normalization layer. The nonlinear activation layer includes, but is not limited to, nonlinear activation functions such as ReLU and Leaky ReLU.

[0052] 103. Use the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model.

[0053] Specifically, the adaptation features are fused with the output results of each stage in the pre-trained visual model.

[0054] 104. Update and embed the output of the pre-trained visual model according to the intermediate stage feature information and then output it.

[0055] Specifically, the adaptive feature f is fused with the output of the first stage of the backbone network using a feature fusion module. The output is then fed into the second stage of the backbone network, where the adaptive feature f is fused with the output of the second stage of the backbone network again using the feature fusion module. The same operation is performed on the third stage of the backbone network. The output of the fourth stage of the backbone network is connected to the detection or segmentation network part of the visual model for learning the visual algorithm. Specifically, the feature fusion module includes multiple convolutional layers and residual connections to achieve the fusion of different features.

[0056] The camera raw image data processing matching method provided by the embodiment of the present application adapts the visual model pre-trained based on an RGB image to camera raw image data, adjusts raw input data by using a learnable ISP process through constructing an input data adapter to obtain input data more suitable for the input of a visual algorithm, and further constructs a model-level intermediate stage feature extraction adapter between the ISP stage and the subsequent visual algorithm network, extracts intermediate stage features from the input-level adapter, embeds the features into the subsequent visual algorithm network, lets the network contain prior knowledge from the ISP stage, and improves the capability of the visual algorithm.

[0057] As Figure 1 As a specific implementation of the method shown, the embodiment of the present application provides a camera raw image data processing matching system, which comprises: Figure 2 A generating module 21 is configured to filter and correct input camera raw image data through an input data adapter to generate input data of a pre-trained visual model.

[0058] An extracting module 22 is configured to extract intermediate stage feature information from intermediate data in the filtering and correction process through a preset intermediate stage feature extraction adapter.

[0059] The generating module 21 is further configured to take the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model.

[0060] An embedding output module 23 is configured to update and embed the output of the pre-trained visual model according to the intermediate stage feature information and output.

[0061] Further, the generating module 21 is specifically configured to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate the input data of the pre-trained visual model.

[0062] Further, the generating module 21 is specifically configured to generate the input data of the pre-trained visual model according to the formula I2'=(g*I1)*k{r1, r2, θ}, I2=I2'+(g*I1-I2')*σ, I4=I3*E ccm , I5(R', G', B')=MLP(I4(R, G, B)), where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the long axis is r1, the short axis is r2, the kernel angle is θ, the filtering parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is E ccm, MLP represents a multi-layer perceptron, D i∈(r,g,b) , and D r

[0063] [D g ] represents a white balance matrix, I2 represents an image after a gain and noise reduction process, and I3 represents an image after white balance processing. b

[0064] , and I4 represents an image after color correction.

[0065] Further, the extraction module 22 is specifically configured to extract intermediate stage feature information from intermediate data in the filtering correction processing process by a convolution model in a preset intermediate stage feature extraction adapter and perform feature splicing, and process the intermediate stage feature information after the feature splicing by a residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptive features.

[0066] Further, the embedding output module 23 is specifically configured to perform feature fusion on the adaptive features and output results of each stage in the pre-trained visual model.

[0067] The camera raw image data processing matching system provided by the embodiment of the application adapts a visual model pre-trained based on an RGB image to camera raw image data, adjusts raw input data by using a learnable ISP process through constructing an input data adapter to obtain input data more suitable for visual algorithm input, and further constructs a model level intermediate stage feature extraction adapter between the ISP stage and a subsequent visual algorithm network, embeds intermediate stage features from the input level adapter into the subsequent visual algorithm network by extracting the intermediate stage features, and lets the network contain prior knowledge from the ISP stage to improve the capability of the visual algorithm.

[0068] It should be understood that the specific order or hierarchy of steps in the disclosed processes is an example of exemplary methods. Based on design preferences, it is understood that the specific order or hierarchy of steps in the processes can be rearranged, and further that implementations may

[0069] In the above detailed description, various features are grouped together in single embodiments for the purpose of streamlining the disclosure. This method of disclosure, however, is not to be interpreted as reflecting a necessity or desire to group the claimed subject matter into single embodiments having multiple aspects. To the contrary, as will be apparent from the appended claims, it is contemplated that various aspects of the claimed subject matter could be more tangibly separated into individual aspects or be implemented with various types of hardware, software, firmware, etc. The claims, therefore, should not be construed as coded claims directed to items "including" multiple aspects or components. Rather, they should be understood as follows:

[0070] The disclosed embodiments herein can be described with reference to Example implementations. Any implementation of the techniques described herein can be implemented to realize one or more of the following advantages. Such advantages can be realized in any combination. The disclosed embodiments are not limited to any advantages, described herein, which will become apparent to one of ordinary skill in the art upon review of this document.

[0071] The foregoing description of various embodiments of the application includes illustrative examples. Of course, not all possible combinations of components or method steps described herein will fall within the scope of the claims. Those skilled in the art will recognize, however, that the various embodiments described herein can be further modified than described above and fall within the scope of the present disclosure. Therefore, the disclosure described herein aims to encompass all such changes and modifications as fall within the scope of the claims. Further, to the extent not already described, the terms "comprising," "including," "containing," and "having" and variations thereof are used as equivalent terms without limitation. Thus, for example, descriptions using the term "comprising" are intended to include into the description also descriptions using the terms "consisting of" and "consisting essentially of."

[0072] Those of skill would further appreciate that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

[0073] The various illustrative logical blocks, modules, or units described in connection with the embodiments disclosed herein can be implemented or performed with a general purpose processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the general purpose processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other such configuration.

[0074] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium can be coupled to the processor, such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC, which can reside in a user terminal. In the alternative, the processor and the storage medium can also reside in different components of a user terminal.

[0075] In one or more exemplary designs, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a computer. By way of example, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or other wire-based, fiber- optic based, or wireless based communications, then the coaxial cable, fiber optic cable, twisted pair, DSL, or other wire-based, fiber-optic based, or wireless based communications are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, DVD, floppy disk, and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0076] The specific implementation described above is illustrative for purposes of teaching the present application. The application should not be limited, however, to these specific implementation. Any modifications, equivalents, or improvements, as long as they are within the spirit and scope of the present application are included in the scope of the present application.

Claims

1. A camera raw image data processing matching method, characterized by, The method comprises: filtering and correcting input camera raw image data through an input data adapter to generate input data of a pre-trained visual model; extracting intermediate stage feature information from intermediate data in the filtering and correcting process through a preset intermediate stage feature extraction adapter; inputting the input data of the pre-trained visual model into the pre-trained visual model to generate output of the pre-trained visual model; updating and embedding the output of the pre-trained visual model according to the intermediate stage feature information and outputting; the step of extracting intermediate stage feature information from the input data of the pre-trained visual model and the intermediate data in the filtering and correcting process through the preset intermediate stage feature extraction adapter comprises: extracting intermediate stage feature information from the intermediate data in the filtering and correcting process through a convolution model in the preset intermediate stage feature extraction adapter and performing feature splicing; and processing the spliced intermediate stage feature information through a residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptive features; the step of updating and embedding the output of the pre-trained visual model according to the intermediate stage feature information comprises: fusing the adaptive features with output results of each stage in the pre-trained visual model.

2. The method of claim 1, wherein, the step of filtering and correcting input camera raw image data through the input data adapter to generate input data of the pre-trained visual model comprises: performing gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate input data of the pre-trained visual model.

3. The method of claim 2, wherein, the step of filtering and correcting input camera raw image data through the input data adapter to generate input data of the pre-trained visual model comprises: According to the formula I2' = (g * I1) * k {r1, r2, θ}, I2 = I2' + (g * I1 - I2') * σ, I4 = I3 * E ccm , I5(R', G', B') = M LP(I4(R, G, B)) generates input data of a pre-trained visual model, wherein the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the long axis r1, the short axis r2, the kernel angle θ, the filtering parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is E ccm , MLP represents a multi-layer perceptron, D i∈(r,g,b) represents a white balance parameter of each color space, i.e., RGB three colors, represents a white balance matrix, I2 represents an image after a gain and noise reduction process, I2' represents an intermediate process image, I3 is an image after white balance, and I4 represents an image after color correction.

4. A camera raw image data processing matching system, characterized by, the system comprises: a generation module configured to filter and correct input camera raw image data through an input data adapter to generate input data of a pre-trained visual model; an extraction module configured to extract intermediate stage feature information from intermediate data in the filtering and correcting process through a preset intermediate stage feature extraction adapter; the generation module is further configured to input the input data of the pre-trained visual model into the pre-trained visual model to generate output of the pre-trained visual model; an embedding and output module configured to update and embed the output of the pre-trained visual model according to the intermediate stage feature information and output; the extraction module is specifically configured to extract intermediate stage feature information from the intermediate data in the filtering and correcting process through a convolution model in the preset intermediate stage feature extraction adapter and perform feature splicing; and process the spliced intermediate stage feature information through a residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptive features; the embedding and output module is specifically configured to fuse the adaptive features with output results of each stage in the pre-trained visual model. 5.The camera raw image data processing matching system of claim 4, wherein the generation module is specifically configured to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate the input data of the pre-trained visual model. 6.The camera raw image data processing matching system of claim 5, wherein the generation module is specifically configured to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate the input data of the pre-trained visual model. ​ The generating module is specifically configured to generate input data of the pre-trained visual model according to a formula I2'= (g*I1)*k{r1, r2, theta}, I2=I2'+ (g*I1-I2')*sigma, I3=I3+ (g*I1-I3)*rho, I4=I3*E, I5(R', G', B')=MLP(I4(R, G, B)), wherein a gain ratio is g, input camera raw image data is I1, a Gaussian kernel parameter is k, a long axis is r1, a short axis is r2, a kernel angle is theta, a filtering parameter is sigma, input data of the pre-trained visual model is I5, a white balance parameter is rho, and a color correction matrix is E. I4=I3*E ccm , I5(R', G', B')=MLP(I4(R, G, B)) generate input data of the pre-trained visual model, wherein the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the long axis r1, the short axis r2, the kernel angle theta, the filtering parameter is sigma, the input data of the pre-trained visual model is I5, the white balance parameter is rho, and the color correction matrix is E ccm , MLP represents a multi-layer perception machine, D i∈(r,g,b) represents a white balance parameter of each color space, i.e., three colors of RGB, represents a white balance matrix, I2 represents an image after a gain and noise reduction process, I2' represents an intermediate process image, I3 is an image after white balance, and I4 represents an image after color correction.

Images