Camera raw image data processing and matching method and system

Abstract

The invention discloses a camera raw image data processing and matching method and system, and relates to the field of intelligent parking management, and the method comprises the steps: enabling a visual model based on RGB image pre-training to be adaptive to camera raw image data, building an input data adapter, employing a learnable ISP process to adjust raw input data, and enabling the raw input data to be matched with the visual model. Input data which is more adaptive to visual algorithm input are obtained; besides, a model-level intermediate stage feature extraction adapter is constructed between an ISP stage and a subsequent visual algorithm network, intermediate stage features from an input stage adapter are extracted, and the features are embedded into the subsequent visual algorithm network, so that the network contains priori knowledge from the ISP stage, and the capability of the visual algorithm is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent parking management, and in particular to a camera raw image data processing and matching method and system. Background Art

[0002] In recent years, with the development of social economy, people's demand for convenience in daily travel has continued to increase, and the number of cars has continued to increase, resulting in parking difficulties and parking chaos, which has become one of the main problems that need to be solved in the development of urban transportation in my country. In recent years, with the development of intelligent technology, in order to effectively alleviate this problem, traffic management departments in various places have tried and built various intelligent parking management technologies. Among them, setting up parking spaces on the roadside, using intelligent sensing equipment to collect video images and other data, and processing and analyzing video image data through computer vision algorithms to achieve intelligent parking management is currently a mainstream solution.

[0003] However, in special scenes such as night, rain and fog, the RGB images captured by the camera have low clarity and large image noise, making it difficult for the visual algorithm to achieve accurate recognition results on such images. In many cases, it is also difficult for the human eye to distinguish the content in the image, which makes it difficult to accurately analyze the high-bit video scene image. For such low-light images, at present, the RGB images generated by the camera are mainly based on the generation of adversarial networks or diffusion models. The algorithm is implemented, but it cannot restore the low-light image under many conditions with poor lighting conditions, and it is easy to introduce some noise information, which affects the application of the visual algorithm. However, the original raw data collected by the camera has richer physical information under these changing and challenging lighting conditions. The raw image is usually 12-bit or 14-bit data, containing a large amount of original image information. During the camera imaging process, the image signal processing process (Image Signal Process, ISP) receives the original raw image, and after a series of image signal processing, an RGB image that is usually more friendly to human visual perception is obtained, which is usually an 8-bit image.

[0004] At present, for computer vision tasks directly based on raw data, the ISP process is usually combined with the network of the back-end visual task. However, the interaction ability between the ISP process and the subsequent visual algorithm network is often ignored, resulting in poor matching between the existing camera raw image data and the visual model. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a camera raw image data processing and matching method and system, which can solve the problem that the matching degree between the existing camera raw image data and the visual model is poor.

[0006] To achieve the above object, the present invention provides a camera raw image data processing and matching method, the method comprising:

[0007] The input camera raw image data is filtered and corrected through the input data adapter to generate input data for the pre-trained visual model;

[0008] Extracting intermediate stage feature information from the intermediate data in the filtering and correction process by presetting an intermediate stage feature extraction adapter;

[0009] Using the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model;

[0010] The output of the pre-trained visual model is updated, embedded and output according to the intermediate stage feature information.

[0011] Furthermore, the step of filtering and correcting the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model includes:

[0012] The input data adapter performs gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data to generate input data for the pre-trained visual model.

[0013] Furthermore, the step of performing gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model includes:

[0014] According to the formula I2 = (g·I1)*k{r1, r2, θ}, I2 = I2 + (g·I1-I2)·σ,

[0015] I4=I3*

[0016] E ccm 、I5(R ’ , G ’ , B ’ )=MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the major axis r1, the minor axis r2, the kernel angle θ, the filter parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is ​​E ccm , MLP stands for multi-layer perceptron, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB. represents the white balance matrix, I2 represents the image after gain and noise reduction process, I3 represents the image after white balance; I4 represents the image after color correction.

[0017] Furthermore, the step of extracting intermediate stage feature information from the input data of the pre-trained visual model and the intermediate data in the filtering and correction process by presetting the intermediate stage feature extraction adapter comprises:

[0018] Extracting intermediate stage feature information from intermediate data in the filtering and correction process through a convolution model in a preset intermediate stage feature extraction adapter and performing feature splicing;

[0019] The intermediate stage feature information after feature splicing is processed through the residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptation features.

[0020] Furthermore, the step of updating and embedding the output of the pre-trained visual model according to the intermediate stage feature information includes:

[0021] The adapted features are fused with the output results of each stage in the pre-trained visual model.

[0022] Furthermore, the present invention provides a camera raw image data processing and matching system, the system comprising:

[0023] A generation module is used to filter and correct the input camera raw image data through an input data adapter to generate input data for a pre-trained visual model;

[0024] An extraction module, used for extracting intermediate stage feature information from the intermediate data in the filtering and correction process by means of a preset intermediate stage feature extraction adapter;

[0025] The generating module is further used to use the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model;

[0026] The embedding output module is used to update, embed and output the output of the pre-trained visual model according to the intermediate stage feature information.

[0027] Furthermore, the generation module is specifically used to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through an input data adapter to generate input data for the pre-trained visual model.

[0028] Furthermore, the generating module is specifically used to generate the following formula: I2=(g·I1)*k{r1, r2, θ},

[0029] I2=I2+(g·I1-I2)·σ,

[0030] I4=I3*E ccm 、I5(R ’ , G ’ , B ’ )=MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the major axis r1, the minor axis r2, the kernel angle θ, the filter parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is ​​E ccm , MLP stands for multi-layer perceptron, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB. represents the white balance matrix, I2 represents the image after gain and noise reduction process, I3 represents the image after white balance; I4 represents the image after color correction.

[0032] Furthermore, the extraction module is specifically used to extract intermediate stage feature information and perform feature splicing from the intermediate data in the filtering and correction processing process through the convolution model in the preset intermediate stage feature extraction adapter; and process the intermediate stage feature information after feature splicing through the residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptation features.

[0033] Furthermore, the embedding output module is specifically used to perform feature fusion of the adaptation features with the output results of each stage in the pre-trained visual model.

[0034] The present invention provides a camera raw image data processing and matching method and system, which adapt a visual model pre-trained based on RGB images to the camera raw image data, and adjust the raw input data by constructing an input data adapter and utilizing a learnable ISP process to obtain input data that is more suitable for visual algorithm input. In addition, a model-level intermediate stage feature extraction adapter is constructed between the ISP stage and the subsequent visual algorithm network, and intermediate stage features from the input-level adapter are extracted and embedded into the subsequent visual algorithm network, so that the network includes prior knowledge from the ISP stage, thereby improving the capability of the visual algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a camera raw image data processing and matching method provided by the present invention;

[0036] Figure 2It is a schematic diagram of a camera raw image data processing and matching system provided by the present invention. DETAILED DESCRIPTION

[0037] The device structure and implementation of the present invention are further described in detail below through the accompanying drawings and embodiments.

[0038] The present invention provides a camera raw image data processing and matching method, such as Figure 1 As shown, the specific steps include:

[0039] 101. Filter and correct the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model.

[0040] Specifically, the step of filtering and correcting the input camera raw image data through the input data adapter to generate the input data of the pre-trained visual model includes: performing gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate the input data of the pre-trained visual model.

[0041] Furthermore, the step of performing gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model includes: according to the formula I2'=(g·I1)*k{r1, r2, θ}, I2=I2'+(g·I1-I2')·σ, I4=I3*E ccm , I5(R', G', B') = MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the major axis r1, the minor axis r2, the kernel angle θ, the filter parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is ​​E ccm , MLP stands for multi-layer perceptron, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB. represents the white balance matrix, I2 represents the image after gain and noise reduction process, I3 represents the image after white balance; I4 represents the image after color correction.

[0042] For example, the input raw data is defined as I1. After the designed input data adapter, the input data I5 for the visual algorithm is obtained. For the visual algorithm, it means that it is more suitable for the input of the visual algorithm. Corresponding to the ISP processing process, the input data adapter includes the following processes: gain and noise reduction, white balance, color correction, and tone mapping process.

[0043] Specifically, before the ISP processing, the input data I1 is first demosaiced. Since the sensor can only capture photons and cannot perceive color light or chromaticity information, mosaics will appear. Interpolation is usually used to remove mosaics. For the parameter learning in the input data adapter, such as gain and noise reduction, white balance, color correction, and tone mapping, the self-attention mechanism is used to calculate the key and value of the attention module by using multiple linear layers. The query is the learnable parameter, which is specifically expressed as:

[0044] Among them, FFN represents the forward propagation computing network, which includes two linear layers and one activation function layer; for the gain and denoising process, firstly, the gain ratio g is defined to adapt the input image I1 to different lighting scenes, and then for denoising, an adaptive anisotropic Gaussian kernel is used to suppress noise under various noise conditions, and the Gaussian kernel parameter k is defined, including the major axis r1, the minor axis r2, and the kernel angle θ, which is simplified to θ = 0, and the filter parameter σ is defined to maintain the clarity and detail information of the generated image;

[0045] Specifically, the process from I1 to I2 image is expressed as: I2' = (g·I1)*k{r1, r2, θ}; I2 = I2' + (g·I1-I2')·σ. For the white balance and color correction process, the input is the image I2 after the gain and noise reduction process. Since the filtered sensor data is affected by the color temperature of the ambient light, the camera applies white balance to generate a color image that is visually satisfactory to the human eye under normal lighting. For white balance, a parameter ρ is defined, and the white balance parameter ρ is calculated by calculating the Minkowski distance; for the color correction process, a 3×3 color correction matrix CCM is used to convert the camera's color space into the output color space, that is, the RGB space, and the parameter E is defined. ccm , represents the color correction matrix, which is obtained from the original data of the RAW file; the calculation process for white balance and color correction is expressed as: I4=I3*E ccm .

[0046] Among them, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB, including: r , D g , Db ;avg() means calculating the average value, represents the white balance matrix, I2 represents the image after gain and noise reduction, I3 represents the image after white balance, E ccm represents the color correction matrix, and I4 represents the image after color correction.

[0047] For the tone mapping process, a table lookup method is usually used. In order to realize the end-to-end learning process of the algorithm, the input pixel intensity RGB is mapped to a continuous coordinate space by using a neural network representation method. This is realized by using a multi-layer perceptron, which is specifically expressed as: I5(R ’ , G ’ , B ’ )=MLP(I4(R,G,B)), MLP represents a multi-layer perceptron, and finally the input adapter output result I5 is obtained.

[0048] For the embodiment of the present invention, before step 101, it may also include: constructing raw data sets for different visual tasks; specifically, the visual tasks include but are not limited to commonly used visual algorithms such as target detection and semantic segmentation; wherein the data set can be acquired in two ways, one is the acquisition of real data, and the other is to obtain the corresponding raw image by synthesizing data using relevant algorithms; for the method of acquiring real data, by using a camera used by a high-position video monitoring device or a video pile monitoring device, the camera is fixed on a stable tripod, so as to ensure that the captured image is aligned at the pixel level, and different scenes can be captured by adjusting the exposure time of the camera to simulate images under different lighting conditions to obtain RGB images and raw data RAW images; since the cost of manually shooting to obtain the above-mentioned images is high and the shooting process is very time-consuming, it takes a lot of time and manpower to obtain a large amount of data for model training, therefore, it is considered to generate the raw image by simulation by using a synthesis method; for raw synthetic data, a large number of RGB images under different lighting conditions are used, and a series of data inverse processing processes and noise simulation and injection are performed to obtain the RAW image of the corresponding scene;

[0049] 102. Extracting intermediate stage feature information from the intermediate data in the filtering and correction process by means of a preset intermediate stage feature extraction adapter.

[0050] Specifically, the intermediate stage feature information is extracted from the intermediate data in the filtering and correction process through the convolution model in the preset intermediate stage feature extraction adapter, and feature splicing is performed; the intermediate stage feature information after feature splicing is processed through the residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptation features.

[0051] For example, for the above-mentioned input-level adapter, it is guaranteed that the input raw image is converted into data for the visual algorithm, but the intermediate results in the ISP data conversion process are often not fully utilized, including I1~I4; the preset intermediate stage feature extraction adapter integrates the intermediate information in the ISP process into the backbone network of the model, realizes the close connection between the ISP process and the subsequent visual model network, and promotes the ability of the visual algorithm; specifically, the pre-trained visual model backbone network represents the feature extraction network part of the visual model, and is later connected to the detection or segmentation network part of the visual model; specifically, the pre-trained visual model backbone network is divided into 4 stages, wherein the output I5 of the above steps will pass through the 4 stages of the backbone network one by one, and be connected to the detection or segmentation network part of the visual model; for I1~I4, firstly, a convolution module is used to extract the features of I1~I4, and the convolution module includes multiple convolution layers, activation function layers, normalization layers, etc.; then, the extracted features are feature spliced, specifically expressed as: C(I1~I4)=C(conv(I1),conv(I2),conv(I3),conv(I4)). The concatenated features C (I1~I4) are sent to the residual convolution module to obtain the adapted features f. The residual convolution module includes the connection of multiple residual convolutional neural networks; the normalization layer includes but is not limited to normalization operations such as instance normalization layer and adaptive instance normalization layer; the nonlinear activation layer includes but is not limited to nonlinear activation functions such as ReLu and Leaky ReLu.

[0052] 103. Using the input data of the pre-trained visual model as the input of the pre-trained visual model to generate an output of the pre-trained visual model.

[0053] Specifically, the adaptation features are fused with the output results of each stage in the pre-trained visual model.

[0054] 104. Update, embed and output the output of the pre-trained visual model according to the intermediate stage feature information.

[0055] Specifically, the adapted feature f is fused with the output of the first stage of the backbone network using a feature fusion module, and then the output is sent to the second stage of the backbone network. The adapted feature f is fused with the output of the second stage of the backbone network using a feature fusion module again. The same operation is performed on the third stage of the backbone network. The output of the fourth stage of the backbone network is connected to the detection or segmentation network part of the visual model to learn the visual algorithm. Specifically, the feature fusion module includes multiple convolutional layers and residual connections to achieve the fusion of different features.

[0056] A camera raw image data processing and matching method provided by an embodiment of the present invention adapts a visual model pre-trained based on RGB images to the camera raw image data, and obtains input data that is more suitable for visual algorithm input by constructing an input data adapter and utilizing a learnable ISP process. In addition, a model-level intermediate stage feature extraction adapter is constructed between the ISP stage and a subsequent visual algorithm network, and intermediate stage features from the input-level adapter are extracted and embedded into the subsequent visual algorithm network, so that the network includes prior knowledge from the ISP stage, thereby improving the capability of the visual algorithm.

[0057] As Figure 1 The specific implementation of the method shown in the figure, the embodiment of the present invention provides a camera raw image data processing and matching system, such as Figure 2 As shown, the system includes: a generating module 21, which is used to perform filtering and correction processing on the input camera raw image data through an input data adapter to generate input data for a pre-trained visual model;

[0058] An extraction module 22, used for extracting intermediate stage feature information from the intermediate data in the filtering and correction process by means of a preset intermediate stage feature extraction adapter;

[0059] The generating module 21 is further used to use the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model;

[0060] The embedding output module 23 is used to update, embed and output the output of the pre-trained visual model according to the intermediate stage feature information.

[0061] Furthermore, the generating module 21 is specifically used to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through an input data adapter to generate input data for the pre-trained visual model.

[0062] Furthermore, the generating module 21 is specifically used to generate the following formulas: I2'=(g·I1)*k{r1, r2, θ}, I2=I2'+(g·I1-I2')·σ, I4=I3*E ccm , I5(R', G', B') = MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the major axis r1, the minor axis r2, the kernel angle θ, the filter parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is ​​E ccm, MLP stands for multi-layer perceptron, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB, r

[0063] [D g ] represents the white balance matrix, I2 represents the image after gain and noise reduction process, and I3 is D b

[0064] I4 is the image after white balance; I4 is the image after color correction.

[0065] Furthermore, the extraction module 22 is specifically used to extract intermediate stage feature information and perform feature splicing from the intermediate data in the filtering and correction process through the convolution model in the preset intermediate stage feature extraction adapter; and process the intermediate stage feature information after feature splicing through the residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptation features.

[0066] Furthermore, the embedding output module 23 is specifically used to perform feature fusion on the adaptation feature and the output results of each stage in the pre-trained visual model.

[0067] A camera raw image data processing and matching system provided by an embodiment of the present invention adapts a visual model pre-trained based on RGB images to the camera raw image data, and obtains input data that is more suitable for visual algorithm input by constructing an input data adapter and utilizing a learnable ISP process to adjust the raw input data. In addition, a model-level intermediate stage feature extraction adapter is constructed between the ISP stage and a subsequent visual algorithm network, and intermediate stage features from the input-level adapter are extracted and embedded into the subsequent visual algorithm network, so that the network includes prior knowledge from the ISP stage, thereby improving the capability of the visual algorithm.

[0068] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0069] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0070] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0071] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0072] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0073] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0074] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0075] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0076] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A camera raw image data processing and matching method, characterized in that: The method comprises: The input camera raw image data is filtered and corrected through the input data adapter to generate input data for the pre-trained visual model; Extracting intermediate stage feature information from the intermediate data in the filtering and correction process by presetting an intermediate stage feature extraction adapter; Using the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model; The output of the pre-trained visual model is updated, embedded and output according to the intermediate stage feature information.

2. A camera raw image data processing and matching method according to claim 1, characterized in that: The step of filtering and correcting the input camera raw image data by the input data adapter to generate input data for the pre-trained visual model comprises: The input data adapter performs gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data to generate input data for the pre-trained visual model.

3. A camera raw image data processing and matching method according to claim 2, characterized in that: The step of performing gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through the input data adapter to generate input data for the pre-trained visual model includes: According to the formula I2' = (g·I1)*k{r1, r2, θ}, I2' = I2 + (g·I1-I2')·σ, I4=I3*E ccm 、I5(R ’ , G ’ , B ’ )=MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the major axis r1, the minor axis r2, the kernel angle θ, the filter parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is ​​E ccm , MLP stands for multi-layer perceptron, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB. represents the white balance matrix, I2 represents the image after gain and noise reduction process, I2' represents the intermediate process image, I3 represents the image after white balance, and I4 represents the image after color correction.

4. A camera raw image data processing and matching method according to claim 3, characterized in that: The step of extracting intermediate stage feature information from the input data of the pre-trained visual model and the intermediate data in the filtering and correction process by presetting the intermediate stage feature extraction adapter comprises: Extracting intermediate stage feature information from intermediate data in the filtering and correction process through a convolution model in a preset intermediate stage feature extraction adapter and performing feature splicing; The intermediate stage feature information after feature splicing is processed through the residual convolution model in the preset intermediate stage feature extraction adapter to generate adaptation features.

5. A camera raw image data processing and matching method according to claim 4, characterized in that: The step of updating and embedding the output of the pre-trained visual model according to the intermediate stage feature information comprises: The adapted features are fused with the output results of each stage in the pre-trained visual model.

6. A camera raw image data processing and matching system, characterized in that: The system comprises: A generation module is used to filter and correct the input camera raw image data through an input data adapter to generate input data for a pre-trained visual model; An extraction module, used for extracting intermediate stage feature information from the intermediate data in the filtering and correction process by means of a preset intermediate stage feature extraction adapter; The generating module is further used to use the input data of the pre-trained visual model as the input of the pre-trained visual model to generate the output of the pre-trained visual model; The embedding output module is used to update, embed and output the output of the pre-trained visual model according to the intermediate stage feature information.

7. A camera raw image data processing and matching system according to claim 6, characterized in that: The generation module is specifically used to perform gain and noise reduction, white balance processing, color correction processing, and tone mapping processing on the input camera raw image data through an input data adapter to generate input data for the pre-trained visual model.

8. A camera raw image data processing and matching system according to claim 7, characterized in that: The generating module is specifically used to generate the product according to the formulas I2'=(g·I1)*k{r1, r2, θ}, I2=I2'+(g·I1-I2')·σ, I4=I3*E ccm , I5(R', G', B') = MLP(I4(R, G, B)) generates the input data of the pre-trained visual model, where the gain ratio is g, the input camera raw image data is I1, the Gaussian kernel parameter is k, the major axis r1, the minor axis r2, the kernel angle θ, the filter parameter is σ, the input data of the pre-trained visual model is I5, the white balance parameter is ρ, and the color correction matrix is ​​E ccm , MLP stands for multi-layer perceptron, D i∈(r,g,b) Indicates the white balance parameters of each color space, namely RGB. represents the white balance matrix, I2 represents the image after gain and noise reduction process, I2' represents the intermediate process image, I3 represents the image after white balance, and I4 represents the image after color correction.

9. A camera raw image data processing and matching system according to claim 8, characterized in that: The extraction module is specifically used to extract intermediate stage feature information from intermediate data in the filtering and correction process through a convolution model in a preset intermediate stage feature extraction adapter and perform feature splicing; the intermediate stage feature information after feature splicing is processed through a residual convolution model in a preset intermediate stage feature extraction adapter to generate adaptation features.

10. A camera raw image data processing and matching system according to claim 9, characterized in that: The embedding output module is specifically used to perform feature fusion on the adaptation features and the output results of each stage in the pre-trained visual model.

Images