Image processing method and device, chip and readable storage medium

Abstract

The invention relates to an image processing method and device, a chip and a readable storage medium, and relates to the technical field of image processing. The method comprises the following steps: in response to an image shooting instruction, triggering an image acquisition device to start acquiring original image data; in the process of adjusting the image processing parameters by adopting the original image data, caching the original image data in parallel; and after the image processing parameters are adjusted to be converged, performing image processing on the target image data based on the converged image processing parameters, and outputting a shot image. By adopting the method, the real-time performance of image shooting can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, chip and readable storage medium. Background Art

[0002] In some short-duration shooting scenarios (such as shooting license plates while driving, hunting in the wild, etc.), since the subject exists for a short time, users need to capture it in time. In order to improve the capture effect, RAW cameras are often used for shooting, and the captured images are processed based on ISP (Image Signal Processing) technology.

[0003] However, the image signal processing process needs to be adjusted according to the current environmental conditions. After the parameters of the ISP image algorithm converge, a valid image can be captured. This causes a large time delay between the actual image presented by the captured image and the execution time of the capture action. Therefore, the existing image capture method has the problem of poor real-time performance. Summary of the invention

[0004] Based on this, it is necessary to provide an image processing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the real-time performance of image capture in order to address the above technical issues.

[0005] In a first aspect, an embodiment of the present application provides an image processing method. The method comprises:

[0006] In response to the image capturing instruction, triggering the image capturing device to start capturing raw image data;

[0007] In the process of adjusting the image processing parameters using the original image data, the original image data is cached in parallel; the image processing parameters are parameters for controlling the image processing method;

[0008] After the image processing parameters are adjusted to converge, image processing is performed on target image data based on the converged image processing parameters to output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

[0009] In one embodiment, the adjusting the image processing parameters using the original image data includes:

[0010] Based on the image processing parameters, performing image processing on the original image data of the current frame, and outputting a processed image;

[0011] When the image evaluation information of the processed image does not meet the preset requirements, adjusting the image processing parameters to obtain new image processing parameters;

[0012] The original image data of the next frame is used as the original image data of the current frame, and the step of performing image processing on the original image data of the current frame based on the image processing parameters and outputting the processed image is returned.

[0013] In one embodiment, the method further comprises:

[0014] When the image evaluation information of the processed image meets the preset requirements, it is determined that the image processing parameters are adjusted to converge.

[0015] In one embodiment, before the step of triggering the image acquisition device to start acquiring raw image data in response to the image capture instruction, the method further includes:

[0016] Sensor data of the current shooting scene acquired based on the sensor associated with the image acquisition device;

[0017] When the sensor data meets a preset capture condition, it is determined that the image capture instruction is acquired.

[0018] In one of the embodiments, the target image data also includes the first N frames of original image data cached before the image processing parameters are adjusted to converge, where N is a positive integer greater than or equal to 1.

[0019] In one of the embodiments, the image processing method includes at least one of image denoising, image color restoration, image white balance adjustment or image exposure control.

[0020] In a second aspect, the present application also provides an image processing device. The device comprises:

[0021] An image acquisition module, configured to trigger an image acquisition device to start acquiring raw image data in response to an image capture instruction;

[0022] An image caching module, used to cache the original image data in parallel during the process of adjusting image processing parameters using the original image data; the image processing parameters are parameters for controlling the image processing method;

[0023] A captured image output module is used to perform image processing on target image data based on the converged image processing parameters after the image processing parameters are adjusted to converge, and output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

[0024] In a third aspect, the present application further provides a chip. When the chip executes a computer program, the following steps are implemented:

[0025] In response to the image capturing instruction, triggering the image capturing device to start capturing raw image data;

[0026] In the process of adjusting the image processing parameters using the original image data, the original image data is cached in parallel; the image processing parameters are parameters for controlling the image processing method;

[0027] After the image processing parameters are adjusted to converge, image processing is performed on target image data based on the converged image processing parameters to output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0029] In response to the image capturing instruction, triggering the image capturing device to start capturing raw image data;

[0030] In the process of adjusting the image processing parameters using the original image data, the original image data is cached in parallel; the image processing parameters are parameters for controlling the image processing method;

[0031] After the image processing parameters are adjusted to converge, image processing is performed on target image data based on the converged image processing parameters to output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

[0032] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0033] In response to the image capturing instruction, triggering the image capturing device to start capturing raw image data;

[0034] In the process of adjusting the image processing parameters using the original image data, the original image data is cached in parallel; the image processing parameters are parameters for controlling the image processing method;

[0035] After the image processing parameters are adjusted to converge, image processing is performed on target image data based on the converged image processing parameters to output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

[0036] The above-mentioned image processing method, device, computer equipment, storage medium and computer program product first respond to the image shooting instruction to trigger the image acquisition device to start acquiring raw image data, and then cache the raw image data in parallel during the process of adjusting the image processing parameters to facilitate subsequent processing and output needs, and then after the image processing parameters are adjusted to convergence, the target image data is finally processed based on the converged parameters to output the required captured image; by caching the raw image data in parallel during the parameter adjustment process and using the raw image data to output the final image after the parameters converge, the time difference between the expected image and the actual scene image is shortened, that is, the delay from the time corresponding to the actual picture presented by the captured image to the execution time of the snapshot action is shortened, thereby achieving low-latency shooting and improving the real-time performance of image shooting. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a schematic flow chart of an image processing method in one embodiment;

[0039] Figure 2 is a schematic flow chart of an image processing method in another embodiment;

[0040] Figure 3 It is a schematic diagram of the principle of the prior art in one embodiment;

[0041] Figure 4 is a schematic diagram of the principle of an image processing method in an embodiment;

[0042] Figure 5 is a schematic diagram of the principle of an image processing method in another embodiment;

[0043] Figure 6 FIG. 4 is a structural block diagram of an image processing device in an embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0046] In one embodiment, Figure 1 As shown, an image processing method is provided. This embodiment takes the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0047] S101, in response to an image shooting instruction, triggering an image acquisition device to start acquiring raw image data.

[0048] The image capture instruction may be a capture instruction input by a user, or a capture instruction triggered by a sensor in a corresponding scene.

[0049] The image shooting instruction may be a camera, such as a RAW camera, also known as a raw image camera or a RAW format camera, which refers to a camera that can output image data in a RAW format.

[0050] Among them, the original image data is the image frames that are captured when the camera starts shooting and are not discarded, for example, the first frame of image captured after the camera starts shooting, for example, the first three frames of image captured after the camera starts shooting, or the first six frames of image captured after the camera starts shooting.

[0051] S102, in the process of adjusting image processing parameters using the original image data, buffering the original image data in parallel.

[0052] Among them, the image processing parameter is a parameter that controls the image processing method. For example, in a scenario where image processing needs to increase the brightness of an image, the image processing parameter may be a contrast factor. By changing the contrast factor, the intensity of the linear transformation process of the image data may be adjusted.

[0053] In step S102, while the original image data is used to adjust the image processing parameters, the original image data is cached. For example, the first three frames of images taken after the camera is started are cached to facilitate subsequent use of the first three frames of images for image output.

[0054] S103, after the image processing parameters are adjusted to converge, image processing is performed on the target image data based on the converged image processing parameters, and a captured image is output.

[0055] In the process of adjusting the image processing parameters using the original image data in step S102, the parameter values ​​of the image processing parameters are adjusted by evaluating the image processing effect of each frame of the original image data. When the image processing effect meets the preset requirements, it is determined that the image processing parameters are adjusted to convergence.

[0056] The image processing may be any ISP processing method, and the captured image is the final output image required by the photographer.

[0057] Among them, the target image data includes the original image data that has been cached before the image processing parameters are adjusted to convergence. For example, six frames of images are captured after the camera starts shooting. When the third frame of image is captured, the image processing parameters are adjusted to convergence. In this case, the target image data can be the first three frames of images captured after the camera starts shooting. More specifically, the target image data can also be the first frame of image captured after the camera starts shooting.

[0058] In the above-mentioned image processing method, first, the image acquisition device is triggered to start acquiring raw image data in response to an image shooting instruction, and then the raw image data is cached in parallel during the process of adjusting the image processing parameters to facilitate subsequent processing and output needs. After the image processing parameters are adjusted to convergence, the target image data is finally processed based on the converged parameters to output the required captured image. By caching the raw image data in parallel during the parameter adjustment process and using the raw image data to output the final image after the parameters converge, the time difference between the expected image and the actual scene image is shortened, that is, the delay from the time corresponding to the actual picture presented by the captured image to the execution time of the snapshot action is shortened, thereby achieving low-latency shooting and improving the real-time performance of image shooting.

[0059] In one embodiment, original image data is used to adjust image processing parameters, including: based on the image processing parameters, performing image processing on the original image data of the current frame, and outputting a processed image; when image evaluation information of the processed image does not meet preset requirements, adjusting the image processing parameters to obtain new image processing parameters; using the original image data of the next frame as the original image data of the current frame, returning to the step of performing image processing on the original image data of the current frame based on the image processing parameters, and outputting a processed image.

[0060] Among them, the image evaluation information can be an evaluation index value for the processed image. For example, in a scenario where image processing needs to increase the brightness of the image, the image evaluation information is the brightness value of the image, and the preset requirement refers to the target value for increasing the brightness.

[0061] Generally, the process of adjusting image processing parameters using original image data requires utilizing each frame of the original image data and performing processing and effect verification frame by frame in the order of the image frames.

[0062] Exemplarily, the target value for increasing the brightness is to adjust the brightness of the image to 70. First, using the initially set image processing parameters (contrast factor), after the first frame of the original image data (current frame) is adjusted, the image brightness is 40, which does not meet the preset requirements (brightness is 70), that is, the image evaluation information of the processed image does not meet the preset requirements. At this time, it is necessary to use the second frame of the original image data (the next frame of the original image data is used as the current frame) to continue adjusting the parameters.

[0063] In this embodiment, the original image data is first used to adjust the image processing parameters, image processing is performed based on the original image data of the current frame and the processed image is output, and then the image evaluation information of the processed image is analyzed. If it is not satisfied, the image processing parameters are adjusted to obtain new parameter values, and then the next frame is used as the new current frame to continue parameter adjustment; this embodiment improves the adjustment efficiency of the image processing parameters by processing each frame of the image frame by frame and verifying the effect.

[0064] In one embodiment, the method further includes: when the image evaluation information of the processed image meets a preset requirement, determining that the image processing parameters are adjusted to converge.

[0065] Exemplarily, the target value for increasing the brightness is to adjust the brightness of the image to 70. First, using the initially set image processing parameters (contrast factor), after the first frame of the original image data (current frame) is adjusted, the image brightness is 40, which does not meet the preset requirements (brightness is 70). At this time, it is necessary to use the second frame of the original image data (the next frame of the original image data is used as the current frame) to continue adjusting the parameters; after the second frame is processed to increase the brightness, the brightness is 50, until the brightness of the fourth frame is increased to 70. When the target is reached, that is, the image evaluation information of the processed image meets the preset requirements, it is determined that the image processing parameters are adjusted to convergence.

[0066] In this embodiment, the image evaluation information of the processed image satisfies the preset requirements and determines that the image processing parameters are adjusted to converge, which provides a stable judgment basis for the convergence of the parameters and improves the accuracy of the recognition parameter convergence process.

[0067] In one embodiment, before the step of triggering the image acquisition device to start acquiring raw image data in response to an image capture instruction, the method also includes: acquiring sensor data of the current shooting scene based on sensor acquisition associated with the image acquisition device; and determining that the image capture instruction has been obtained when the sensor data meets a preset capture condition.

[0068] Among them, the sensors associated with the image acquisition device refer to the sensors of other associated systems in the current shooting scene. For example, in the shooting scene where the driving recorder is capturing accident characteristics, if the collision sensor detects a collision, it is determined that the preset capturing conditions are met; for example, in the shooting scene where the hunting camera is used to capture the morphological characteristics of animals, if the photosensitive sensor detects changes in light, it is determined that the preset capturing conditions are met.

[0069] In this embodiment, before triggering the image acquisition device to start acquiring raw image data, sensor data in the current shooting scene is acquired based on the sensor associated with the image acquisition device, and then it is determined whether the sensor data meets the preset capture conditions, and then it is determined that the image capture instruction is obtained when the conditions are met, and then the image acquisition device is triggered again to start acquiring raw image data; this embodiment improves the accuracy and timeliness of image acquisition by combining the analysis of sensor data and capture conditions, ensures that the image is captured at the appropriate time, and thus improves the real-time performance of shooting.

[0070] In one embodiment, the target image data also includes the first N frames of original image data that have been cached before the image processing parameters are adjusted to convergence, where N is a positive integer greater than or equal to 1.

[0071] For example, after the camera starts shooting, 6 frames of images are captured. When the shooting of the 3rd frame is completed, the image processing parameters are adjusted to convergence. In this case, if N=3, it means that the target image data is the first 3 frames of images collected after the camera starts shooting. For another example, if N=1, it means that the target image data is the 1st frame of image collected after the camera starts shooting.

[0072] In this embodiment, the image processing parameters are adjusted to the first few frames of original image data cached before convergence. The specific number of frames can be adjusted according to actual needs. As long as it is original image data, the condition of shortening the delay can be met, thereby providing shooting users with a variety of options on the basis of improving the real-time performance of shooting.

[0073] In one embodiment, the image processing method includes at least one of image denoising, image color restoration, image white balance adjustment or image exposure control.

[0074] In another embodiment, Figure 2 As shown, an image processing method is provided, comprising the following steps:

[0075] S201, sensor data of the current shooting scene acquired by a sensor associated with an image acquisition device;

[0076] S202, when the sensor data meets the preset capture condition, determining that an image capture instruction is obtained, triggering the image acquisition device to start acquiring raw image data;

[0077] S203, performing image processing on the original image data of the current frame based on the image processing parameters, and outputting a processed image;

[0078] S204, when the image evaluation information of the processed image does not meet the preset requirements, adjusting the image processing parameters to obtain new image processing parameters;

[0079] S205, using the original image data of the next frame as the original image data of the current frame, returning to the step of performing image processing on the original image data of the current frame based on the image processing parameters, and outputting the processed image;

[0080] S206, when the image evaluation information of the processed image meets the preset requirements, determining that the image processing parameters are adjusted to convergence;

[0081] S207, in the process of adjusting the image processing parameters using the original image data, caching the original image data in parallel;

[0082] S208, after the image processing parameters are adjusted to converge, image processing is performed on the target image data based on the converged image processing parameters, and a captured image is output.

[0083] It should be noted that the specific limitations of the above steps can be found in the specific limitations of an image processing method mentioned above, which will not be repeated here.

[0084] like Figure 3 As shown, in the prior art, the ISP (Image Signal Processing) module needs to make corresponding adjustments according to the current environment's brightness, color temperature and other light conditions to output a better image. Therefore, it is necessary to continuously output N frames of image data through the RAW camera and continuously provide them to the ISP module for image encoding processing. The ISP module adjusts its own image parameters according to the first few frames of data provided by the ISC module until the parameters of the ISP image algorithm converge, and then discards the previous N frames of image data. At this time, the N+1th frame processed by the ISP module is the actual output valid picture.

[0085] In this way, the existing technology first allows the ISP image processing parameter curve to converge, and then captures the effective image information, which makes the interval between the first capture time and the output time of the effective image longer, resulting in poor real-time performance of the recorded image. Take a 25FPS (Frames per Second) RAW camera as an example, assuming that the ISP module needs 3 frames of data to converge, then the effective output data frame is the 4th frame. At this time, the theoretical time point of the output image and the start time of the capture action have consumed: 40ms * 4 = 160ms. Therefore, the existing image capture method has a large delay, which is easy to cause the loss of the required key image frames, and the real-time performance of image capture is poor.

[0086] Based on this, the embodiment of the present application provides an image processing method, also known as a real-time image processing method for low-latency shooting, which can be implemented in a real-time image processing system for low-latency shooting, as shown below. Figure 4 and Figure 5 , an image processing method and a real-time image processing system for low-latency shooting are described in detail with a specific embodiment. It is worth noting that the following description is only an exemplary description and not a specific limitation of the application.

[0087] like Figure 4 As shown, the real-time image processing system for low-latency shooting provided by the present application includes: an image acquisition module (such as a RAW camera): acquiring image data; an IMC (image channel) module: parsing image features according to image data; an ISC (image CMOS sensor optical sensor) module / CSI (camera serial interface) module: belonging to the hardware interface module of the camera, which is the core component of the camera module, converting the image signal into a digital signal, adopting a differential signal transmission method, and being able to provide a high-speed image data transmission rate for real-time image acquisition and processing; a cache module: according to the first-in-first-out rule, iteratively outputting the image data to the REP module in sequence as training data for ISP convergence; a REP (replay) recording module: mainly responsible for reading image and video data, and transmitting the data to the ISP or IMC for processing; a first storage module: storing the image data acquired by the image acquisition module before the ISP module converges; a second storage module: storing the valid image data processed by the ISP after the ISP module converges; an ISP (image signal processing) module: used to process the image data output by the front-end image sensor (denoising, color restoration, white balance, exposure control, etc.).

[0088] Based on the above-mentioned real-time image processing system with low latency shooting, the image processing method provided by the present application may include: (1) starting the camera device to acquire image data; (2) when the ISP image algorithm curve has not converged, the first switch is closed, the second and third switches are opened, and the first IMC module outputs the image data to the cache module and the first cache module in two parallel paths respectively, and the cache module outputs the image data to the REP module, and then enters the ISP module to train the ISP module until the ISP image algorithm curve converges; (3) when the ISP image algorithm curve converges, the first switch is opened, the second and third switches are closed, the first IMC module outputs the image data to the first storage module, and then inputs it to the REP module, and then to the ISP module, and then to the IMC module, and finally the processed image data is stored in the second storage module, and the image is output with zero latency, thereby achieving what you shoot is what you get.

[0089] Among them, the ISP process of the present application can be applied to both traditional image signal processing methods (such as filtering, sharpening and other basic functions) and artificial intelligence-based processing methods. As long as there is a parameter setting and adjustment process, the first N frames of images can be used as the basis for adjusting the algorithm parameters.

[0090] More specifically, the image processing method provided by the present application may also include:

[0091] Step S1: According to the pre-stored camera shooting start feature, when the capture conditions are met, the RAW camera is turned on to collect image data frames. The image data frames received by the ISC module / CSI module are output through the IMC module as the first valid image frame (RAW data) after capture.

[0092] Among them, pre-storage refers to a parameter configuration for a specific camera. During the development process, developers will debug parameters for specific products and specific cameras to meet specific application scenarios.

[0093] Step S2: The data frame continues to be output to the first storage module and cache module of the CPU through the ISC module / CSI module and the IMC module.

[0094] Among them, the image data output by the RAW camera in the prior art does not pass through the CPU, but is directly given to the ISP module. The image processing method provided in this application gives the camera data to the first storage module of the CPU for storage, that is, pre-stored backup.

[0095] Step S3: The data frames from the 1st frame to the Nth frame are sent to the ISP module through the REPLAY module via hardware connection.

[0096] Step S4: Let the ISP module perform data statistics training of the image algorithm.

[0097] Step S5: Before the ISP module is stable, these N frames of data will not be processed by the next-level hardware module. After the image processing curve of the ISP module converges, the first frame of data stored previously will be pushed to the REPLAY module for processing by the ISP module, and then sent to the lower-level IMC module for further processing, and finally a valid picture will be output.

[0098] Among them, the next level module refers to imc, jpeg (Joint Photographic Experts Group) processing, etc.

[0099] Among them, the effective picture refers to the output jpeg picture (final output end), including but not limited to jpeg pictures, and refers to the data content that the camera user can obtain. The ISP module in the image method provided in this application can store multiple camera parameter debugging schemes, and each debugging scheme algorithm will output a fixed type of picture. For example, if the filter function is used, the user adjusts a filter, then the photo will only output the photo under this filter.

[0100] Among them, the capture mainly relies on feature points, which include: accident feature capture (driving recorder), radar speed measurement results (road vehicle capture), animal morphological features (hunting camera), for example: according to the image features, key point matching recognition, when the image features meet the preset rules, the RAW camera starts the photo function and quickly captures the current picture. The trigger condition for taking a photo must be other sensors other than the camera, such as the collision sensor detecting a collision, the light sensor detecting a change in light, or the shutter trigger manually pressed by the user.

[0101] The convergence process can be as follows: first, design a model for ISP function training and design several parameters for the model; then, perform image processing on the first N frames of collected data, observe the image processing effect of the model, and calculate the evaluation index value; draw a parameter image (the parameter is the dependent variable and the evaluation index value is the function), iterate the model through the first N frames of data, and update the parameters and parameter image after each iteration until the parameters converge to a certain value, or the parameter image shows a stable trend, then end the parameter adjustment, and the stabilized parameters are used to implement the ISP function.

[0102] The image processing method provided in the present application can be applied to scenarios such as quick capture, shooting, driving recorders, hunting cameras, etc. For example, in the application scenario of a driving recorder, when the device is turned off, when a vehicle accident is detected, the driving recorder can quickly take a photo with zero delay the moment it is turned on; when used in mobile devices, it has high requirements on battery life (because the device cannot be turned on all the time). Some high-end driving recorders also have intelligent recognition functions, which can automatically trigger capture when specific events are detected (such as vehicle collisions, sudden braking of vehicles, etc.), and save images of these critical moments.

[0103] In addition, the quick capture function can also be achieved through voice commands. For example, the user can say commands such as "take a photo" or "snap shot", and the dash cam will immediately capture the current picture and save it.

[0104] For users who need to use the snapshot function frequently, they can optimize the performance by adjusting the dashcam's shooting settings. For example, lower the sensitivity of the gravity sensor and the threshold of the collision level to enable the snapshot function of the camera device.

[0105] like Figure 5 As shown, the image processing method provided by the present application optimizes the image acquisition and image storage algorithms of the RAW camera, stores the data of the first frame image or the previous N frames of images output by the IMC module, waits for the ISP module parameters to converge and stabilize, and then processes the first frame image or the previous N frames of images through the ISP module. This can shorten the time difference between the desired scene image and the actual scene image to a small enough size to be ignored, that is, achieve zero delay between the desired scene image and the actual scene image, thereby achieving what you shoot is what you get.

[0106] In the prior art, images that have been processed by the converged ISP module are considered valid data frames. The data frames processed during the period from the start of the ISP module to the stable convergence are usually dim and green because the data statistics of the algorithm have not been completed. Then, the stable first frame (not the real first frame) is obtained by discarding the earlier data frames.

[0107] The image processing method provided in the present application regards the first frame of image data collected by the RAW camera as a valid data frame, saves the first frame output by the RAW camera (the first frame in the true sense), and simultaneously sends the image data frame to the ISP module for training convergence. After it is stabilized, the saved first frame is sent to the ISP module for processing, thereby achieving what you shoot is what you get.

[0108] The image processing method provided by the present application can be applied to some high-speed moving scenes, and can shorten the time difference between the expected scene image and the actual scene image to be negligible, that is, zero delay is achieved between the expected scene image and the actual scene image. Specifically, the delay on the image data time stream is reduced from 160ms to 200ms to 40ms, that is, the interval from pressing the shutter to capturing the effective picture of the real-time stream is 40ms, quickly capturing the fleeting picture.

[0109] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0111] In one embodiment, Figure 6 As shown, an image processing device is provided, including: an image acquisition module 601, an image buffer module 602, and a captured image output module 603, wherein:

[0112] The image acquisition module 601 is used to trigger the image acquisition device to start acquiring raw image data in response to an image capture instruction;

[0113] The image buffer module 602 is used to buffer the original image data in parallel in the process of adjusting the image processing parameters using the original image data; the image processing parameters are parameters for controlling the image processing method;

[0114] The captured image output module 603 is used to perform image processing on the target image data based on the converged image processing parameters after the image processing parameters are adjusted to convergence, and output the captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to convergence.

[0115] Each module in the above image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0116] In one embodiment, a chip is provided for implementing the steps in the above method embodiments when executing a computer program.

[0117] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0118] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises In response to the image capturing instruction, triggering the image capturing device to start capturing raw image data; In the process of adjusting the image processing parameters using the original image data, the original image data is cached in parallel; the image processing parameters are parameters for controlling the image processing method; After the image processing parameters are adjusted to converge, image processing is performed on target image data based on the converged image processing parameters to output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

2. The method according to claim 1, characterized in that The adjusting the image processing parameters by using the original image data comprises: Based on the image processing parameters, performing image processing on the original image data of the current frame, and outputting a processed image; When the image evaluation information of the processed image does not meet the preset requirements, adjusting the image processing parameters to obtain new image processing parameters; The original image data of the next frame is used as the original image data of the current frame, and the step of performing image processing on the original image data of the current frame based on the image processing parameters and outputting the processed image is returned.

3. The method according to claim 2, characterized in that The method further comprises: When the image evaluation information of the processed image meets the preset requirements, it is determined that the image processing parameters are adjusted to converge.

4. The method according to claim 1, characterized in that Before the step of triggering the image acquisition device to start acquiring raw image data in response to the image capture instruction, the method further includes: Sensor data of the current shooting scene acquired based on the sensor associated with the image acquisition device; When the sensor data meets a preset capture condition, it is determined that the image capture instruction is acquired.

5. The method according to any one of claims 1 to 4, characterized in that: The target image data also includes the first N frames of original image data cached before the image processing parameters are adjusted to converge, where N is a positive integer greater than or equal to 1.

6. The method according to any one of claims 1 to 4, characterized in that: The image processing method includes at least one of image denoising, image color restoration, image white balance adjustment or image exposure control.

7. An image processing device, characterized in that: The device comprises: An image acquisition module, configured to trigger an image acquisition device to start acquiring raw image data in response to an image capture instruction; An image caching module, used to cache the original image data in parallel during the process of adjusting image processing parameters using the original image data; the image processing parameters are parameters for controlling the image processing method; A captured image output module is used to perform image processing on target image data based on the converged image processing parameters after the image processing parameters are adjusted to converge, and output a captured image; the target image data includes the original image data cached before the image processing parameters are adjusted to converge.

8. A chip, which implements the steps of the method according to any one of claims 1 to 6 when executing a computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images