Shooting method and system applied to image acquisition equipment collaborative lighting device

Abstract

The invention relates to a shooting method and system applied to an image acquisition equipment cooperative lighting device in the field of image shooting processing, and the method comprises the steps: controlling the image acquisition equipment to continuously capture multiple frames of images in a target exposure time period, and enabling the multiple frames of images to form a to-be-processed image sequence; the lighting device generates flash lighting when the image acquisition device captures an image, calculates a flash lighting intensity or an illumination contribution index of a coverage area for each frame in the image sequence, screens the image sequence based on at least one of the illumination contribution index and the acquisition time of each frame, and forms a target image set. And carrying out selective output or image synthesis processing on at least one frame of image in the target image set to generate the output image, thereby effectively solving the problem of inaccurate synchronization, not needing accurate hardware-level synchronization, enabling the output image to be superior to any single-frame image in the brightness and coverage area of flash illumination, remarkably reducing the roller shutter door effect, and improving the efficiency of the roller shutter door. And the flash illumination integrity and uniformity of the output image are improved.

Description

Technical Field

[0001] This invention relates to the field of image capture and processing, and in particular to a capture method and system for use with an image acquisition device and a lighting device. Background Technology

[0002] With the rapid development of mobile device photography capabilities, users' demands for image quality are increasing, especially in low-light environments. Built-in flashes on mobile devices often produce harsh and unnatural lighting due to their low power, limited color temperature, and fixed light direction. Therefore, in situations requiring high-quality photography, connecting an external professional flash to the mobile device for supplemental lighting has become an important way to improve image quality.

[0003] In existing technologies, the main methods for enabling mobile terminals to work in conjunction with external lighting devices include: wireless connections such as Bluetooth and Wi-Fi, or wired connections such as hot shoes and synchronization cables, whereby the mobile terminal or a matching trigger sends control commands to the external lighting device, causing the external lighting device to produce flash illumination within the expected exposure period. Compared to traditional professional camera systems that achieve hardware-level synchronization between the mechanical shutter and flash unit through a physical hot shoe interface, mobile terminal camera systems typically use electronic shutters for image acquisition. They lack a precise hardware synchronization mechanism with external lighting devices, and the transmission and execution of trigger commands introduce uncertainties such as communication delays and operating system scheduling jitter, resulting in a non-negligible time deviation between the actual flash time and the image acquisition time.

[0004] Existing methods for using external flashes on mobile devices have significant drawbacks. First, the unavoidable communication delay and synchronization error between the electronic shutter trigger and the flash result in a mismatch between the flash timing and the shutter opening window. More importantly, mobile device image sensors commonly employ rolling shutter technology, which operates by exposing line by line. When capturing the brief flash moment with a high-speed shutter, this line-by-line exposure characteristic leads to the rolling shutter effect: the flash only illuminates a few lines of the sensor being exposed at the moment of impact. The final image only captures a small portion of the area illuminated by the flash, such as a narrow band or half an image, while the rest remains in darkness. This fails to capture the full effect of the flash, severely impacting the image's usability. Summary of the Invention

[0005] To overcome the shortcomings of existing technical solutions, this invention provides a shooting method and system for image acquisition equipment in conjunction with an external lighting device. This method effectively solves the technical problems that occur when a mobile terminal is used in conjunction with an external lighting device for shooting, such as communication delay, synchronization error, and the characteristics of a rolling shutter, resulting in flash illumination only covering a local area of ​​the image, only a small part of the image capturing the flash, and uneven brightness distribution.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A method for capturing images using an image acquisition device in conjunction with a lighting device, characterized by comprising the following steps:

[0008] During at least one target exposure time period, the mobile device is controlled to continuously capture multiple frames of images, which constitute an image sequence to be processed. The illumination device generates flash illumination during the image capture by the mobile device, and at least one frame in the image sequence is captured with flash illumination.

[0009] For each frame in the image sequence, calculate the illumination contribution index of flash illumination intensity and coverage area, with at least one of the flash illumination intensity and illumination contribution index being calculated;

[0010] The image sequence is filtered and a target image set is formed based on at least one of the illumination contribution index and the acquisition time of each frame. The target image set contains at least one bright frame or contains a bright frame and a second bright frame.

[0011] A bright frame is at least one frame that contributes the most to the illumination, and a second bright frame is multiple frames within the time period before and after the bright frame.

[0012] At least one frame from the target image set is selected for output or image synthesis processing to generate the output image.

[0013] Furthermore, the illumination contribution index is used to measure the lighting effect of the image. Its calculation is based on any one or a combination of the following: the average brightness of all pixels in the image, the area or proportion of pixel regions whose brightness exceeds a preset threshold, the integral value of the bright region in the image brightness histogram, and the illumination score output by the machine learning model.

[0014] Furthermore, during the image sequence screening, at least one of the following is used: illumination contribution index and temporal distribution characteristics, to select at least one secondary bright frame.

[0015] Furthermore, the selection of the second brightest frame is achieved through any of the following strategies:

[0016] (a) Taking the acquisition time position of the bright frame as the center, extend a preset time window before and after it, and select the frames with higher light contribution index as the second bright frames in the time window in a time-uniform distribution manner.

[0017] (b) When there are multiple bright frames, the time window is expanded with the acquisition time position of each bright frame as the center, and the frames with higher light contribution index and different temporal distribution are selected as the second bright frames in each time window.

[0018] (c) Sort the multiple frames of images from high to low according to the illumination contribution index, select a number of frames with higher illumination contribution index from the sorting results as a candidate frame set, and remove the bright frames from the candidate frame set, and determine one or more of the remaining frames as the second bright frames.

[0019] Furthermore, when selecting the second brightest frame using strategy (a), the time axis is divided into several sub-intervals of approximately equal length within a preset time window, and frames with higher illumination contribution index are selected as second brightest frames within each sub-interval, so as to achieve uniformly distributed frame selection over time.

[0020] Furthermore, when the images in the target image set are synthesized into an output image, a mixing mode is used that retains the brightest brightness value of each pixel position in each input frame.

[0021] Furthermore, the image synthesis process also includes an early termination mechanism, which involves periodically calculating the similarity between the current synthesized image and the previous synthesized image. When the similarity exceeds a preset threshold, the synthesis process for subsequent frames is terminated.

[0022] Furthermore, after acquiring the multi-frame images and before performing image synthesis processing, edge smoothing preprocessing is performed on the multi-frame images to reduce the jagged edges caused by multi-frame image synthesis.

[0023] An image acquisition system in conjunction with an illumination device, the system comprising:

[0024] An image capture module is used to control an image acquisition device to acquire multiple frames of images within at least one target exposure time period, so that the multiple frames of images constitute an image sequence to be processed, wherein at least some frames in the image sequence contain flash illumination generated by a lighting device in the shooting scene;

[0025] The frame processing module is used to calculate at least one of the illumination contribution indicators that characterize the flash illumination intensity and coverage of each frame in the image sequence, and to filter multiple frames of images based on the illumination contribution indicators or the acquisition time of each frame to determine the target image set. The target image set includes at least one bright frame with a high illumination contribution indicator or includes a bright frame and a second bright frame.

[0026] The image compositing module is used to perform selection output or image compositing processing on the bright frames or bright frames and second-bright frames in the target image set to generate an output image.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] 1. It effectively solves the problem of inaccurate synchronization. It does not require precise hardware-level synchronization. Instead, it acquires multiple frames of images within at least one target exposure time period, calculates the illumination contribution index of each frame, and performs intelligent filtering. As long as the lighting device generates flash illumination in any way within the above time period, the image frame containing flash illumination can be automatically selected in the post-processing stage. This overcomes the synchronization error problems caused by wireless communication delay, trigger link instability, and rolling shutter effect.

[0029] 2. By calculating the illumination contribution of each frame and intelligently selecting the brightest and second-brightest frames representing different exposure times, it supports both directly outputting the brightest frame with the highest illumination contribution index when the target image set contains only a single frame image, and pixel-level synthesis of multiple frames when the target image set contains multiple frames image. This makes the output image superior to any single frame image in terms of flash illumination brightness and coverage area, significantly reducing the rolling shutter effect and improving the flash illumination integrity and uniformity of the output image. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the single-flash multiple shutter speeds and working time axis of the present invention;

[0031] Figure 2 This is a schematic diagram of the multiple shutter speeds and working time axis of the multiple flash lamp of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] A method and system for capturing images using an image acquisition device in conjunction with a lighting device, comprising the following steps:

[0034] In response to the user's shooting command, the lighting device generates flash illumination in the shooting scene within a preset duration, and controls the mobile device to continuously capture multiple frames of images, so that the multiple frames of images constitute an image sequence to be processed. At least some frames in the image sequence contain flash illumination generated by the lighting device in the shooting scene. The flash illumination can be generated by the image acquisition device sending a trigger command to the lighting device through a wired or wireless connection, or it can be generated by the user manually triggering a trial flash on the lighting device.

[0035] For each frame in the multi-frame image, calculate the illumination contribution index that characterizes the flash illumination intensity or coverage of that frame;

[0036] Based on the calculated illumination contribution index or the acquisition time of each frame, the images are filtered from multiple frames to select at least one bright frame with a higher illumination contribution index. According to the preset time distribution rules or illumination contribution index, one or more secondary bright frames are selected from the time periods before and after the bright frame or from the entire shooting time period. The bright frames and secondary bright frames together constitute the target image set.

[0037] At least one frame from the target image set is selected for output or image compositing to generate an output image. When the target image set contains multiple frames and image compositing is performed, the area illuminated by the flash in the output image may have higher integrity than any single frame from the target image set.

[0038] Furthermore, the illumination contribution is an indicator used to measure the illumination effect of an image. It is calculated based on any one or a combination of the following: the average brightness of all pixels in the image, the area of ​​pixel regions whose brightness exceeds a preset threshold, the integral value of the bright areas in the image brightness histogram, and the illumination score of the frame image output based on a machine learning model.

[0039] Average brightness of all pixels: Convert the image to a luminance-chrominance color space, such as YUV, and calculate the arithmetic mean of all pixels in its Y channel, i.e., the luminance channel.

[0040] Highlight area: Set a brightness threshold, for example, a value of 200, with a brightness range of 0-255. Count the total number of pixels in the entire image whose brightness value is greater than this threshold. This number is the highlight area.

[0041] Highlight region integration: Generates a brightness histogram of the image and calculates the sum of the brightness values ​​of all pixels within a specified highlight range, such as 180-255.

[0042] Lighting score based on machine learning model: The frame image is input into a pre-trained machine learning model, such as a convolutional neural network, and the model outputs a score value to characterize the overall lighting quality or flash coverage effect. The score value is used as a light contribution indicator, or is weighted and fused with one or more of the above indicators.

[0043] Combined use: The results of the above two or more calculation bases can be weighted and integrated to form a comprehensive light contribution score.

[0044] Furthermore, one or more sub-bright frames are selected based on illumination contribution indicators or temporal distribution characteristics. Specifically, this can be achieved using any of the following frame selection strategies or a combination thereof:

[0045] (a) Using the time position of the bright frame as the center, extend a time window before and after it. Within this window, select the next brightest frame in a time-uniform distribution. This strategy is suitable for single-flash scenes. Specifically, using the time point of the bright frame as the center, extend a fixed time window before and after it, for example, 200 milliseconds. Then, within this window, calculate several time-uniformly distributed sampling points using a linear interpolation algorithm, and select the frame closest to each sampling point and with a high light contribution as the next brightest frame.

[0046] (b) When multiple bright frames exist, the next brightest frames are selected based on the acquisition time position of each bright frame. This strategy is applicable to multiple flash scenes. Specifically, multiple local peaks that meet the constraints of brightness threshold, minimum time interval, and maximum number are detected on the brightness curve. The frame corresponding to each peak is regarded as a representative frame of a significant flash event. Frames with higher illumination contribution index are selected within the time range before and after each representative frame as the next brightest frames to ensure that a representative frame is included in the target image set for each significant flash.

[0047] (c) Frame selection strategy based on global illumination contribution index: Select several frames with higher illumination contribution indices from the multi-frame images as a candidate frame set. When the number of target frames selected by other strategies is insufficient to meet the preset target frame number, all frames can be sorted globally in descending order according to the illumination contribution index, and frames with higher illumination contribution indices that have not yet entered the candidate set can be added to the candidate set until the number of frames in the candidate set reaches the preset target frame number. For frames in the candidate set, the frame with the highest illumination contribution index can be regarded as a bright frame, and one or more of the remaining frames can be regarded as a less bright frame.

[0048] Furthermore, when using strategy (a), the time-uniform distribution is achieved through a linear interpolation algorithm. The specific implementation steps of the linear interpolation algorithm include:

[0049] Determine the start and end indices of the local time window.

[0050] Calculate the number of secondary bright frames that need to be selected.

[0051] For the i-th frame to be selected, calculate its position ratio:

[0052] positionRatio=i / (remainingSlots+1).

[0053] Calculate the index offset of this frame within the window:

[0054] indexOffset=(int)((endIndex-startIndex)*positionRatio).

[0055] The index of this bright frame was finally determined to be:

[0056] selectedIndex=startIndex+indexOffset.

[0057] Then, frames with higher illumination contribution near `selectedIndex` are selected as the final second brightest frames. The above implementation is only an example; other algorithms that can achieve a more uniform temporal frame selection effect are also applicable.

[0058] Furthermore, when combining images from the target image set into a single output image, a pixel-level blending mode that selects the larger value can be used. At the same pixel location, the pixel values ​​of multiple input frames in the target image set are compared, and the pixel with the larger brightness or pixel value is retained as the pixel value for that location in the output image. For example, in the Android system, PorterDuff.Mode.LIGHTEN is used. During compositing, this mode compares the brightness values ​​of the source and target images pixel by pixel and automatically selects and retains the brighter pixel value as the output.

[0059] Furthermore, an early termination mechanism is implemented during the synthesis process. This mechanism includes periodically comparing the similarity between the current synthesis result and the previous synthesis result, and terminating the synthesis process when the similarity exceeds a preset threshold.

[0060] The complete process for terminating the synthesis includes:

[0061] Triggering timing: During the synthesis process, a detection is triggered after each predetermined number of synthesized items are completed.

[0062] Comparison object: Compare the currently synthesized intermediate result image with the intermediate result image saved when the detection was triggered last time.

[0063] Similarity calculation: Simultaneous pixel sampling is performed on two images, and the RGB Manhattan distance of each pair of sampled pixels is calculated. If the distance is less than a tolerance threshold, the pixels are considered similar. The similarity is the ratio of the number of similar pixels to the total number of sampled pixels.

[0064] Termination judgment: When the similarity exceeds the preset threshold, it is determined that the flash effect has stabilized, and the synthesis operation of all subsequent frames is immediately stopped.

[0065] Furthermore, after capturing multiple frames of images consecutively but before compositing them, edge smoothing preprocessing is performed on the multiple frames to mitigate the jagged edges that may result from multi-frame compositing. This is achieved through image scaling filtering. Specifically, each frame is first reduced to 95% of its original size, and then immediately enlarged back to its original size. This process should enable bilinear filtering and an anti-aliasing flag, utilizing interpolation operations during scaling to achieve a low-pass filtering effect, thereby eliminating jagged edges caused by alignment errors during multi-frame compositing.

[0066] This invention provides a variety of frame selection strategies, including a local time window uniform frame selection strategy centered on bright frames, a frame selection strategy based on global illumination contribution index ranking, and a multi-window combination strategy based on multiple bright frames. These strategies can adapt to various scenarios such as single flash, multiple flashes, and complex brightness changes, and have good versatility and scalability.

[0067] By using a pixel-level "brighter value" synthesis method and optional edge smoothing preprocessing, this invention preserves the details of the bright parts of each frame while suppressing the jagged edges and seams caused by the superposition of multiple frames, thereby improving the integrity of flash coverage while maintaining good image sharpness and visual appeal.

[0068] Example 1:

[0069] Implementation scheme based on standard luminance calculation

[0070] 1. System initialization and parameter settings

[0071] Shooting duration: 1000 milliseconds; Single frame shutter speed: 1 / 100 second; Number of composite target frames: 8 frames; Number of sampling points: 100; Maximum frame buffer size: 100 frames; Similarity threshold: 95%

[0072] 2. Brightness Calculation Process

[0073] The system employs a spatially uniform sampling strategy, selecting 100 sampling points in each frame. The distribution of sampling points is determined by calculating the step size in the horizontal and vertical directions, ensuring that the sampling points are uniformly distributed in the image.

[0074] For each sampling point, the system extracts its RGB channel value. The system uses the international standard brightness perception calculation formula: Brightness = 0.299×R + 0.587×G + 0.114×B. This formula is based on the characteristics of human vision and can accurately reflect the perceived brightness of the image.

[0075] This formula is based on the characteristics of human vision, with the green channel having the highest weight, followed by red, and then blue.

[0076] The system accumulates the brightness values ​​of all sampled points and calculates the average value as the final brightness evaluation value for that frame. If the number of sampled points is zero, the system returns a default brightness value of 0.

[0077] 3. Frame Management and Update Process: The system maintains a frame list to store image frames and their corresponding brightness values. To avoid excessive memory usage, the system only retains the most recent 100 frames. When the frame count reaches the limit, the system automatically removes the oldest frame and reclaims its memory.

[0078] When storing a new frame, the system creates a copy of the bitmap to ensure that the internal data is not modified externally. It also records the frame's luminance value and capture timestamp to inform subsequent frame selection.

[0079] 4. Brightest Frame Dynamic Update Mechanism: The system continuously tracks the current brightest frame and its brightness value. When the brightness value of a new frame exceeds the current maximum value, the brightest frame update process is triggered.

[0080] The update strategy is determined based on the current mode:

[0081] Single-frame mode: Directly use the new frame as the brightest frame.

[0082] Multi-frame mode: Whether to perform frame blending depends on the attenuation parameters.

[0083] In multi-frame mode, the system creates a copy of the brightest frame and draws the new frame using a Canvas, using transparency to control the blending ratio and achieve a smooth transition between frames.

[0084] 5. Intelligent frame selection and synthesis process: The system adopts a three-step process to implement a frame selection strategy based on a local time window:

[0085] Step 1: Determine the selection range

[0086] Centered on the brightest frame, expand forward and backward by a specified number of frames to form a selection window. Ensure that the window boundaries do not exceed the start and end positions of the frame sequence.

[0087] Step 2: Apply the selection strategy

[0088] First, add the brightest frame to the results list. Then, use a linear interpolation algorithm to evenly distribute the positions of the remaining frames within the selection window.

[0089] Calculation formula: Position ratio = i / (Number of remaining slots + 1);

[0090] Index offset = total range × position ratio;

[0091] Ensure that the selected frames are unique and valid.

[0092] Step 3: Brightness sorting optimization. Sort the selected frames in descending order of brightness value, prioritize retaining frames with higher brightness, and limit the number of frames returned in the end to not exceed the maximum value.

[0093] 6. Image Composition Processing

[0094] The compositing process uses the LIGHTEN blending mode, which preserves the maximum brightness value at each pixel location. Specific steps:

[0095] Create an output bitmap with the same size as the input frame. Use a Canvas and a Paint object set to LIGHTEN mode. Draw all selected frames sequentially. Automatically preserve the highest brightness value at each pixel location.

[0096] 7. Similarity detection and early termination mechanism

[0097] The system performs similarity detection during the synthesis process:

[0098] Detection frequency: Once every 5 frames synthesized;

[0099] Sampling strategy: Sample once every 100 pixels;

[0100] Similarity calculation: Compare the brightness difference between the current synthesized result and the previous result;

[0101] Brightness comparison method:

[0102] Extract pixels at the same location from the two images and calculate the pixel brightness value for each.

[0103] Judgment criteria: The brightness of the current synthesized image is greater than 50 and the brightness of the new frame is less than or equal to 1.1 times the brightness of the current image;

[0104] When the similarity of pixels reaches 95%, it is considered that the overall image change has been small as subsequent frames continue to be synthesized, and the flash effect has been basically captured completely, thus terminating the synthesis process of subsequent frames in advance.

[0105] 8. Edge Smoothing Processing: In multi-frame synthesis mode, the system performs edge smoothing preprocessing on the image:

[0106] The image is reduced to 95% of its original size and then enlarged back to its original size. Bilinear filtering and anti-aliasing flags are then applied. This process effectively eliminates edge jaggedness that may occur when combining multiple frames.

[0107] Workflow

[0108] 1. Users start shooting via mobile device APP.

[0109] 2. Initialize system parameters and establish a connection with the external flash.

[0110] 3. Use flash and continuous camera shooting.

[0111] 4. Capture multiple frames of images continuously.

[0112] 5. Perform brightness analysis and management on each frame of the image.

[0113] 6. Dynamically update the brightest frame information.

[0114] 7. Use the intelligent frame selection algorithm to determine the target frame set.

[0115] 8. Perform multi-frame synthesis, and perform similarity detection during the process.

[0116] 9. Output the final composite image.

[0117] Example 2: Top-N frame selection scheme based on global brightness ranking:

[0118] This embodiment is applicable to scenarios where the frame contributing the most brightness is selected from the entire shooting process, rather than being limited to a specific local time period. Its core idea is to directly select the N+1 brightest frames through global comparison to participate in the synthesis, ensuring that the final image is dominated by the frame with the largest brightness contribution.

[0119] Parameter configuration and system initialization: Shooting duration: 800 milliseconds; Single frame shutter speed: 1 / 125 second; Target frame count limit: 6 frames (i.e., N=5); Brightness calculation sampling points: 120; Frame buffer capacity: 80 frames.

[0120] Specific implementation steps:

[0121] Step 1: Image Acquisition and Brightness Calculation

[0122] The system controls the mobile device to continuously capture images within 800 milliseconds, while simultaneously triggering the external flash. For each frame, a uniform sampling strategy is used to select 120 sampling points. The standard brightness calculation formula (0.299R + 0.587G + 0.114B) is applied to calculate the brightness value of each sampling point, and the average value is taken as the brightness score for that frame.

[0123] Step 2: Global Sorting and Frame Selection

[0124] 1. The system globally sorts all frames according to their brightness values ​​from high to low;

[0125] 2. Select the first 6 frames (i.e., Top-N+1 frames) from the sorting results to form a preliminary target set;

[0126] 3. Rearrange these selected frames according to their original shooting time order;

[0127] 4. Ensure that the final output frame sequence includes both the brightest frames and maintains the chronological order.

[0128] Step 3: Multi-frame synthesis processing

[0129] The LIGHTEN blending mode is used for compositing. This mode automatically compares the brightness values ​​of each pixel location across all input frames and retains the maximum value. During compositing, the system performs similarity detection:

[0130] Detection frequency: Once every 3 frames synthesized;

[0131] Similarity threshold: 96%;

[0132] Sampling interval: Sample once every 8 pixels;

[0133] When the synthesis results are detected to be stabilizing, the synthesis process is terminated early to optimize processing efficiency.

[0134] It ensures that the selected frame represents the moment with the highest brightness during the entire shooting process. The algorithm is simple and efficient with low computational complexity, and is suitable for scenes with relatively uniform flash effect distribution.

[0135] Example 3: A combined frame selection scheme based on multi-peak detection and global Top-N fallback:

[0136] This embodiment is specifically designed for complex scenarios where an external flash fires multiple times within a given duration. By combining multi-peak detection and global sorting mechanisms, it ensures that each significant flash has a representative frame, while also providing a fallback mechanism to guarantee a sufficient number of frames.

[0137] Parameter configuration and system initialization: Shooting duration: 1200 milliseconds;

[0138] Peak detection parameters:

[0139] Peak threshold factor 0.6; minimum peak interval 5 frames; maximum number of peaks 4; target total number of frames 8; brightness threshold 180.

[0140] Specific implementation steps:

[0141] Step 1: Multi-peak detection stage

[0142] 1. The system constructs a brightness variation curve and calculates the global maximum brightness value;

[0143] 2. Set peak detection threshold: global maximum brightness × 0.6;

[0144] 3. Scan local peaks in the frame sequence that meet the following conditions: the brightness of the current frame is greater than that of the previous frame and not less than that of the next frame; the brightness value exceeds the peak detection threshold; the time interval between the current frame and the detected peak is greater than 5 frames; and the total number of peaks does not exceed 4.

[0145] 4. Add all frames corresponding to the detected peaks to the candidate set.

[0146] Step 2: Cubble Frame Interpolation Mechanism

[0147] 1. Check if the number of frames in the candidate set meets the target requirement;

[0148] 2. If insufficient, initiate global Top-N sorting: Sort all frames in descending order of brightness value, and select unselected frames from high to low until the candidate set reaches 8 frames or all frames have been traversed.

[0149] 3. Arrange the final candidate set in chronological order of shooting time.

[0150] Step 3: Adaptive Synthesis Processing

[0151] 1. Perform edge smoothing preprocessing on candidate frames;

[0152] 2. A progressive synthesis strategy is adopted: frames selected by peak detection are synthesized first, and then frames supplemented by the fallback mechanism are synthesized in sequence.

[0153] 3. Dynamically adjust similarity detection parameters: detect once every 4 frames in the first 50% of the synthesis process, and once every 2 frames in the second 50% of the synthesis process;

[0154] 4. The synthesis will be terminated when the overall similarity reaches 95%.

[0155] It can effectively handle multiple flash scenes, ensuring that each important flash event has a representative frame. It avoids insufficient frame count due to missed peak detection through a fallback mechanism. The adaptive compositing strategy optimizes processing efficiency and is particularly suitable for complex flash environments such as stage photography and sports photography.

[0156] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A shooting method applied to an image acquisition device in conjunction with an illumination device, characterized in that, Includes the following steps: During at least one target exposure time period, the image acquisition device is controlled to continuously capture multiple frames of images, which constitute an image sequence to be processed. The illumination device generates flash illumination during the image acquisition device's image capture, and at least one frame in the image sequence is captured with flash illumination. For each frame in the image sequence, calculate the illumination contribution index of flash illumination intensity or coverage area, and at least one of the flash illumination intensity and illumination contribution index shall be calculated; The image sequence is filtered and a target image set is formed based on at least one of the illumination contribution index and the acquisition time of each frame. The target image set contains at least one bright frame or contains a bright frame and a second bright frame. A bright frame is at least one frame that contributes the most to the illumination, and a second bright frame is multiple frames within the time period before and after the bright frame. At least one frame from the target image set is selected for output or image synthesis processing to generate the output image.

2. The shooting method applied to an image acquisition device in conjunction with an illumination device according to claim 1, characterized in that: The illumination contribution index is used to measure the lighting effect of an image. It is calculated based on any one or a combination of the following: the average brightness of all pixels in the image, the area or proportion of pixel regions whose brightness exceeds a preset threshold, the integral value of the bright areas in the image brightness histogram, and the illumination score output by a machine learning model.

3. A shooting method for an image acquisition device in conjunction with an illumination device according to claim 1 or 2, characterized in that: When filtering the image sequence, at least one of the following is used: illumination contribution index and temporal distribution characteristics, to select at least one secondary bright frame.

4. The shooting method applied to an image acquisition device in conjunction with an illumination device according to claim 3, characterized in that: The selection of the second brightest frame is achieved through any of the following strategies: (a) Taking the acquisition time position of the bright frame as the center, extend a preset time window before and after it, and select the frames with higher light contribution index as the second bright frames in the time window in a time-uniform distribution manner. (b) When there are multiple bright frames, the time window is expanded with the acquisition time position of each bright frame as the center, and the frames with higher light contribution index and different temporal distribution are selected as the second bright frames in each time window. (c) Sort the multiple frames of images from high to low according to the illumination contribution index, select a number of frames with higher illumination contribution index from the sorting results as a candidate frame set, and remove the bright frames from the candidate frame set, and determine one or more of the remaining frames as the second bright frames.

5. A shooting method for an image acquisition device with a coordinated lighting device according to claim 4, characterized in that: When using strategy (a) to select the second brightest frame, the time axis is divided into several sub-intervals of approximately equal length within a preset time window, and the frame with higher illumination contribution index is selected as the second brightest frame in each sub-interval, so as to achieve frame selection that is evenly distributed over time.

6. The shooting method applied to an image acquisition device in conjunction with an illumination device according to claim 1, characterized in that: When the images in the target image set are combined into an output image, a blending mode is used that retains the brightest brightness value of each pixel position in each input frame.

7. A shooting method for an image acquisition device in conjunction with a lighting device according to claim 6, characterized in that: The image synthesis process also includes an early termination mechanism, which involves periodically calculating the similarity between the current synthesized image and the previous synthesized image. When the similarity exceeds a preset threshold, the synthesis process for subsequent frames is terminated.

8. A shooting method for an image acquisition device with a coordinated lighting device according to claim 1 or 6, characterized in that: After acquiring the multi-frame images and before performing image synthesis processing, edge smoothing preprocessing is performed on the multi-frame images to reduce the jagged edges caused by multi-frame image synthesis.

9. A shooting system for an image acquisition device and a lighting device, used in the shooting method for an image acquisition device and a lighting device as described in any one of claims 1-7, characterized in that, The system includes: An image capture module is used to control an image acquisition device to acquire multiple frames of images within at least one target exposure time period, so that the multiple frames of images constitute an image sequence to be processed, wherein at least some frames in the image sequence contain flash illumination generated by a lighting device in the shooting scene; The frame processing module is used to calculate at least one of the illumination contribution indicators that characterize the flash illumination intensity and coverage of each frame in the image sequence, and to filter multiple frames of images based on the illumination contribution indicators or the acquisition time of each frame to determine the target image set. The target image set includes at least one bright frame with a high illumination contribution indicator or includes a bright frame and a second bright frame. The image compositing module is used to perform selection output or image compositing processing on the bright frames or bright frames and second-bright frames in the target image set to generate an output image.

Images