Multi-sensor fusion shooting method and device and computer readable storage medium

Abstract

The invention discloses a multi-sensor fusion shooting method and device and a computer readable storage medium, and the method comprises the steps: synchronously collecting the data of a plurality of sensors, and carrying out the dynamic weight fusion calculation, so as to determine a current shooting scene; calling an imaging strategy corresponding to the shooting scene, and carrying out collaborative smooth transition on a plurality of parameters in the imaging strategy to determine a current shooting parameter; according to the method, the recognition accuracy of the complex mixed scene can be improved and the misjudgment rate can be reduced without adding special hardware, so that no picture mutation occurs during shooting parameter switching, and the shooting experience satisfaction degree of a user is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multi-sensor fusion imaging method, device, and computer-readable storage medium. Background Technology

[0002] In existing technologies, the photography technology of mobile phones and other smart devices has defects such as high scene recognition misjudgment rate, insufficient sensor data fusion, abrupt switching of imaging parameters, poor rule library adaptability, and dependence on special hardware. In complex scenarios such as low light + motion, backlight + motion, and glass reflection, it is difficult to achieve accurate scene recognition and optimal imaging effect, resulting in a poor user shooting experience. Summary of the Invention

[0003] In order to overcome the shortcomings of the prior art, the present invention aims to provide a multi-sensor fusion imaging method, device and computer-readable storage medium to solve the technical problems of low accuracy in complex scene recognition, poor adaptability to dynamic scenes, fragmented experience in switching imaging strategies, insufficient scene adaptability and limited versatility.

[0004] This invention proposes a multi-sensor fusion imaging method, which includes: Simultaneously collect data from multiple sensors and perform dynamic weighted fusion calculations to determine the current shooting scene; An imaging strategy corresponding to the shooting scene is invoked, and multiple parameters in the imaging strategy are smoothly transitioned in a coordinated manner to determine the current shooting parameters.

[0005] Optionally, the simultaneous acquisition of data from multiple sensors includes, prior to: During the initialization phase, the preset rule base and mapping table are loaded; Activate multiple of the aforementioned sensors and calibrate the synchronization clock.

[0006] Optionally, the simultaneous acquisition of data from multiple sensors specifically includes: Illumination data, motion data, and positioning data are collected by an ambient light sensor, a three-axis gyroscope, and a positioning module, respectively. A unified timestamp is added to the illumination data, the motion data, and the positioning data to align the 3D data.

[0007] Optionally, the simultaneous acquisition of data from multiple sensors then includes: The illumination data, motion data, and positioning data are processed using moving average filtering, Kalman filtering, and the Raida criterion, respectively. Generate illumination data L, motion data G, and positioning data V after data normalization.

[0008] Optionally, the dynamic weighted fusion calculation specifically includes: Call the preset weighting coefficients α, β, γ; The fusion feature value F is calculated according to the formula F=α×L+β×G+γ×V, and the shooting scene is determined based on the fusion feature value and the rule base.

[0009] Optionally, the invocation of the imaging strategy corresponding to the shooting scene specifically includes: Call the mapping table; The mapping table determines the imaging strategy corresponding to the shooting scene, as well as a plurality of parameters in the imaging strategy.

[0010] Optionally, the coordinated smooth transition of multiple parameters in the imaging strategy specifically includes: Calculate the differences between the multiple parameters and the preset target parameters; The shooting parameters are obtained by gradually changing the difference within a preset time using a linear interpolation algorithm.

[0011] Optionally, the method further includes: During the shooting process, user operation data is collected; The rule base is optimized and updated based on the operational data.

[0012] The present invention also proposes a multi-sensor fusion imaging device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the multi-sensor fusion imaging method as described in any of the preceding claims.

[0013] The present invention also proposes a computer-readable storage medium storing a multi-sensor fusion imaging program, wherein when the multi-sensor fusion imaging program is executed by a processor, the steps of the multi-sensor fusion imaging method as described in any of the preceding claims are implemented.

[0014] The multi-sensor fusion shooting method, device, and computer-readable storage medium of the present invention simultaneously collect data from multiple sensors and perform dynamic weighted fusion calculations to determine the current shooting scene; invoke an imaging strategy corresponding to the shooting scene, and perform a coordinated and smooth transition on multiple parameters in the imaging strategy to determine the current shooting parameters; thereby improving the accuracy of complex mixed scene recognition and reducing the false judgment rate without adding special hardware, ensuring no sudden changes in the image when switching shooting parameters, and improving user shooting experience satisfaction. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart of the first embodiment of the multi-sensor fusion imaging method of the present invention; Figure 2 This is a flowchart of the second embodiment of the multi-sensor fusion shooting method of the present invention; Figure 3 This is a flowchart of the third embodiment of the multi-sensor fusion imaging method of the present invention; Figure 4 This is a flowchart of the fourth embodiment of the multi-sensor fusion shooting method of the present invention; Figure 5 This is a flowchart of the fifth embodiment of the multi-sensor fusion imaging method of the present invention; Figure 6 This is a flowchart of the sixth embodiment of the multi-sensor fusion imaging method of the present invention; Figure 7 This is a flowchart of the seventh embodiment of the multi-sensor fusion imaging method of the present invention; Figure 8 This is a flowchart of the eighth embodiment of the multi-sensor fusion imaging method of the present invention; Figure 9 This is a schematic diagram illustrating the working principle of the multi-sensor fusion imaging method of the present invention; Figure 10 This is a flowchart of the multi-sensor fusion imaging method of the present invention. Detailed Implementation

[0016] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0018] Example 1 Figure 1 This is a flowchart of the first embodiment of the multi-sensor fusion imaging method of the present invention. A multi-sensor fusion imaging method, the method comprising: S1. Simultaneously collect data from multiple sensors and perform dynamic weighted fusion calculations to determine the current shooting scene; S2. Invoke the imaging strategy corresponding to the shooting scene, and perform a smooth transition on multiple parameters in the imaging strategy to determine the current shooting parameters.

[0019] Please refer to Figure 9The diagram illustrates the connections and signal logic between modules. In this embodiment, the modules involved include a data acquisition unit, a data processing unit, an imaging control unit, and a storage unit. The various units communicate with each other via a high-speed bus.

[0020] In this embodiment, the data acquisition unit includes an ambient light sensor, a three-axis gyroscope, and a positioning module, used to synchronously acquire light intensity, attitude angle, angular velocity, and movement speed data, and add timestamps; specifically, the unit uses a built-in general-purpose sensor from a mobile phone or other terminal and supports I2C / SPI interfaces; furthermore, each sensor is synchronously triggered and driven, and the timestamp accuracy is ≤10ms.

[0021] In this embodiment, the data processing unit includes a dedicated ISP chip, an NPU processor, and memory, used for data preprocessing, dynamic weighted fusion, scene classification, and rule base self-learning. Specifically, the ISP of this unit is responsible for image data processing, and the NPU accelerates AI algorithms. Furthermore, this unit applies Kalman filtering, weighted fusion algorithms, and gradient descent optimization procedures.

[0022] In this embodiment, the imaging control unit includes a camera module driver and a parameter control interface, used to receive scene classification results, switch imaging strategies, and dynamically adjust shooting parameters; specifically, the camera module of the unit includes a CMOS sensor and a lens driver, and the unit applies a strategy mapping table and a parameter interpolation adjustment program.

[0023] In this embodiment, the storage unit includes flash memory and cache, used to store the rule base, policy mapping table, weight configuration file, and self-learning dataset; specifically, the unit can be the built-in flash memory of a terminal such as a mobile phone, and the unit includes a data encryption storage module to support real-time updates of the rule base.

[0024] In this embodiment, a dynamic weighted fusion model of three-dimensional sensors is created. Specifically, on the one hand, multi-dimensional data from ambient light sensors, three-axis gyroscopes, and GPS modules are fused and synchronized through timestamps to solve the one-sidedness of single sensor data. On the other hand, a dynamic weighted fusion algorithm (F=α×L+β×G+γ×V) is designed, with the weight coefficients adaptively adjusted according to the scene type to achieve accurate feature extraction in mixed scenes.

[0025] In this embodiment, a dual-layer judgment rule base and a self-learning mechanism are designed. Specifically, on the one hand, a dual-layer judgment logic of single sensor threshold and fused feature value is constructed to avoid misjudgment from a single dimension. On the other hand, a gradient descent algorithm is introduced to optimize the rule base. By collecting user manual correction operations, the threshold and weight are dynamically adjusted to improve the adaptability to complex scenarios.

[0026] In this embodiment, a multi-parameter collaborative smooth transition mechanism is adopted. Specifically, on the one hand, a parameter adjustment time window is established, and a linear interpolation algorithm is used to achieve gradual switching of parameters such as ISO, shutter speed, and focus frequency. On the other hand, core parameters and auxiliary parameters are defined. In mixed scenes, the core parameters are given priority, while the auxiliary parameters change synchronously and gradually to avoid sudden changes in the image.

[0027] In this embodiment, a dynamic mapping relationship between scenes and strategies is established. Specifically, on the one hand, dedicated imaging strategies are designed for various complex scenes, rather than general optimizations. On the other hand, a priority mechanism is established in mixed scenes to adapt to the needs of core scenes while taking into account the optimization of secondary scenes.

[0028] Specifically, in this embodiment, firstly, data from multiple sensors are simultaneously collected and dynamically weighted fusion calculations are performed to determine the current shooting scene; then, an imaging strategy corresponding to the shooting scene is invoked, and multiple parameters in the imaging strategy are collaboratively and smoothly transitioned to determine the current shooting parameters; furthermore, during this process, a dynamic weighted fusion model is constructed by fusing multi-dimensional data from an ambient light sensor, a three-axis gyroscope, and a GPS module; a two-layer judgment rule base of single-sensor thresholds and fused feature values ​​is established, and a gradient descent self-learning mechanism is introduced; a multi-parameter collaborative smooth transition mechanism is designed, and a linear interpolation algorithm is used to achieve parameter gradual change; a dynamic mapping relationship between the scene and the strategy is constructed, and a dedicated imaging strategy is designed for complex scenes, and a hybrid scene priority mechanism is established.

[0029] The beneficial effects of this embodiment are that by synchronously collecting data from multiple sensors and performing dynamic weighted fusion calculations, the current shooting scene is determined; the imaging strategy corresponding to the shooting scene is invoked, and multiple parameters in the imaging strategy are smoothly transitioned in a coordinated manner to determine the current shooting parameters; thus, the accuracy of complex mixed scene recognition and the false judgment rate can be improved without adding special hardware, so that there is no sudden change in the picture when switching shooting parameters, and the user shooting experience satisfaction is improved.

[0030] Example 2 Figure 2 This is a flowchart of the second embodiment of the multi-sensor fusion imaging method of the present invention. Based on the above embodiment, the step of simultaneously acquiring data from multiple sensors includes: S01. During the initialization phase, load the preset rule base and mapping table; S02. Start multiple sensors and calibrate the synchronization clock.

[0031] In this embodiment, considering that existing technologies often rely on data from a single sensor for scene determination, such as identifying weak light and backlight solely through an ambient light sensor, or determining motion state solely through a gyroscope, resulting in a high rate of scene misjudgment; this embodiment, however, loads the initial rule base (including scene thresholds and initial values ​​of weight coefficients) and scene-policy mapping table using the aforementioned storage unit during the initialization phase, starts the sensor using the data acquisition unit, calibrates the synchronization clock (timestamp synchronization accuracy ≤ 10ms), and initializes the camera module parameters (e.g., default ISO 100, shutter speed 1 / 60s, focus frequency 10fps) using the imaging control unit.

[0032] The beneficial effect of this embodiment is that by loading a preset rule base and mapping table during the initialization phase, starting multiple sensors, and calibrating the synchronization clock, the problem of the one-sidedness of data from a single sensor is solved.

[0033] Example 3 Figure 3 This is a flowchart of the third embodiment of the multi-sensor fusion imaging method of the present invention. Based on the above embodiment, the simultaneous acquisition of data from multiple sensors specifically includes: S11. Acquire illumination data, motion data, and positioning data respectively through an ambient light sensor, a three-axis gyroscope, and a positioning module; S12. Add a unified timestamp to the illumination data, the motion data, and the positioning data to align the three-dimensional data.

[0034] In this embodiment, considering the existing technology, in low-light scenes: relying solely on light intensity data, for example, it is easy to misjudge low-light and slightly moving scenes as purely low-light, resulting in blurred images when using long exposure strategies; for example, in backlight scenes, without considering the motion state, when backlight is accompanied by object movement, traditional HDR multi-frame fusion will produce motion blur due to inter-frame displacement; for example, in glass reflection scenes: lacking dedicated recognition sensors and judgment logic, existing technologies often misjudge them as strong light scenes, and simply reducing exposure compensation cannot effectively eliminate reflections; while this embodiment integrates multi-dimensional data from the aforementioned ambient light sensor, three-axis gyroscope, and GPS module, and synchronizes them through timestamps, thereby solving the problem of the one-sidedness of single sensor data.

[0035] Furthermore, in this embodiment, a barometric pressure sensor can be used to replace the GPS module, and the rate of change of barometric pressure can be used to assist in determining the movement status (suitable for scenarios without GPS signal, such as indoor environments); this embodiment can also introduce a Hall sensor to detect whether the phone is held in hand and adjust the gyroscope weight (for example, the gyroscope weight is increased when the phone is held in hand and decreased when the phone is fixed on a stand); it should be noted that the advantage of using the above-mentioned GPS positioning module is that the movement speed data is directly related to the dynamics of the shooting scene, and it has greater versatility, while this alternative solution is suitable for supplementing special scenarios.

[0036] It is easy to see that this embodiment only relies on the phone's built-in general sensors (ambient light, gyroscope, GPS), without the need for special hardware such as ToF, and can be deployed on phones of all price ranges; at the same time, the software algorithm is designed to be lightweight and does not consume additional computing resources.

[0037] The beneficial effect of this embodiment is that it collects illumination data, motion data, and positioning data through an ambient light sensor, a three-axis gyroscope, and a positioning module, respectively; it adds a unified timestamp to the illumination data, the motion data, and the positioning data to align the three-dimensional data; and thus constructs a dynamic weighted fusion model by fusing the multi-dimensional data from the ambient light sensor, the three-axis gyroscope, and the GPS module.

[0038] Example 4 Figure 4 This is a flowchart of the fourth embodiment of the multi-sensor fusion imaging method of the present invention. Based on the above embodiment, the step of simultaneously acquiring data from multiple sensors includes: S13. The illumination data, motion data, and positioning data are processed using moving average filtering, Kalman filtering, and the Raida criterion, respectively. S14. Generate illumination data L, motion data G, and positioning data V after data normalization.

[0039] In this embodiment, considering that sensor data in the prior art has not achieved deep fusion, although attempts have been made to use dual-sensor collaboration (e.g., ambient light and gyroscope), only a simple "logical OR / AND" judgment method is used, without establishing a data fusion model or considering the correlation of sensor data (the complementarity between GPS movement speed and gyroscope angular velocity). This results in a one-sided judgment of motion scenes (when shooting moving objects from a stationary position, GPS speed is 0, and relying solely on the gyroscope easily leads to missed detections). Furthermore, the lack of a dynamic weight adjustment mechanism makes it unable to prioritize the imaging needs of the core scene in mixed scenes (backlighting and motion). In contrast, this embodiment uses a Kalman filter algorithm for gyroscope data to eliminate random noise caused by hand tremors, outputting smoothed angular velocity data Gfiltered; for ambient light data, a moving average filter (window size 3-5 frames) is used to smooth high-frequency fluctuations such as light flickering, outputting Lfiltered; and for GPS data, a 3σ filter is used. The criteria remove outliers (such as speed jumps caused by tunnel positioning deviations) and output Vvalid; further, data normalization is adopted: Lfiltered, Gfiltered, and Vvalid are mapped to the 0-1 interval to obtain L, G, and V.

[0040] The beneficial effect of this embodiment is that by processing the illumination data, motion data, and positioning data using moving average filtering, Kalman filtering, and the Raida criterion respectively, illumination data L, motion data G, and positioning data V after data normalization are generated; thereby providing a data foundation for the subsequent two-layer judgment logic that combines single sensor thresholds and fused feature values, and avoiding misjudgment from a single dimension.

[0041] Example 5

[0042] Figure 5 This is a flowchart of the fifth embodiment of the multi-sensor fusion imaging method of the present invention. Based on the above embodiment, the dynamic weighted fusion calculation specifically includes: S15. Call the preset weight coefficients α, β, γ; S16. Calculate the fusion feature value F according to the formula F=α×L+β×G+γ×V, and determine the shooting scene according to the fusion feature value and the rule base.

[0043] In this embodiment, on the one hand, a two-layer judgment logic combining single sensor thresholds and fused feature values ​​is constructed to avoid misjudgment based on a single dimension. For example, in low-light scenes, L < 5 lux and F < 0.3 must be satisfied simultaneously. On the other hand, a gradient descent algorithm is introduced to optimize the rule base. By collecting user manual correction operations, such as the system misjudging glass reflection as strong light and the user manually enabling the reflection elimination function, the threshold and weight are dynamically adjusted to improve the adaptability to complex scenes.

[0044] Specifically, in this embodiment, the data processing unit adjusts the weighting coefficients α, β, and γ based on the current preliminary scenario prediction.

[0045] Among them, the prediction of weak light is: α=0.6, β=0.15, γ=0.15; the prediction of motion is: α=0.25, β=0.5, γ=0.25; substituting into the formula F=α×L+β×G+γ×V, the fusion feature value F is calculated.

[0046] Specifically, in this embodiment, the first layer of the two-layer scene classification is single-sensor threshold determination (screening candidate scenes), and the second layer is feature value fusion determination (determining the target scene). The specific logic is as follows:

[0047] Furthermore, as shown in the table above, when the candidate scene is in low light, α=0.5-0.7, β=0.1-0.2, and γ=0.1-0.2; when the candidate scene is in motion, α=0.2-0.3, β=0.4-0.6, and γ=0.1-0.2; when the candidate scene is in backlight, α=0.3-0.5, β=0.2-0.3, and γ=0.1-0.2; and when the candidate scene is in reflective light, α=0.3-0.5, β=0.2-0.3, and γ=0.2-0.3.

[0048] Furthermore, in this embodiment, a neural network fusion model (such as CNN) can be used to replace the linear weighted algorithm to optimize the data fusion accuracy through training samples; furthermore, a fusion algorithm based on fuzzy logic can be used to map sensor data into fuzzy subsets (such as "strong light", "weak light", "high speed motion"), and calculate fusion feature values ​​through fuzzy inference; it should be noted that the advantage of the above linear weighted algorithm is that the amount of computation is small (processing latency ≤100ms), thus it is suitable for the computing power of mobile devices, while the above alternative is suitable for high-end models and can improve accuracy through NPU acceleration.

[0049] The beneficial effect of this embodiment is that by calling preset weight coefficients α, β, and γ; calculating the fusion feature value F according to the formula F=α×L+β×G+γ×V; and determining the shooting scene according to the fusion feature value and the rule base; thus, it can flexibly adapt to different user shooting habits and new complex scenes.

[0050] Example 6 Figure 6 This is a flowchart of the sixth embodiment of the multi-sensor fusion shooting method of the present invention. Based on the above embodiment, the step of calling the imaging strategy corresponding to the shooting scene specifically includes: S21. Call the mapping table; S22. Determine the imaging strategy corresponding to the shooting scene in the mapping table, as well as a plurality of parameters in the imaging strategy.

[0051] In this embodiment, the imaging control unit queries the scene-strategy mapping table and invokes the corresponding imaging strategy; specifically:

[0052] It is clear that this embodiment designs specific imaging strategies for the four types of complex scenarios mentioned above (e.g., "polarization filtering + multi-angle exposure fusion" for glass reflection scenarios), rather than general optimization. Simultaneously, a priority mechanism is established in mixed scenarios (e.g., motion priority is higher than backlight), adapting to the core scenario requirements while also considering secondary scenario optimization (e.g., in motion + backlight scenarios, a focus tracking strategy + light HDR fusion is used). In other words, by utilizing dynamic weights and priority mechanisms, core requirements are prioritized in mixed scenarios (such as backlight + motion, low light + glass reflection), resulting in image quality superior to the compromise solutions of existing technologies.

[0053] Furthermore, the above mapping table can be extended to add new judgment rules and imaging strategies for new scenes (such as starry sky, macro reflection).

[0054] The beneficial effect of this embodiment is that by calling the mapping table, the imaging strategy corresponding to the shooting scene and multiple parameters in the imaging strategy are determined in the mapping table, thereby using a dedicated scene strategy to specifically solve core problems, such as multi-angle exposure fusion in glass reflection scenes, which improves the reflection elimination rate.

[0055] Example 7 Figure 7 This is a flowchart of the seventh embodiment of the multi-sensor fusion imaging method of the present invention. Based on the above embodiment, the step of performing a coordinated and smooth transition on multiple parameters in the imaging strategy specifically includes: S23. Calculate the difference between the multiple parameters and the preset target parameters; S24. The shooting parameters are obtained by performing a gradual transition based on the difference within a preset time using a linear interpolation algorithm.

[0056] In this embodiment, considering that in the prior art, when scene switching occurs, the imaging parameters are directly switched (such as suddenly switching from low-light multi-frame synthesis to motion tracking focus), which leads to sudden changes in image brightness and sharpness, affecting the user's shooting experience, and the adjustment of multiple parameters is not synchronized (such as the shutter speed has been switched, but the ISO remains at a high value), resulting in overexposure or underexposure; this embodiment calculates the difference between the current parameter and the target parameter, and completes the gradual change in a short time through a linear interpolation algorithm; and in mixed scenes, the core parameters are adjusted first. For example, in motion and backlight, the shutter speed is switched to 1 / 500s first, and then the exposure compensation is gradually adjusted.

[0057] In this embodiment, a multi-parameter collaborative smooth transition mechanism is established. Specifically, on the one hand, a parameter adjustment time window (0.3-0.5s) is established, and a linear interpolation algorithm is used to achieve gradual switching of parameters such as ISO, shutter speed, and focus frequency. On the other hand, core parameters and auxiliary parameters are defined. In mixed scenes, core parameters are given priority (such as adjusting shutter speed first in motion scenes), while auxiliary parameters change gradually in sync to avoid sudden changes in the image.

[0058] Furthermore, exponential interpolation can be used instead of linear interpolation, with parameter adjustments changing rapidly in the early stages and gradually slowing down in the later stages, which better matches the visual adaptation characteristics of the human eye. Furthermore, the transition time can be dynamically adjusted based on the scene change rate (e.g., the transition time is shortened to 0.2s when the scene changes abruptly, and extended to 0.5s when it changes gradually). It should be noted that the advantages of the above linear interpolation algorithm compared to the alternative are its simplicity of implementation and better parameter synchronization.

[0059] The beneficial effect of this embodiment is that by calculating the difference between multiple parameters and the preset target parameters, and by using a linear interpolation algorithm to perform gradual processing according to the difference within a preset time period, the shooting parameters after a smooth transition are obtained; thereby enabling a smooth transition of parameters, avoiding abrupt changes in the image, and improving the user's shooting experience.

[0060] Example 8 Figure 8 This is a flowchart of the eighth embodiment of the multi-sensor fusion imaging method of the present invention. Based on the above embodiment, the method further includes: S31. During the shooting process, collect user operation data; S32. Optimize and update the rule base based on the operation data.

[0061] In this embodiment, a mechanism combining a dual-layer judgment rule base and self-learning is constructed. On the one hand, a dual-layer judgment logic combining single sensor thresholds and fused feature values ​​is constructed to avoid misjudgment based on a single dimension (for example, in low-light scenes, L < 5 lux and F < 0.3 must be satisfied simultaneously). On the other hand, a gradient descent algorithm is introduced to optimize the rule base. By collecting user manual correction operations (for example, the system misjudges glass reflection as strong light, and the user manually enables the reflection elimination function), the threshold and weight are dynamically adjusted to improve the adaptability to complex scenes.

[0062] Specifically, in this embodiment, the data processing unit collects user operation data: on the one hand, if the system determines that the scene is inconsistent with the scene manually switched by the user, for example, if the system determines that the light is weak and the user manually switches to the anti-reflection mode, the sensor data and fusion feature values ​​under this scene are recorded; on the other hand, the gradient descent algorithm is used to optimize the threshold and weight coefficient of the rule base, for example, adjusting the F peak interval threshold of the glass reflection scene; further, the optimized data is stored in the storage unit for automatic loading when shooting is started next time.

[0063] The beneficial effect of this embodiment is that by collecting user operation data during the shooting process, optimizing and updating the rule base based on the operation data, and thus using a self-learning mechanism to adapt to different user habits and new scenarios, the misjudgment rate is greatly reduced after long-term use.

[0064] Please refer to Figure 10 In another specific example, firstly, the system initializes, loads the rule base and mapping table, and performs multi-dimensional data synchronous acquisition. Then, it performs data preprocessing, dynamic weighted fusion calculation, and two-layer scene classification. For mixed scenes, the core scene strategy is matched according to priority; for single scenes, the single scene strategy is matched. Next, a multi-parameter collaborative smooth transition is performed, enabling the camera module to execute the strategy and capture images. Finally, based on user operation data, the database is optimized using the gradient descent algorithm, and the updated rule base is stored. It should be noted that the above multi-dimensional data acquisition and rule base update steps are executed cyclically, meaning an update is performed with each shot, thus adapting to different user habits and new scenarios. Over long-term use, this significantly reduces the misjudgment rate.

[0065] Example 9 Based on the above embodiments, the present invention also proposes a multi-sensor fusion imaging device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the multi-sensor fusion imaging method as described in any of the above embodiments.

[0066] It should be noted that the above-described device embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the device embodiments, which will not be repeated here.

[0067] Example 10 Based on the above embodiments, the present invention also proposes a computer-readable storage medium storing a multi-sensor fusion imaging program, wherein when the multi-sensor fusion imaging program is executed by a processor, the steps of the multi-sensor fusion imaging method as described in any of the above claims are implemented.

[0068] It should be noted that the above-described medium embodiments and method embodiments belong to the same concept. The specific implementation process can be found in the method embodiments, and the technical features in the method embodiments are also applicable to the medium embodiments, which will not be repeated here.

[0069] The multi-sensor fusion imaging method, device, and computer-readable storage medium of the present invention are implemented through It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0070] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0072] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A multi-sensor fusion photographing method, characterized by, The method includes: Simultaneously collect data from multiple sensors and perform dynamic weighted fusion calculations to determine the current shooting scene; An imaging strategy corresponding to the shooting scene is invoked, and multiple parameters in the imaging strategy are smoothly transitioned in a coordinated manner to determine the current shooting parameters.

2. The multi-sensor fusion photographing method of claim 1, wherein The simultaneous acquisition of data from multiple sensors previously included: During the initialization phase, the preset rule base and mapping table are loaded; Activate multiple of the aforementioned sensors and calibrate the synchronization clock.

3. The multi-sensor fusion photographing method of claim 2, wherein The simultaneous acquisition of data from multiple sensors specifically includes: Illumination data, motion data, and positioning data are collected by an ambient light sensor, a three-axis gyroscope, and a positioning module, respectively. A unified timestamp is added to the illumination data, the motion data, and the positioning data to align the 3D data.

4. The multi-sensor fusion imaging method according to claim 3, characterized in that, The simultaneous acquisition of data from multiple sensors then includes: The illumination data, motion data, and positioning data are processed using moving average filtering, Kalman filtering, and the Raida criterion, respectively. Generate illumination data L, motion data G, and positioning data V after data normalization.

5. The multi-sensor fusion imaging method according to claim 4, characterized in that, The dynamic weighted fusion calculation specifically includes: Call the preset weighting coefficients α, β, γ; The fusion feature value F is calculated according to the formula F=α×L+β×G+γ×V, and the shooting scene is determined based on the fusion feature value and the rule base.

6. The multi-sensor fusion imaging method according to claim 5, characterized in that, The invocation of the imaging strategy corresponding to the shooting scene specifically includes: Call the mapping table; The mapping table determines the imaging strategy corresponding to the shooting scene, as well as a plurality of parameters in the imaging strategy.

7. The multi-sensor fusion imaging method according to claim 6, characterized in that, The coordinated smooth transition of multiple parameters in the imaging strategy specifically includes: Calculate the differences between the multiple parameters and the preset target parameters; The shooting parameters are obtained by gradually changing the difference within a preset time using a linear interpolation algorithm.

8. The multi-sensor fusion imaging method according to claim 7, characterized in that, The method further includes: During the shooting process, user operation data is collected; The rule base is optimized and updated based on the operational data.

9. A multi-sensor fusion imaging device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the multi-sensor fusion imaging method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an application of multi-sensor fusion imaging program, which, when executed by a processor, implements the steps of the multi-sensor fusion imaging method as described in any one of claims 1 to 8.

Images