A camera-based intelligent decision control system, method and mobile terminal

Abstract

The application discloses a kind of intelligent decision control system, method and mobile terminal based on camera shooting, system is deployed in mobile terminal, using the three-layer architecture of input layer, decision layer, execution layer;Input layer is collected image, motion, user behavior and social label data by multi-modal perception module and is preprocessed;Decision layer extracts image features by improved ResNet network, and combines attention mechanism to fuse multi-modal features, then by LSTM network Time series decision, output scene classification result and optimal shooting parameter set;Execution layer dynamically allocates lens task and computing power resources by cross-camera cooperative unit, and executes parameter adjustment and feedback fine tuning by parameter control unit;The application solves the problems of poor scene adaptability, user intention fragmentation and inefficient hardware collaboration in the prior art, realizes global collaboration of environment, user intention and hardware resources, significantly improves the success rate of complex scene shooting, reduces system power consumption, and does not require user manual operation.

Description

Technical Field

[0001] This invention belongs to the field of camera control technology for mobile terminals such as smartphones and tablets, and in particular relates to an intelligent decision control system, method, mobile terminal and storage medium based on camera shooting. Background Technology

[0002] With the rapid development of mobile communication technology, smartphones, tablets and other mobile terminals have become indispensable tools in life. Their built-in camera functions have also evolved from simply recording images to complex systems that meet users' needs for high-quality and diverse shooting.

[0003] Currently, the configuration of multiple rear cameras on mobile terminals has become an industry standard, and users' core needs for shooting experience have been upgraded from the basic "clear imaging" to "adaptive to complex scenes, extremely simple operation, and precise matching of needs".

[0004] However, existing camera technologies mostly focus on single-dimensional optimization, lacking a holistic and coordinated consideration of environmental information, user intent, and hardware status, resulting in a serious disconnect between the actual shooting experience and user needs.

[0005] For example, Chinese patent document CN119785051A discloses a multi-branch neural network image processing method, but it only optimizes the post-processing of images and does not involve dynamic decision-making strategies before shooting; another example is Chinese patent document CN119865706A, which discloses an HDSR technology that mainly focuses on frame rate optimization and does not integrate multimodal data for comprehensive decision-making.

[0006] Specifically, existing camera control technology has the following shortcomings: ① Poor scene adaptability: Existing technologies mostly adopt fixed scene modes, such as "night scene", "portrait" and "sports". These modes are based on static parameter presets and rely on users to manually identify the scene and switch modes. They cannot effectively deal with common complex scenes in the real world (such as moving objects under backlight conditions and distant scenes in low light conditions). This often leads to problems such as blurry images, exposure imbalance and loss of details. ② User intent fragmentation: The parameter adjustments of existing solutions mostly rely solely on hardware sensor data (such as ambient light intensity and distance), failing to effectively connect with the user's deeper intent, such as social sharing needs (taking ID photos, food photos, and landscape photos for different platforms) and personal historical preferences (liking bright tones and large aperture bokeh effects). Users still need to make cumbersome manual adjustments, resulting in high learning and operational costs. ③ Inefficient hardware collaboration: In multi-camera systems, the common collaboration logic is to bind lenses with specific focal lengths to fixed tasks (such as telephoto lenses for portraits and ultra-wide-angle lenses for landscapes). This fixed strategy fails to dynamically schedule lenses and computing power according to the real-time scene, often resulting in inactive lenses still occupying system resources (such as memory and power consumption), causing overall power consumption to increase and response delay. ④ Single decision-making dimension: The decision-making process of shooting strategy usually relies only on optical image data, failing to make full use of the rich sensor data (such as motion sensor, ambient light sensor) and user data on mobile terminal, resulting in insufficient decision accuracy.

[0007] Therefore, there is an urgent need in this field for an intelligent shooting decision-making scheme that can achieve global coordination of environment, user intent and hardware resources, in order to improve the shooting success rate and user experience in complex scenarios. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent decision control system based on camera shooting, which aims to solve the problems of weak scene adaptability, insufficient user intent matching and low hardware coordination efficiency in the prior art.

[0009] To achieve the above objectives, the present invention primarily provides an intelligent decision control system based on camera shooting. This system is deployed in a mobile terminal and adopts a three-layer architecture of input layer, decision layer and execution layer. Each layer interacts with data and transmits instructions through a hardware abstraction layer.

[0010] Specifically, the input layer is a multimodal perception module used to collect and preprocess multi-dimensional data. This module includes: Image acquisition unit: Utilizing the multi-camera module (such as a 1 / 1.3-inch 50-megapixel main camera, a 10-megapixel 5x periscope telephoto lens) and a ToF (Time-of-Flight) depth sensor or binocular camera configured in the mobile terminal, it simultaneously acquires RGB images and distance information of the target being measured. Motion acquisition unit: Using the terminal's built-in inertial measurement unit (IMU), data from the accelerometer and gyroscope are acquired at a sampling rate of no less than 100Hz, and the real-time jitter amplitude and attitude angle (such as pitch angle and yaw angle) of the device are calculated. User behavior analysis unit: Reads the user's historical shooting records from the local encrypted storage space of the terminal, and statistically analyzes their preference parameters, such as frequently using f / 1.8 aperture in "Portrait" mode, or preferring a higher ISO value in "Night Scene" mode. Social tag extraction unit: On the device itself, natural language processing technology is used to analyze the sharing history of pictures in the album (such as the titles and tags added when sharing to social media) and extract scene keywords such as "#Night Scene", "#ID Photo", and "#Travel". All processing is completed on the device itself to protect user privacy. Furthermore, the multimodal perception module performs preprocessing such as normalization and outlier removal on the collected data to form a standardized data matrix for use by the decision-making level.

[0011] Specifically, the decision layer is the core scene intent prediction model, which is built based on a deep learning network and used for data fusion, scene classification, and parameter recommendation. This model consists of three sub-parts: Feature extraction sub-model: For image data, an improved ResNet-50 deep convolutional neural network is used to extract a 2048-dimensional deep feature vector from the input image; for motion data, it is converted into a 64-dimensional feature vector through a fully connected layer; for text data such as social tags, it is mapped into a 32-dimensional feature vector through word embedding technology. Feature fusion sub-model: Employs an attention mechanism to assign different importance weights to features of different modalities; for example, image features are given a weight of 0.6, motion features a weight of 0.25, and social tag features a weight of 0.15; then the weighted features are concatenated into a 128-dimensional fusion feature vector. Temporal decision sub-model: A three-layer long short-term memory network (LSTM) is used to perform temporal modeling on the fused feature vectors to capture dynamic changes during the shooting process; finally, the Softmax classifier outputs eight preset scene categories (such as "backlighting motion", "low-light portrait", "macro close-up", "standard landscape", etc.) and outputs a set of optimal shooting parameters for each scene, including focal length, aperture, shutter speed, ISO sensitivity, image stabilization mode, etc.

[0012] Furthermore, the scene intent prediction model is trained using a joint loss function L, which is a weighted sum of the cross-entropy classification loss L_cls and the root mean square error regression loss L_reg, i.e., Formula 1: L = 0.7 L_cls + 0.3L_reg. The scene intent prediction model is trained on a mixed dataset containing the public ImageNet-1K dataset and a self-built labeled dataset of 100,000 images, achieving a classification accuracy of 92.3% on the validation set, with inference latency controlled within 100 milliseconds.

[0013] Specifically, the execution layer includes a cross-camera collaboration unit and a parameter control unit; wherein: Cross-camera collaboration unit: Receives scene classification results from the decision layer, dynamically allocates camera tasks according to scene requirements, and activates / suspends corresponding lenses based on scene type and target distance via HAL commands; for example, in a "distant view" scene, it activates the telephoto camera and suspends the ultra-wide-angle camera; or in a "macro" scene, it activates a dedicated macro camera; simultaneously, this cross-camera collaboration unit dynamically schedules the computing resources of the image signal processor (ISP) and neural network processor (NPU), for example, allocating 70% of the computing power to the currently working main camera and reserving 30% of the computing power for rapid scene switching, thereby reducing system power consumption by approximately 25% while ensuring performance.

[0014] The parameter control unit is responsible for sending the set of shooting parameters recommended by the decision-making layer to the activated cameras. At the same time, it monitors the feedback indicators of imaging in real time (such as the Laplacian variance calculated by the image sharpness algorithm and the exposure value obtained by histogram analysis). If the indicators do not meet the preset standards, it fine-tunes the parameters by no more than 10% (such as appropriately increasing the shutter speed to improve sharpness or adjusting the exposure compensation to correct the brightness), forming a closed-loop control to ensure the best final image quality.

[0015] Compared with the prior art, the present invention has the following significant advantages: 1) Global collaboration and high adaptability: By integrating multimodal data such as images, motion, user behavior, and social intent, global perception and collaborative decision-making of the environment, users and hardware are achieved, and the recognition accuracy of complex scenes is improved to more than 89.7%, which is more than 40% higher than traditional technologies.

[0016] 2) User experience greatly optimized: The system can automatically and accurately match user intent and scenario needs, reducing the frequency of manual adjustments by more than 60%, and the accuracy of demand matching exceeds 91%, truly realizing "shooting blockbuster movies anytime, anywhere".

[0017] 3) Efficient use of hardware resources: By dynamically scheduling camera and computing resources, idle resources are avoided and the power consumption of the multi-camera collaborative system is reduced by about 25%, the scene switching latency is less than 50 milliseconds, and the terminal's battery life is improved.

[0018] 4) Excellent real-time performance: The entire system's response time, from data acquisition to parameter execution, is controlled within 300 milliseconds, fully meeting the smoothness requirements of real-time shooting. Attached Figure Description

[0019] Figure 1 This is an overall architecture block diagram of an intelligent decision control system provided in an embodiment of the present invention; Figure 2 This is an algorithm flowchart of a scene intent prediction model in one embodiment of the present invention; Figure 3 This is a timing diagram of task allocation for a cross-camera collaboration unit in one embodiment of the present invention; Figure 4 This is a flowchart illustrating an intelligent decision-making and control method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of a mobile terminal in one embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings. It should be emphasized that the embodiments described below are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0021] refer to Figure 1 As shown, the system hardware platform and software initialization ( Example 1 The system of this invention adopts a three-layer architecture of input layer, decision layer and execution layer on a mobile terminal. This embodiment will describe in detail the hardware foundation and system startup process required to implement this invention. The mobile terminal takes a high-end smartphone as an example. Its core hardware configuration includes a camera and vision subsystem, a processing and computing subsystem, a sensor subsystem and a storage subsystem.

[0022] 1. The camera and vision subsystem includes: 1) Main camera: It uses a 50-megapixel Sony IMX766 sensor with a sensor size of 1 / 1.56 inches and a single pixel size of 1.0μm. It supports four-pixel binning technology (outputting 12.5-megapixel, equivalent single pixel size of 2.0μm), is equipped with an f / 1.8 aperture lens, and supports optical image stabilization (OIS). 2) Telephoto camera: It uses a 10-megapixel Samsung S5K3K1 sensor, supports 3x optical zoom, and is equipped with an f / 2.4 aperture lens and OIS; 3) Ultra-wide-angle camera: It uses a 13-megapixel OmniVision OV13B10 sensor, has a 120° diagonal field of view, and is equipped with an f / 2.2 aperture lens; 4) Depth Sensing Unit: Employs AMS's TMF8801 ToF (Time-of-Flight) lidar sensor, with a ranging range of up to 5 meters and an accuracy within ±3%. Alternatively, a dual-camera-based stereo vision ranging solution can be used.

[0023] 2. The processing and computing subsystem includes: 1) Central Processing Unit (CPU): It adopts the Qualcomm Snapdragon 8 Gen 2 mobile platform, which includes one Cortex-X3 super core, two Cortex-A715 large cores, two Cortex-A710 medium cores and three Cortex-A510 small cores; 2) Graphics Processing Unit (GPU): Integrated Adreno GPU; 3) Image Signal Processor (ISP): Integrated Spectra ISP, supporting concurrent processing of three cameras; 4) Neural Processing Unit (NPU): Integrates a Hexagon processor, providing up to 26 TOPS of AI computing power, specifically designed to accelerate inference of deep learning models.

[0024] 3. The sensor subsystem includes: 1) Inertial Measurement Unit (IMU): The Bosch BMI260 six-axis sensor is used, which includes a three-axis accelerometer and a three-axis gyroscope. The sampling rate can reach up to 1.6kHz. In this system, it is configured to operate at a frequency of 100Hz to balance accuracy and power consumption.

[0025] 2) Ambient light sensor: used to sense the intensity of ambient light.

[0026] 4. The storage subsystem includes: 1) Running memory (RAM): 12GB LPDDR5X.

[0027] 2) Internal storage (ROM): 256GB UFS 4.0 flash memory.

[0028] 5. The system software initialization process is as follows: When a user launches the camera application, the operating system kernel loads the corresponding device driver. Subsequently, core software modules such as the multimodal perception module, the pre-trained scene intent prediction model (stored in ROM in ONNX or TFLite format), and the cross-camera collaboration unit are loaded into RAM. The weight parameters of the scene intent prediction model are loaded into the NPU's internal cache through the NPU's dedicated runtime environment to achieve high-speed, low-power inference. The Hardware Abstraction Layer (HAL) completes the enumeration and initialization of hardware devices such as the camera and IMU, establishing a high-speed data path from sensor data acquisition to model inference and then to actuator control. The entire initialization process is completed within 500 milliseconds.

[0029] Combination Figure 1 and Figure 4 As shown, the complete workflow of a typical application scenario ( Example 2)This embodiment uses a typical and complex scenario, taking "moving human figures in backlight" as an example, to detail the entire collaborative working process of the system of the present invention from the perspective of system architecture, from perception to execution.

[0030] The specific scenario of this embodiment is as follows: at dusk, a user wants to take a picture of a child running in a park (with a bright sky in the background); at this time, the lighting conditions are backlit, and the subject is in motion.

[0031] (i) Input layer: Perform multimodal data acquisition and fine preprocessing When the user opens the camera app and enters the preview interface, the image acquisition unit of the multimodal perception module immediately activates the main camera, telephoto camera, and ToF sensor (S101) simultaneously via hardware abstraction layer (HAL) instructions: 1. Image Data Acquisition (S102): The main camera outputs a preview stream image at a frame rate of 30fps (resolution set to 1080p to reduce processing load). After the image data undergoes preliminary bad pixel correction and lens shadow correction by the ISP, it is sent to the system buffer. The preview image analysis shows that the brightness value (Y) of the background (sky) area exceeds 200 (range 0-255), while the brightness value of the foreground (child) area is less than 50, indicating a wide dynamic range and typical backlight characteristics. The ToF sensor measures the depth of the scene in real time, and the distance between the child and the mobile phone is found to be approximately 4.5 meters.

[0032] 2. Motion Data Acquisition (S102): The motion acquisition unit reads raw data of triaxial acceleration and triaxial angular velocity from the IMU at a frequency of 100Hz. The raw data is fused and denoised using a Kalman filter algorithm to calculate the device's real-time attitude angles (pitch: -10°, yaw: +5°) and jitter amplitude (quantized as a one-dimensional vector with a magnitude of 0.15 rad / s), indicating that the device has slight translational jitter that follows the child's movement direction.

[0033] 3. User behavior data analysis (S103): The user behavior analysis unit accesses the local encrypted SQLite database to query the user's shooting records in the past month; through Apriori association rule algorithm analysis, it is found that when the scene EXIF ​​information contains the keyword "backlight", the user has an 85% probability of enabling "HDR" mode; when the shooting subject is "person", the user has a 90% probability of selecting "portrait" mode; these preferences are quantified into weight vectors.

[0034] 4. Social tag extraction (S103): The social tag extraction unit runs a lightweight BERT model locally on the device (without network connection) to scan the sharing records of the 100 most recent pictures in the user's album; extracts the high-frequency keywords "#parent-child outdoor", "#sunset" and "#running", and converts them into 32-dimensional word embedding vectors.

[0035] 5. Data Preprocessing and Standardization (S103): All collected raw data are fed into the preprocessing pipeline: 1) Image data: Pixel values ​​are normalized to the [0,1] range; 2) Motion data: Attitude angles were normalized to [-1,1], and jitter amplitude was normalized to [0,1]; 3) User preferences and social tags: already in vector form, directly perform L2 normalization; Ultimately, the data from all modalities are integrated into a standardized multidimensional data matrix, awaiting use by the decision-making level.

[0036] (ii) Decision-making layer: Perform feature fusion and intelligent decision-making for scene intent prediction models. Combination Figure 2 As shown, the scene intent prediction model of the decision layer consists of three sub-parts: feature extraction, feature fusion, and temporal decision-making. 1. Feature extraction (S105) includes the extraction of image features, motion features, and contextual features: 1) Image features: The normalized preview frame image (downsampled to 224x224 resolution) is fed into the ResNet-50 convolutional neural network of the feature extraction sub-model, and the last fully connected layer of ResNet-50 is removed. The 2048-dimensional feature vector after its global average pooling layer is taken as the high-level semantic features of the image. This feature vector contains information about scene content (people, sky), lighting conditions (backlighting) and composition. 2) Motion characteristics: The 6-dimensional motion data (attitude angle + jitter vector) calculated from IMU data is processed by a micro neural network containing two fully connected layers (64-dimensional -> 32-dimensional) to output a 32-dimensional motion feature vector, which characterizes the stability and motion state of the device; 3) Contextual features: After concatenating the user preference vector and social tag vector, a 64-dimensional contextual feature vector is generated through a fully connected layer to represent the user's shooting intention and scene semantics.

[0037] 2. Feature fusion (S105): 1) The three feature vectors mentioned above (image 2048-dimensional, motion 32-dimensional, and context 64-dimensional) are input into the attention mechanism layer (S104) of the feature fusion sub-model. The decision layer is a small feedforward neural network that calculates an attention weight score (Scaled Dot-Product Attention) for each feature vector. In this example, the calculated weights are: image features 0.65, motion features 0.20, and context features 0.15. This reflects that image information dominates the current decision, but motion information and user intent provide key auxiliary judgments. 2) Each feature vector is multiplied by its weight and then concatenated into a unified 2144-dimensional fused feature vector.

[0038] 3. Timing Decision (S106): 1) The fused feature vector is fed into the three-layer LSTM network of the temporal decision sub-model. The LSTM network has 512 hidden units and continuously receives the fused features of the current frame and the previous 9 frames (a total of 10 frames, with a time window of about 330 milliseconds). 2) The LSTM network learns the dynamic changes of the scene during this time period through its gating mechanism (such as the child's movement trend and the persistence of device shaking); the output of the last time step of the LSTM network is fed into a fully connected layer and a Softmax classifier. 3) The model output is the probability distribution of 8 preset scene categories; in this example, the output is: the probability of "backlit moving portrait" is 89.7%, the probability of "standard portrait" is 8.1%, and the sum of the probabilities of other scenes is 2.2%; therefore, the system can determine that the current scene is "backlit moving portrait". 4) At the same time, a set of recommended shooting parameters bound to the "backlit moving portrait" scene is output. This set of parameters is generated by the model regression head and is as follows: {Camera used: main camera; HDR mode: on; shutter speed: 1 / 500 second; ISO sensitivity: 400; focus mode: continuous autofocus (CAF) + face tracking; image stabilization mode: OIS + electronic image stabilization (EIS) sports mode}.

[0039] (iii) Execution layer: performs dynamic scheduling and closed-loop control of hardware resources. 1. Cross-camera collaborative scheduling (S107): 1) The cross-camera collaboration unit receives the "backlit moving portrait" scene tag and recommended parameter set from the decision-making layer; 2) Combining Figure 3As shown, the collaborative unit sends instructions to the camera controller via HAL according to a predefined strategy (such as "prioritize the use of the main camera in portrait and motion scenes"): wake up the main camera and put it into a ready state to receive high-resolution shooting instructions; at the same time, it sends sleep instructions to the telephoto and ultra-wide-angle cameras to put them into a low-power state. 3) In terms of computing power scheduling, the collaborative unit dynamically allocates 70% of the computing resources of the NPU and ISP to the image processing pipeline of the main camera through the operating system scheduler, and reserves 30% of the resources to maintain the smoothness of the system UI and cope with possible rapid scene switching.

[0040] 2. Parameter control and feedback fine-tuning: 1) The parameter control unit sends the recommended parameter set to the main camera's driver and ISP; 2) The user presses the shutter button, and the camera captures a frame of image (S108); the parameter control unit does not save it immediately, but first performs an image quality assessment (S109), including sharpness assessment and exposure assessment: (1) Sharpness assessment: The Laplacian variance of the image was calculated in the detected face area (the child's face). The measured value was 45, which is lower than the preset sharpness threshold (e.g., 100 under good lighting conditions, which can be appropriately reduced to 60 under backlight conditions), indicating that the image may be slightly blurred due to motion. (2) Exposure assessment: Analyze the image histogram and check the average brightness of the face area. The measured value is 40 (target value is mid-gray 128), indicating that the subject is underexposed; 3) Feedback Fine-tuning (S110): Since the image quality is not fully up to standard, the parameter control unit starts the fine-tuning cycle; based on the evaluation results, adjustments are made in an increment of no more than 10% of the recommended value: the shutter speed is slightly increased from 1 / 500 second to 1 / 640 second to further freeze the motion, while the ISO is increased from 400 to 500 to compensate for the reduced light intake due to the faster shutter speed, and the exposure compensation (EV) is fine-tuned by +0.3; 4) The system immediately takes a second shot (or simulates a shot) using the finely tuned parameters {shutter speed: 1 / 640s, ISO: 500, EV+0.3} (S108); during the re-evaluation (S109), the Laplacian variance of the face area reaches 65, and the average brightness reaches 55. Although not perfect, it is within an acceptable range, and the motion blur and noise levels are balanced; the parameter control unit confirms that the imaging effect meets the standard, and finally sends the image data to the JPEG encoder and saves it to the album (S111).

[0041] To cover a broader range of protection and address potential circumvention designs, the present invention also includes, but is not limited to, the following variations ( Example 3 ): 1. Hardware configuration changes: 1) Low-cost solution: In cost-sensitive devices, the ToF sensor can be replaced with a binocular stereo vision system, which uses the parallax of the main camera and the ultra-wide-angle camera for distance measurement. This solution can reduce hardware costs by about 30%. Although the absolute distance measurement accuracy drops to an error of ≤8%, it can still meet the distance perception needs of most scenarios. The camera combination can be simplified to "main camera + ultra-wide-angle + macro", and a "macro close-up" category can be added to the scene classification. The parameter recommendations will focus on the closest focusing distance and depth of field control.

[0042] 2) High-end professional solutions: For foldable screen phones or tablets, their multi-screen advantages can be leveraged to display the analysis results of multimodal data (such as attention weight distribution and scene classification confidence) in real time on the secondary screen, providing users with a transparent decision-making process and enhancing their sense of control.

[0043] 2. Algorithm model transformation: 1) Precision-first transformation: To pursue higher temporal modeling accuracy, the LSTM network can be replaced with a Transformer encoder; through the self-attention mechanism, the Transformer can better capture the dependencies between different time points in a long sequence; experiments have shown that, when trained on the same dataset, its classification accuracy for complex dynamic scenes can be improved by about 3-5 percentage points compared to LSTM.

[0044] 2) Efficiency-first transformation: To adapt to low-end and mid-range chips or pursue ultra-fast startup, the feature extraction network ResNet-50 can be replaced with the lightweight MobileNetV3-Small; through depthwise separable convolution, the number of model parameters can be reduced by about 60%, and the inference speed can be more than doubled, with only a slight sacrifice (about 2-3 percentage points) in classification accuracy.

[0045] 3) Loss function variation: The weight coefficients (α, β) in the joint loss function can be adjusted according to actual needs. If the accuracy of parameter recommendations is more important, it can be adjusted to L = 0.5 L_cls + 0.5 L_reg; if the model is overfitting, an L2 regularization term is added to the loss function.

[0046] 3. Decision-making logic transformation: 1) Low Power Smart Mode: The system continuously monitors the battery level; when the battery level is below the 20% threshold, the decision logic of the scene intent prediction model will introduce "power consumption" as one of the optimization objectives; under the premise of ensuring basic image quality, it will tend to recommend more power-saving parameter combinations (such as reducing photo resolution, limiting high frame rate recording, and avoiding starting multiple cameras at the same time), and can appropriately reduce the frequency of dynamic scheduling and adopt a more conservative static allocation strategy; experiments show that this mode can extend the continuous use time of the camera by about 20%.

[0047] 2) Professional Mode Compatibility: The system provides a "Professional Mode" switch; when enabled, the parameter control unit will allow users to completely manually override the parameters such as shutter speed, ISO, and white balance automatically recommended by the system; at the same time, the multimodal perception module and scene intent prediction model are still running in the background, and can display the analysis results (such as the measured shake amplitude and scene classification suggestions) to the user in the form of visual prompts (such as safe shutter speed prompts and histograms) as a reference for manual adjustment, realizing artificial intelligence-assisted manual photography.

[0048] In addition, to quantify the effectiveness of the present invention, a comparative experiment was specifically designed to verify the experimental data of the technical effect (Example 4): 1. Experimental setup Mainstream flagship mobile phones on the market (equipped with fixed scene modes) were selected as the control group, and mobile phones on the same platform equipped with the system of this invention were selected as the experimental group. Blind testing and evaluation were conducted on a test set containing 500 images covering 8 composite scenes such as "backlighting motion", "low-light portrait", and "macro landscape". Fifty professional photographers and ordinary users were invited to participate in subjective evaluation.

[0049] 2. Experimental Results 1) Scene recognition accuracy: The scene recognition accuracy of the experimental group reached 89.7%, which was significantly higher than the 52.1% of the control group (which relied on manual selection by the user). 2) Image quality satisfaction: Using a 5-point mean opinion score (MOS), the experimental group scored 4.45 points, while the control group scored 3.62 points. The improvement was mainly reflected in exposure accuracy under complex lighting conditions and the sharpness of moving subjects; 3) User experience: The experimental group required 68% less manual intervention compared to the control group; 4) System power consumption: In a continuous 30-minute shooting test, the experimental group, thanks to its dynamic scheduling strategy, reduced the average power consumption of the overall camera module by 26.5% compared to the control group (multiple cameras always on). 5) Response speed: The entire process from starting the camera to taking the first high-quality photo took 280 milliseconds in the experimental group, which meets the real-time requirements.

[0050] In summary, combining Figure 5 The mobile terminal hardware structure shown in this invention effectively solves the various problems mentioned in the background art through the deep optimization design of the above-mentioned hardware and software collaboration, and realizes an intelligent, adaptive, and highly efficient mobile terminal shooting experience.

[0051] It should be understood that the above description is only a preferred embodiment of the present invention and is not sufficient to limit the technical solution of the present invention. For those skilled in the art, within the spirit and principles of the present invention, additions, subtractions, substitutions, transformations or improvements can be made based on the above description, and all such additions, subtractions, substitutions or improvements should fall within the protection scope of the appended claims of the present invention.

Claims

1. A camera-based intelligent decision-making control system, deployed on a mobile terminal, characterized in that, The system includes: The multimodal perception module is used to collect and preprocess image data, motion data, user behavior data, and social tag data; The scene intent prediction model is connected to the multimodal perception module. It is used to perform feature extraction, fusion and temporal decision-making through a deep learning network based on the preprocessed multimodal data, and output scene classification results and corresponding recommended shooting parameter sets. A cross-camera collaboration unit is connected to the scene intent prediction model and is used to dynamically schedule the hardware resources and corresponding computing resources of multiple cameras on the mobile terminal based on the scene classification results and the recommended shooting parameter set. The parameter control unit, connected to the scene intent prediction model and the cross-camera collaboration unit, is used to send the recommended shooting parameter set to the activated camera and perform feedback fine-tuning based on real-time imaging effects.

2. The intelligent decision-making control system based on camera shooting according to claim 1, characterized in that, The multimodal sensing module includes: The image acquisition unit is used to simultaneously acquire RGB images and target distance data through the main camera, telephoto camera and depth sensor; The motion acquisition unit is used to acquire data on device jitter amplitude and shooting posture through accelerometers and gyroscopes; The user behavior analysis unit is used to read the user's historical shooting parameters and scene preference data from local encrypted storage; The social tag extraction unit is used for localized parsing of album sharing records and extraction of contextual keywords.

3. The intelligent decision-making control system based on camera shooting according to claim 1 or 2, characterized in that, The scene intent prediction model includes: The feature extraction sub-model uses a ResNet network to extract deep features from image data, a fully connected layer to transform motion data features, and a word embedding model to generate label data features. The feature fusion sub-model uses an attention mechanism to assign different weights to image features, motion features, and label features, and then concatenates the weighted features into a fused feature vector. The temporal decision sub-model uses an LSTM network to process the fused feature vector and outputs a set of preset scene classifications and their corresponding shooting parameters through the Softmax function. The shooting parameter set includes at least focal length, aperture, shutter speed, ISO sensitivity and image stabilization mode.

4. The intelligent decision-making control system based on camera shooting according to claim 3, characterized in that: The scene intent prediction model is trained using a joint loss function L, which is a weighted sum of the classification loss L_cls and the parametric regression loss L_reg, expressed as: L = α L_cls + β L_reg; (Formula 1) Here, α and β are weighting coefficients, and α + β = 1.

5. The intelligent decision-making control system based on camera shooting according to claim 1, characterized in that, The specific method by which the cross-camera collaborative unit dynamically schedules hardware resources is as follows: Based on the scene classification results, the camera that best matches the current scene is activated via hardware abstraction layer instructions, while other non-core cameras are put into hibernation. At the same time, 60%-80% of the system's total computing resources are dynamically allocated to the activated cameras for processing.

6. The intelligent decision-making control system based on camera shooting according to claim 1 or 5, characterized in that, The specific method by which the parameter control unit performs feedback fine-tuning is as follows: After the shutter is pressed and before the image is finally saved, the sharpness index and exposure value index of the main area in the image are calculated in real time. If the sharpness index is lower than the first threshold or the exposure value index deviates from the second threshold, the shutter speed and ISO sensitivity are iteratively adjusted with the preset maximum adjustment range, and the imaging index is re-evaluated after each adjustment until the imaging index meets the standard or the maximum number of iterations is reached.

7. A camera-based intelligent decision control method, applied in the camera-based intelligent decision control system as described in any one of claims 1-6, characterized in that, The method includes: Collect and preprocess multimodal data; The preprocessed data is input into the scene intent prediction model to obtain scene classification results and recommended shooting parameter sets; Based on scene classification results, dynamically allocate cameras and computing resources; The recommended shooting parameter set is sent to the activated camera, and shooting is performed; Based on the real-time imaging results, the shooting parameters are fine-tuned based on feedback.

8. The intelligent decision-making and control method based on camera shooting according to claim 7, characterized in that, The steps for dynamically scheduling cameras and computing resources include: allocating a main camera and one or more auxiliary cameras to the current scene; and allocating primary computing resources to the main camera.

9. A mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, characterized in that, When the processor executes the computer program, it implements the steps of the intelligent decision-making and control method based on camera shooting as described in claim 7 or 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the intelligent decision control method based on camera capture as described in claim 7 or 8.

Images