Intelligent composition auxiliary method and system for tourism photography

Abstract

The invention relates to the technical field of intelligent photography, discloses an intelligent composition auxiliary method and system for tourism photography, and aims to solve the problems that scene semantics cannot be deeply understood and dynamic active composition guidance is difficult to provide in the prior art. The method comprises the following steps: acquiring multi-modal data such as images, geographic coordinates, equipment postures and the like in real time; constructing a scene semantic map for describing visual elements and relationships thereof; outputting a current composition score and an optimization gradient through a pre-trained composition aesthetics quantitative evaluation model; and generating a dynamic visual servo guidance instruction, and superposing the instruction on a view-finding interface in an augmented reality mode to guide a user to adjust the pose and focal length of the camera. The system comprises a multi-modal data acquisition module, a map construction module, an aesthetic evaluation module, a guide generation module and an AR rendering module. According to the method and the device, the crossing from a static reference line to semantization and closed-loop intelligent composition assistance is realized, the operation burden of a user is remarkably reduced, and the imaging aesthetic quality is improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent photography technology, specifically relating to an intelligent composition assistance method and system for tourism photography. Background Technology

[0002] With the widespread use of smartphones and digital cameras, travel photography has become an important way for people to record their travel experiences. However, ordinary users often lack professional composition knowledge in complex and ever-changing outdoor scenes, making it difficult to quickly judge key compositional elements such as the subject, horizon tilt, and visual center of gravity shift, resulting in low shooting efficiency and inconsistent image quality. Existing photography assistance technologies mainly rely on static rule bases (such as the rule of thirds and symmetrical composition) for post-processing scoring or simple prompts. Their core logic is based on preset aesthetic templates and cannot dynamically perceive the semantic content (such as mountains, buildings, and groups of people) and environmental features (such as light direction, depth of field, and distribution of dynamic elements) of the current scene. When faced with unstructured natural landscapes or scenes with multiple people interacting, such methods often provide compositional suggestions that are detached from the actual shooting intention due to a lack of understanding of the scene context, and may even mislead users to adjust their framing angles. In addition, existing systems mostly adopt offline processing modes, which cannot fuse visual recognition results and composition strategies with low latency during real-time framing, making it difficult to support continuous dynamic composition guidance, which seriously restricts the practicality and user experience of intelligent photography assistance in real-world travel scenarios.

[0003] Therefore, there is an urgent need for an intelligent auxiliary method that can combine scene semantic understanding with dynamic composition strategy generation to achieve real-time, accurate, and personalized composition guidance for tourism photography. Summary of the Invention

[0004] The technical problem that this invention aims to solve is that existing tourism photography assistance technologies only provide static, non-semantic composition reference lines, which cannot deeply understand the internal structure and aesthetic relationship of the shooting scene, nor can they provide real-time and proactive composition adjustment instructions based on dynamically changing scene content and shooting intentions. This results in a high cognitive load for users and the final image quality being limited by their subjective composition ability.

[0005] To address the aforementioned technical issues, this invention provides an intelligent composition assistance method and system for tourism photography. Its core technical solution involves: constructing a multimodal scene perception and semantic understanding framework. First, scene data is captured and fused in real-time by integrating visual, spatial geographic, and camera pose sensors. Next, a scene graph construction algorithm is used to deeply analyze the multimodal data, transforming the complex visual scene into a structured knowledge graph composed of the subject, supporting elements, foreground, background, and their spatial and semantic relationships. Then, a composition aesthetics quantitative evaluation model, pre-trained with a vast amount of professional photographic works, scores the current composition's scene graph in real-time and calculates the composition adjustment gradient to achieve the optimal aesthetic state. Finally, this gradient is transformed into specific and intuitive visual servo guidance commands, dynamically rendered on the viewfinder using augmented reality technology, actively guiding the user to perform camera panning, rotation, and zoom operations until the composition reaches a preset aesthetic quality threshold, thereby achieving closed-loop intelligent assistance for photographic composition.

[0006] According to one aspect of the present invention, an intelligent composition assistance method for tourism photography is provided, comprising the following steps: Real-time acquisition of multimodal scene data, including image frame sequences within the current field of view, global geospatial coordinates of the device, three-axis attitude information of the device, and time information; Based on the multimodal scene data, a scene semantic graph is constructed. The scene semantic graph is a structured data used to describe the visual elements in a scene and their interrelationships, where nodes represent visual elements and edges represent a set of preset relationships between visual elements. The real-time image frame sequence and the scene semantic map are input into a preset composition aesthetic quantification evaluation model. The model outputs the quantification aesthetic score of the current composition and a composition optimization gradient vector to improve the score. Based on the optimized gradient vector of the composition, a set of dynamic visual servo guidance instructions are generated, which indicate the pose and focal length parameters that the camera needs to adjust in order to achieve a higher aesthetic score. The dynamic visual servo guidance command is rendered as an augmented reality guidance layer and overlaid on the live view interface to guide the user in operating the camera.

[0007] As one embodiment of the present invention, the real-time acquisition of multimodal scene data specifically includes: The device's image sensor module continuously acquires raw image data with a resolution of no less than 4,000 by 3,000 pixels at a frame rate of no less than 30 frames per second, forming an image frame sequence. The device's three-dimensional geographic coordinates, consisting of longitude, latitude, and altitude, are obtained through the device's global navigation satellite system receiver module. The inertial measurement unit module of the equipment obtains the three-axis attitude information consisting of the current roll angle, pitch angle, and yaw angle, as well as the angular velocity information along these three axes. The current Gregorian calendar date and local time are obtained through the device's system clock module.

[0008] As one embodiment of the present invention, the construction of the scene semantic graph specifically includes the following sub-steps: Each frame in the image frame sequence is segmented to identify and segment all independent object instances in the image, and a pixel-level mask, a bounding box, and a category label are generated for each instance. For each identified object instance, a set of preset visual attribute features are extracted. The visual attribute features include color histogram, texture descriptor, scale-invariant feature transform descriptor, and centroid coordinates and normalized area of ​​the instance in the image coordinate system. Based on the three-dimensional geospatial coordinates and three-axis attitude information, combined with the camera intrinsic parameter model, the approximate three-dimensional position and orientation of each object instance in the world coordinate system are calculated. Construct a relational reasoning network, input the category labels, visual attribute features, and three-dimensional spatial positions of all object instances, and reason and output the predefined spatial and semantic relationships between object instances. The spatial relationships include left, right, above, below, front, back, containment, and intersection. The semantic relationships include support, occlusion, orientation, and accompaniment. By treating all object instances as nodes in the graph and the inferred relationships as directed edges connecting the nodes, a complete scene semantic graph data structure describing the current scene is generated.

[0009] In one embodiment of the present invention, the composition aesthetics quantitative evaluation model is a deep neural network model, the internal structure of which includes an image feature extraction backbone network, a graph feature embedding network, and a multimodal feature fusion and regression network. The image feature extraction backbone network adopts a convolutional neural network structure to extract high-dimensional visual feature maps from the real-time image frame sequence. The graph feature embedding network adopts a graph convolutional network structure to learn the embedding representations of nodes and edges in the scene semantic graph. The multimodal feature fusion and regression network fuses the visual feature maps and graph embedding representations through a cross-attention mechanism, and finally outputs the quantitative aesthetic score and the composition optimization gradient vector through a fully connected layer.

[0010] Furthermore, the training process of the quantitative evaluation model for sentence structure aesthetics includes: Construct a dataset containing at least one million photographic works with professional aesthetic rating labels. Each work is scored from one to ten by no fewer than ten professional photographers, and the average score is used as the ground truth label. For each image in the dataset, the steps of constructing the scene semantic graph are performed offline to generate the corresponding scene semantic graph; The images and their corresponding scene semantic graphs are used as training sample pairs and input into the model. The difference between the quantized aesthetic score output by the model and the true label is calculated using the mean squared error loss function, and the model parameters are updated by backpropagation using the gradient descent optimization algorithm until the model converges.

[0011] The composition optimization gradient vector is a six-dimensional vector, which corresponds to the translation adjustment amount of the camera along the three axes of front-back, left-right, and up-down in three-dimensional space, as well as the rotation adjustment amount around these three axes. This vector indicates the direction and magnitude of the camera pose adjustment that needs to be made in order to improve the aesthetic score as quickly as possible.

[0012] As one embodiment of the present invention, generating a set of dynamic visual servoing guidance instructions specifically includes: The six-dimensional graph optimization gradient vector is decomposed into translation and rotation components; The translation component is mapped to a two-dimensional arrow marker at the center of the viewfinder. The direction of the arrow indicates the direction in which the camera needs to translate, and the length and transparency of the arrow are proportional to the magnitude of the required translation. The rotation component is mapped to a set of arc-shaped arrow markers on the edge of the viewfinder, and the direction and curvature of the arc-shaped arrows indicate the direction and angle in which the camera needs to rotate. When the quantitative aesthetic score is below a preset first threshold, the guidance instruction is displayed in a high-contrast color; when the score is above the first threshold but below the second threshold, the color contrast is reduced; when the score is above the second threshold, all guidance instructions disappear.

[0013] As one embodiment of the present invention, rendering the dynamic visual servoing guidance command into an augmented reality guidance layer further includes: In the scene semantic map, object instances that are identified as the main subject, leading lines, or framed foreground are identified; In the augmented reality guide layer, the outline of the main subject is highlighted; for objects identified as guide lines, such as roads, rivers, or railings, a flowing light effect curve is rendered to match them; and for objects identified as framed foreground objects, such as door frames or tree branches, a semi-transparent darkening process is applied to enhance the depth of the image and the visual focus.

[0014] According to another aspect of the present invention, an intelligent composition assistance system for travel photography is provided, comprising: The multimodal scene data acquisition module is configured to acquire, in real time, the image frame sequence, global geospatial coordinates of the device, and three-axis attitude information within the current field of view through an internally integrated image sensor, global navigation satellite system receiver, and inertial measurement unit. The scene semantic graph construction module is connected to the output end of the multimodal scene data acquisition module. It is used to receive multimodal scene data and transform the scene data into a structured scene semantic graph through built-in instance segmentation algorithm, feature extraction algorithm and relation reasoning network. The composition aesthetics quantitative evaluation module has its input end connected to the output end of the multimodal scene data acquisition module and the scene semantic graph construction module, respectively. It has a pre-trained deep neural network model embedded inside, which is used to jointly analyze the received image and graph, and calculate the quantitative aesthetic score of the current composition and a composition optimization gradient vector in real time. The dynamic servo guidance instruction generation module is connected to the output of the composition aesthetics quantification evaluation module and is used to analyze and transform the received composition optimization gradient vector into a set of specific camera pose adjustment parameters. The augmented reality guidance interface rendering module is connected to the output of the dynamic servo guidance instruction generation module and to the display screen of the device. It is used to visualize the camera pose adjustment parameters as a set of dynamic visual guidance symbols superimposed on the real-time viewfinder. By highlighting or darkening specific semantic elements, it can achieve closed-loop composition guidance for the user.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: First, it achieves a leap from static geometric assistance to dynamic semantic guidance. This invention constructs a scene semantic graph, enabling the system to understand what objects are in the scene and the relationships between them, rather than simply relying on indiscriminate grid lines, thereby providing truly intelligent composition suggestions with scene context awareness.

[0016] Second, this invention objectively quantifies and optimizes subjective photographic aesthetics. By introducing a deep learning model trained on a massive amount of professional works, it transforms abstract compositional aesthetics into calculable quantitative scores and optimization gradients, providing clear and actionable optimization goals and paths for composition assistance.

[0017] Third, it significantly reduces the cognitive and operational burden on users. By generating intuitive visual servo guidance instructions, this invention simplifies the complex process of composition adjustment into a simple operation of following on-screen instructions. Users do not need professional photography knowledge to quickly complete high-quality compositions in dynamic and complex travel environments.

[0018] Fourth, it provides an immersive and contextualized augmented reality interactive experience. By directly rendering dynamic guide icons and semantic highlighting effects in the viewfinder, this invention seamlessly integrates composition guidance into the shooting process, which not only enhances the intuitiveness of the guidance but also increases the fun and creativity of photography. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the intelligent composition assistance system for tourism photography proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the scene semantic graph construction and composition aesthetics quantitative evaluation model in this invention; Figure 3 This is a flowchart illustrating the logical flow of the intelligent composition assistance method for tourism photography proposed in this invention. Figure 4 This is a schematic diagram of the data flow between the terminal device and each functional module in this invention. Detailed Implementation

[0020] This invention provides an intelligent composition assistance method and system for tourism photography. Its core lies in achieving a closed-loop assistance system for users to compose high-quality images in complex tourism scenarios through four stages: multimodal perception, semantic understanding, aesthetic quantitative evaluation, and dynamic visual servo guidance. The following will describe in detail the specific implementation steps of this method, and simultaneously disclose the system structure supporting its operation.

[0021] like Figure 3 As shown, the method includes the following steps: S1, acquiring multimodal scene data in real time; S2, constructing a scene semantic map based on the multimodal scene data; S3, inputting the real-time image frame sequence and the scene semantic map into a preset composition aesthetics quantitative evaluation model, and outputting the quantitative aesthetic score of the current composition and the composition optimization gradient vector; S4, generating a set of dynamic visual servo guidance instructions based on the composition optimization gradient vector; S5, rendering the dynamic visual servo guidance instructions as an augmented reality guidance layer and overlaying it on the real-time viewfinder interface.

[0022] Combination Figure 2As shown, in step S1, multimodal scene data is acquired in real time. This multimodal scene data includes an image frame sequence within the current field of view, the device's global geospatial coordinates, the device's three-axis attitude information, and time information. Specifically, the device's built-in image sensor module continuously acquires raw image data with a resolution of at least 4,000 x 3,000 pixels at a frame rate of at least 30 frames per second, forming an image frame sequence. This image sensor module uses a back-illuminated complementary metal-oxide-semiconductor image sensor, which possesses high dynamic range and low noise characteristics, ensuring clear image frames can be acquired in both strong and low light environments. Simultaneously, the device's global navigation satellite system receiver module acquires the device's current longitude, latitude, and altitude, forming three-dimensional geospatial coordinates. This receiver module supports multi-system joint positioning using BeiDou, Global Positioning System, Galileo, and GLONASS, with a positioning accuracy better than two meters. Furthermore, the device's inertial measurement unit module acquires the device's current three-axis attitude information, consisting of roll, pitch, and yaw angles, as well as angular velocity information along these three axes. The inertial measurement unit (IMU) module is composed of a three-axis gyroscope and a three-axis accelerometer, with a sampling frequency of no less than 200 Hz. It uses a Kalman filter algorithm to denoise and calculate the attitude of the raw data, outputting stable and reliable attitude parameters. Finally, the current Gregorian calendar date and local time are obtained through the device's system clock module for subsequent time context modeling and scene lighting state inference. All the above data streams are timestamped in the device's main control processor to ensure strict synchronization of each modality's data within a millisecond-level time window, forming a unified multimodal scene data packet.

[0023] In step S2, a scene semantic graph is constructed based on the multimodal scene data. The scene semantic graph is structured data used to describe visual elements within a scene and their interrelationships, where nodes represent visual elements and edges represent a set of predefined relationships between visual elements. This step specifically includes the following sub-steps: First, instance segmentation is performed on each frame of the image frame sequence. This instance segmentation is executed by a deep learning model based on a mask-based convolutional neural network. This model is pre-trained on the Microsoft COCO dataset and has undergone domain-adaptive fine-tuning for typical objects in tourism scenes, such as buildings, people, vehicles, and natural landscapes. The model outputs a pixel-level mask, bounding box, and category label for each identified object instance. The category labels cover 128 predefined tourism-related semantic categories. Second, a set of predefined visual attribute features is extracted for each identified object instance. The visual attribute features include a color histogram, which is divided into sixteen hue intervals, eight saturation intervals, and eight brightness intervals in the red-green-blue color space, forming a 1024-dimensional histogram vector; a texture descriptor, calculated at multiple scales using a local binary mode operator, forming a 512-dimensional joint distribution vector; a scale-invariant feature transform descriptor, which extracts no fewer than fifty key points from the mask region of each instance and generates a 128-dimensional descriptor for each key point, ultimately aggregating them into a 256-dimensional global texture feature through clustering; and the centroid coordinates and normalized area of ​​the instance in the image coordinate system, where the normalized area is the number of instance pixels divided by the total number of image pixels. Furthermore, based on the three-dimensional geospatial coordinates and three-axis pose information, combined with the camera intrinsic parameter model, the approximate three-dimensional position and orientation of each object instance in the world coordinate system are calculated. The camera intrinsic parameter model includes focal length, principal point coordinates, and distortion coefficients, which are pre-calibrated using the Zhang Zhengyou calibration method and stored in the system memory. Through inverse perspective projection transformation, the centroid of instances in the image coordinate system is back-projected to a device-centric world coordinate system, and combined with geospatial coordinates for global coordinate transformation, resulting in a 3D position estimate for each instance. For distant objects whose depth cannot be accurately estimated using monocular vision, a digital elevation model and open street view map data are introduced for auxiliary localization. Subsequently, a relational reasoning network is constructed. This network is a multilayer perceptron structure, whose inputs are the category label embedding vectors, visual attribute feature vectors, and 3D spatial position vectors of all object instances. The output is the probability of whether a predefined relationship exists between each pair of instances. The predefined relationships include eight spatial relationships: left, right, above, below, in front, behind, containment, and intersection; and four semantic relationships: support, occlusion, orientation, and accompaniment. The relational reasoning network traverses all instance pairs, calculates the relationship score between each pair, and sets a threshold of 0.7. Relationships higher than this threshold are considered valid.Finally, all object instances are treated as nodes in the graph, and the inferred valid relationships are used as directed edges connecting the nodes, thus generating a complete scene semantic graph data structure. This graph is stored in memory in the form of an adjacency matrix and a list of node attributes for subsequent modules to access.

[0024] In step S3, the real-time image frame sequence and the scene semantic graph are input into a preset composition aesthetic quantification evaluation model. The model outputs a quantified aesthetic score for the current composition and a composition optimization gradient vector to improve the score. The composition aesthetic quantification evaluation model is a deep neural network model, whose internal structure includes three core components: an image feature extraction backbone network, a graph feature embedding network, and a multimodal feature fusion and regression network. The image feature extraction backbone network adopts the EfficientNet-B4 architecture, which is pre-trained on the ImageNet dataset. The input is the current image frame, and the output is a 7x7x1792-dimensional high-dimensional visual feature map. The graph feature embedding network adopts a graph convolutional network structure, containing three graph convolutional layers, each followed by batch normalization and modified linear unit activation functions. This network takes the adjacency matrix and node feature matrix of the scene semantic graph as input, aggregates neighbor node information through a message passing mechanism, and finally generates a 256-dimensional embedding vector for each node. Global average pooling is then used to obtain a 512-dimensional embedding representation of the entire graph. The multimodal feature fusion and regression network first flattens the visual feature map into a 12,544-dimensional vector, then concatenates it with the graph embedding representation to form a 13,056-dimensional joint feature vector. This joint feature vector is input into a fusion module consisting of a cross-attention mechanism, where visual features serve as keys and values, and graph embeddings serve as queries. The relevance weights between the two are calculated to achieve semantically guided reweighting of visual features. The fused feature vector passes through two fully connected layers. The first layer outputs 1,024 dimensions, and the second layer outputs seven dimensions. The first six dimensions constitute the composition optimization gradient vector, and the seventh dimension, after sigmoid activation, is multiplied by a ten-point scale to obtain a quantified aesthetic score. The composition optimization gradient vector is a six-dimensional vector, corresponding to the translational adjustments of the camera along the front-back, left-right, and up-down axes in three-dimensional space, as well as the rotational adjustments around the roll, pitch, and yaw axes. The physical meaning of this gradient vector is that, under the current composition state, a small adjustment of the camera pose along this direction can maximize the instantaneous increase in the aesthetic score. The model training process is based on a dataset containing 1.2 million professional photographs. Each photograph is independently scored from 1 to 10 by twelve nationally certified photographers, and the average score is used as the ground truth label. For each training image, the scene semantic map construction process described in step S2 is executed offline to generate the corresponding map. During training, the mean squared error loss function is used, with the loss value being the square of the difference between the model's output score and the ground truth label. L2 regularization is applied to the gradient vector. The optimizer uses the AdamW algorithm with an initial learning rate of 0.0001, a batch size of 32, and 50 training cycles until the validation set loss converges.

[0025] In step S4, a set of dynamic visual servo guidance instructions are generated based on the composition optimization gradient vector. This step first decomposes the six-dimensional composition optimization gradient vector into translation and rotation components. The translation component is the first three dimensions, corresponding to the X, Y, and Z axes in the world coordinate system; the rotation component is the last three dimensions, corresponding to the Euler angle changes around the X, Y, and Z axes. Since the user operation occurs in a two-dimensional viewfinder, the three-dimensional gradient needs to be mapped to two-dimensional guidance symbols. For the translation component, it is projected onto the image plane: translation along the Z axis (forward and backward) is mapped to a concentric circle scaling animation at the center of the viewfinder; an increase in the radius of the circle indicates that it needs to move closer to the subject, and a decrease indicates that it needs to move away. Translation along the X axis (left and right) and the Y axis (up and down) is mapped to a two-dimensional arrow marker at the center of the viewfinder. The direction of the arrow is determined by the ratio of the X and Y components. The length and transparency of the arrow are proportional to the required translation amplitude; the larger the amplitude, the longer and less transparent the arrow. For the rotation component, it is decomposed into three independent rotation commands: rotation around the Z-axis (yaw) is mapped to a pair of horizontal arc arrows at the top and bottom edges of the viewfinder, indicating left and right rotation; rotation around the X-axis (roll) is mapped to a pair of vertical arc arrows at the left and right edges, indicating clockwise or counterclockwise tilt; rotation around the Y-axis (pitch) is mapped to a pair of vertical arc arrows at the top and bottom edges, indicating up and down tilt. The radius of curvature of all arc arrows is inversely proportional to the rotation angle; the larger the angle, the smaller the curvature, and the straighter the arrow. Furthermore, the visual intensity of the guidance commands is dynamically adjusted based on the current quantitative aesthetic score: when the score is below six, all guidance indicators are displayed in high-contrast red; when the score is between six and eight, the color changes to orange, and the contrast is reduced by forty percent; when the score is above eight, all guidance commands disappear completely, indicating that the composition has reached its aesthetic optimum.

[0026] In step S5, the dynamic visual servo guidance command is rendered as an augmented reality guidance layer and overlaid on the real-time viewfinder. This rendering process includes not only the geometric guidance symbols in step S4, but also semantic highlighting of specific elements in the scene semantic map. Specifically, in the scene semantic map, object instances that are determined to be the main subject, guiding lines, or framed foreground are identified by preset rules. The rules for determining the main subject are: the instance category belongs to core tourist objects such as "people," "famous buildings," and "landmark sculptures," and its normalized area is greater than 5%, while having the most "facing" or "accompanying" relationships in the map. The rules for determining the guiding line are: the instance category is "road," "river," "railing," or "coastline," and its shape is long and thin with an aspect ratio greater than 5, and it shows a clear converging or extending trend in the image. The rules for determining the framed foreground are: the instance is located in the edge area of ​​the image, the category is "door frame," "window frame," "tree branch," or "archway," and its mask forms a closed or semi-closed ring structure. In the augmented reality guide layer, the outline of the main subject is highlighted with a five-pixel wide, cyan color, and 70% opacity. For objects identified as guide lines, a flowing light effect curve is rendered that fits them. This curve is generated along the center line of the instance, with white light effect and a gradient of transparency from the start to the end, flowing along the curve at a speed of 10 centimeters per second. For objects identified as framed foreground objects, a semi-transparent darkening effect is applied, specifically by overlaying a black mask within its mask area with 30% opacity, to reduce their visual weight and enhance the sense of depth and focus. All augmented reality elements are rendered in real-time using the device's graphics processor's fragment shader, ensuring seamless integration with the underlying image frames with a latency of no more than 30 milliseconds.

[0027] like Figure 1 and Figure 4As shown, the system supporting the above method comprises five functional modules: a multimodal scene data acquisition module, a scene semantic graph construction module, a composition aesthetics quantitative evaluation module, a dynamic servo guidance instruction generation module, and an augmented reality guidance interface rendering module. The multimodal scene data acquisition module is integrated into the device hardware layer, directly driving the image sensor, global navigation satellite system receiver, inertial measurement unit, and system clock, responsible for the acquisition and initial synchronization of raw data. The scene semantic graph construction module is deployed on a dedicated neural network acceleration unit of the device's application processor, calling instance segmentation models, feature extraction algorithms, and relational inference networks to complete the conversion from raw images to a structured graph. The composition aesthetics quantitative evaluation module has a pre-trained deep neural network model optimized in TensorRT format, enabling inference speeds of 15 frames per second on mobile devices. The dynamic servo guidance instruction generation module is a lightweight logic parser that receives gradient vectors and generates two-dimensional guidance parameters according to preset mapping rules. The augmented reality guidance interface rendering module interfaces with the device's graphics driver interface, receiving guidance parameters and semantic highlighting instructions, and calling OpenGL ES or Vulkan APIs to complete layer composition and display. The modules exchange data through shared memory and message queues to ensure that the end-to-end processing latency is controlled within 200 milliseconds, meeting the requirements for real-time interaction.

[0028] In summary, this embodiment constructs a complete intelligent composition assistance loop through rigorous multimodal data fusion, structured scene semantic modeling, quantifiable aesthetic evaluation, and intuitive augmented reality guidance. This solution not only addresses the fundamental shortcomings of traditional composition assistance technologies, such as lack of semantic understanding and dynamic response capabilities, but also enables ordinary users to achieve high-quality compositions that conform to aesthetic principles in complex and ever-changing travel scenarios without requiring specialized knowledge by encoding professional photographic aesthetic knowledge into a computable model.

[0029] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A smart composition assistance method for tourism photography, characterized in that, include: Real-time acquisition of multimodal scene data, including image frame sequences within the current field of view, global geospatial coordinates of the device, three-axis attitude information of the device, and time information; Based on the multimodal scene data, a scene semantic graph is constructed. The scene semantic graph is a structured data used to describe the visual elements in a scene and their interrelationships, where nodes represent visual elements and edges represent a set of preset relationships between visual elements. The image frame sequence and the scene semantic map are input into a preset composition aesthetic quantification evaluation model. The model outputs the quantification aesthetic score of the current composition and a composition optimization gradient vector to improve the score. Based on the optimized gradient vector of the composition, a set of dynamic visual servo guidance instructions are generated, which indicate the pose and focal length parameters that the camera needs to adjust in order to achieve a higher aesthetic score. The dynamic visual servo guidance command is rendered as an augmented reality guidance layer and overlaid on the live view interface to guide the user in operating the camera.

2. The intelligent composition assistance method for tourism photography according to claim 1, characterized in that, The real-time acquisition of multimodal scene data includes: The device's image sensor module continuously acquires raw image data with a resolution of no less than 4,000 by 3,000 pixels at a frame rate of no less than 30 frames per second, forming an image frame sequence. The device's three-dimensional geographic coordinates, consisting of longitude, latitude, and altitude, are obtained through the device's global navigation satellite system receiver module. The inertial measurement unit module of the equipment obtains the three-axis attitude information consisting of the current roll angle, pitch angle, and yaw angle, as well as the angular velocity information along these three axes. The current Gregorian calendar date and local time are obtained through the device's system clock module.

3. The intelligent composition assistance method for tourism photography according to claim 2, characterized in that, The construction of the scene semantic graph includes: Each frame in the image frame sequence is segmented to identify and segment all independent object instances in the image, and a pixel-level mask, a bounding box, and a category label are generated for each instance. For each identified object instance, a set of preset visual attribute features are extracted. The visual attribute features include color histogram, texture descriptor, scale-invariant feature transform descriptor, and centroid coordinates and normalized area of ​​the instance in the image coordinate system. Based on the three-dimensional geospatial coordinates and three-axis attitude information, combined with the camera intrinsic parameter model, the approximate three-dimensional position and orientation of each object instance in the world coordinate system are calculated. Construct a relational reasoning network, input the category labels, visual attribute features, and three-dimensional spatial positions of all object instances, and reason and output the predefined spatial and semantic relationships between object instances. The spatial relationships include left, right, above, below, front, back, containment, and intersection. The semantic relationships include support, occlusion, orientation, and accompaniment. By treating all object instances as nodes in the graph and the inferred relationships as directed edges connecting the nodes, a complete scene semantic graph data structure is generated.

4. The intelligent composition assistance method for tourism photography according to claim 1, characterized in that, The composition aesthetics quantitative evaluation model is a deep neural network model, whose internal structure includes an image feature extraction backbone network, a graph feature embedding network, and a multimodal feature fusion and regression network. The image feature extraction backbone network adopts a convolutional neural network structure to extract high-dimensional visual feature maps from the real-time image frame sequence. The graph feature embedding network adopts a graph convolutional network structure to learn the embedding representations of nodes and edges in the scene semantic graph. The multimodal feature fusion and regression network fuses the visual feature maps and graph embedding representations through a cross-attention mechanism, and finally outputs the quantitative aesthetic score and the composition optimization gradient vector through a fully connected layer.

5. The intelligent composition assistance method for tourism photography according to claim 4, characterized in that, The composition optimization gradient vector is a six-dimensional vector, which corresponds to the translation adjustment amount of the camera along the three axes of front-back, left-right, and up-down in three-dimensional space, as well as the rotation adjustment amount around these three axes. This vector indicates the direction and magnitude of the camera pose adjustment that needs to be made in order to improve the aesthetic score as quickly as possible.

6. The intelligent composition assistance method for tourism photography according to claim 5, characterized in that, The generation of a set of dynamic visual servo guidance instructions includes: The six-dimensional graph optimization gradient vector is decomposed into translation and rotation components; The translation component is mapped to a two-dimensional arrow marker at the center of the viewfinder. The direction of the arrow indicates the direction in which the camera needs to translate, and the length and transparency of the arrow are proportional to the magnitude of the required translation. The rotation component is mapped to a set of arc-shaped arrow markers on the edge of the viewfinder, and the direction and curvature of the arc-shaped arrows indicate the direction and angle in which the camera needs to rotate. When the quantitative aesthetic score is below a preset first threshold, the guidance instruction is displayed in a high-contrast color; when the score is above the first threshold but below the second threshold, the color contrast is reduced; when the score is above the second threshold, all guidance instructions disappear.

7. The intelligent composition assistance method for tourism photography according to claim 6, characterized in that, The step of rendering the dynamic visual servoing guidance command into an augmented reality guidance layer also includes: In the scene semantic map, object instances that are identified as the main subject, leading lines, or framed foreground are identified; In the augmented reality guide layer, the outline of the main subject is highlighted; for objects identified as guide lines, such as roads, rivers, or railings, a flowing light effect curve is rendered to match them; and for objects identified as framed foreground objects, such as door frames or tree branches, a semi-transparent darkening process is applied to enhance the depth of the image and the visual focus.

8. The intelligent composition assistance method for tourism photography according to claim 7, characterized in that, The rules for determining the main subject are as follows: the instance category belongs to people, famous buildings or landmark sculptures, and its normalized area is greater than 5%, and it has the most orientation or accompanying relationship in the image. The rules for determining the leading line are as follows: the instance category is roads, rivers, railings or coastlines, and its shape is long and thin with an aspect ratio greater than 5, and it shows a clear convergence or extension trend in the image. The rules for determining the framed foreground are as follows: the instance is located in the edge area of ​​the image, the category is door frame, window frame, tree branch or arch, and its mask forms a closed or semi-closed ring structure.

9. An intelligent composition assistance system for tourism photography, characterized in that, include: The multimodal scene data acquisition module is used to acquire, in real time, the image frame sequence, global geospatial coordinates of the device, and three-axis attitude information within the current field of view through the internally integrated image sensor, global navigation satellite system receiver, and inertial measurement unit. The scene semantic graph construction module is used to receive multimodal scene data and transform the scene data into a structured scene semantic graph through built-in instance segmentation algorithm, feature extraction algorithm and relation reasoning network; The composition aesthetics quantitative evaluation module has a pre-trained deep neural network model embedded inside, which is used to jointly analyze the received image and map, and calculate the quantitative aesthetic score of the current composition and a composition optimization gradient vector in real time. The dynamic servo guidance command generation module is used to parse and transform the received composition optimization gradient vector into a set of specific camera pose adjustment parameters. The augmented reality guidance interface rendering module is used to visualize the camera pose adjustment parameters as a set of dynamic visual guidance symbols superimposed on the real-time viewfinder, and to achieve closed-loop composition guidance for the user by highlighting or darkening specific semantic elements.

10. The intelligent composition assistance system for tourism photography according to claim 9, characterized in that, The scene semantic graph construction module is used for: Each frame in the image frame sequence is segmented to identify and segment all independent object instances in the image, and a pixel-level mask, a bounding box, and a category label are generated for each instance. For each identified object instance, a set of preset visual attribute features are extracted. The visual attribute features include color histogram, texture descriptor, scale-invariant feature transform descriptor, and centroid coordinates and normalized area of ​​the instance in the image coordinate system. Based on the three-dimensional geospatial coordinates and three-axis attitude information, combined with the camera intrinsic parameter model, the approximate three-dimensional position and orientation of each object instance in the world coordinate system are calculated. Construct a relation reasoning network, take the category labels, visual attribute features and three-dimensional spatial positions of all object instances as input, and reason and output the predefined spatial and semantic relationships between object instances; By treating all object instances as nodes in the graph and the inferred relationships as directed edges connecting the nodes, a complete scene semantic graph data structure is generated.

Images