Realtime-Transformer-based real-time guidance method, system and electronic equipment for photographic composition

Abstract

This invention discloses a real-time photography composition guidance method, system, and electronic device based on Realtime-Transformer, comprising the following steps: acquiring the camera view as the input image in real time; inputting the input image into a pre-trained composition rule classification model for multi-scale feature map extraction, and predicting the composition rule category of the input image based on the multi-scale feature maps; inputting the input image into a pre-trained RT-DETR model for original feature extraction, and performing stacking and filling prediction on the extracted original features to obtain the boundary expansion features of the original image; based on the original features and the boundary expansion features, predicting and generating a composition box by jointly using the corresponding composition rule category; generating framing guidance based on the position coordinates of the composition box and the input image, and displaying it in real time in the camera view. This invention integrates composition principles into photography guidance, realizing frame prediction outside the boundaries of the original image, providing users with real-time camera adjustment suggestions, and enabling the capture of photos that meet the target aesthetic requirements.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and photography guidance technology, and in particular to a real-time photography composition guidance method, system and electronic device based on Realtime-Transformer. Background Technology

[0002] In photography, image composition plays a crucial role in the overall aesthetic appeal of a photograph. Traditional composition optimization methods primarily rely on post-processing techniques, such as image cropping, to highlight the visual focal point and improve the composition. However, these methods have limitations, mainly in that they can only operate within the boundaries of the original image and cannot extend beyond it. This limitation hinders photographers from achieving optimal composition during the shooting process and increases the burden of post-processing.

[0003] Existing photographic composition optimization technologies cannot provide users with real-time guidance when framing shots, nor can they predict and dynamically adjust compositional boundaries to obtain photos that better meet aesthetic standards. There is an urgent need for a new technological solution that can provide real-time composition suggestions during the shooting process and break the limitations of the original image boundaries.

[0004] Therefore, how to provide a real-time composition guidance method and system that can offer real-time composition suggestions during the shooting process and break the limitations of the original image boundaries is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This invention addresses the aforementioned research status and existing problems by providing a real-time photographic composition guidance method, system, and electronic device based on Realtime-Transformer. It utilizes composition rules to guide the RT-DETR model in predicting aesthetically pleasing compositions for photographs. The predicted composition frame can be extended beyond the current framing, providing dynamic and real-time guidance for image composition in photography, resulting in photographs that meet target aesthetic requirements.

[0006] This invention provides a real-time guidance method for photographic composition based on Realtime-Transformer, comprising the following steps:

[0007] Image acquisition: Real-time acquisition of the camera viewfinder as input image;

[0008] Composition rule classification: The input image is input to a pre-trained composition rule classification model, multi-scale feature map extraction is performed, and the composition rule category of the input image is predicted based on the multi-scale feature map;

[0009] Constructing bounding box prediction: The input image is input to a pre-trained RT-DETR model for original feature extraction, and the extracted original features are subjected to stacking and filling prediction to obtain the boundary expansion features of the original image. Based on the original features and the boundary expansion features, the corresponding construction rule category is used to predict and generate the construction bounding box of the input image.

[0010] Framing guide generation: A framing guide is generated based on the position coordinates of the composition frame and the input image, and the framing guide is displayed in real time in the viewfinder of the camera.

[0011] Preferably, the composition rule categories include: central composition, diagonal composition, horizontal line composition, symmetrical composition, triangular composition, vanishing point composition, vertical line composition, or regular rule of thirds composition.

[0012] Preferably, the composition rule classification model includes:

[0013] The VGG network is used to extract multi-scale features from the input image and generate multiple feature maps of different resolutions.

[0014] CNN convolutional layers are used to extract semantic features from each of the feature maps;

[0015] A global adaptive average pooling layer is used to reduce the dimensionality of the semantic features of each feature map;

[0016] The fully connected layer receives the semantic features of all feature maps after dimensionality reduction and outputs the classification results of the mapping rules.

[0017] Preferably, the RT-DETR model includes a boundary expansion model and a bounding box prediction model, and the pre-training steps of the RT-DETR model include:

[0018] Boundary expansion model pre-training:

[0019] Construct a cropping dataset, including multiple cropping data pairs, wherein each cropping data pair includes an original-size image and a local region image cropped from the original-size image;

[0020] The local region image is used as the input to the boundary expansion model, and the original size image is used as the output of the boundary expansion model for model weight training until the training stopping condition is met, thus obtaining a pre-trained boundary expansion model.

[0021] Pre-training of the bounding box prediction model:

[0022] Construct a bounding box dataset, which includes multiple sets of framed images labeled with bounding boxes;

[0023] The composition image is used as the input to the composition box prediction model, and the position coordinates of the composition box are used as the output of the composition box prediction model to train the model weights until the training stopping condition is met, thus obtaining a pre-trained composition box prediction model.

[0024] Preferably, the composition box dataset includes multiple sets of images with composition box annotations, composition line annotations, and composition scene object annotations, wherein the composition scene object is an object element in the image.

[0025] Preferably, the step of generating framing guidance based on the position coordinates of the composition frame and the input image includes:

[0026] Obtain the center point coordinates of the frame and the input image, calculate the distance vector difference between the two center point coordinates, and generate a lens movement guide symbol based on the distance vector difference, which is displayed in real time in the viewfinder of the camera; and / or,

[0027] The dimensions of the composition frame and the input image are obtained, the scaling vector difference between the two dimensions is calculated, and a lens scaling guide symbol is generated based on the scaling vector difference and displayed in real time in the viewfinder of the camera.

[0028] Preferably, it also includes a step of generating icon annotations, wherein the icon annotations include icon lines:

[0029] The input image is detected by depth Hough transform to obtain several semantic lines;

[0030] Calculate the contrast of the image pixels on both sides of each semantic line, and filter all semantic lines based on the contrast to obtain the composition lines;

[0031] The composition lines are displayed in real time at a designated position in the viewfinder of the camera.

[0032] Preferably, the designated position for displaying the composition line includes the pixel area where the corresponding semantic line is located, and / or, a guide box is displayed at the designated position in the viewfinder, and the guide box displays the corresponding proportional position of the composition line in the viewfinder.

[0033] Preferably, it also includes a step of generating composition annotations, which include markers for composition scene objects:

[0034] The input image is used to detect the composition scene objects in the image using the YOLO model;

[0035] The objects in the framed scene of the camera are marked and displayed in real time.

[0036] This invention also provides a real-time photography composition guidance system based on Realtime-Transformer, comprising:

[0037] The image acquisition module is used to acquire the camera viewfinder image in real time as the input image;

[0038] The composition rule classification module is used to input the input image into a pre-trained composition rule classification model, extract multi-scale feature maps, and predict the composition rule category of the input image based on the multi-scale feature maps.

[0039] The bounding box prediction module is used to input the input image into a pre-trained RT-DETR model to extract the original features, and to perform stacking and filling prediction on the extracted original features to obtain the boundary expansion features of the original image. Based on the original features and the boundary expansion features, the module predicts and generates the bounding box of the input image by combining the corresponding composition rule category.

[0040] The framing guide generation module is used to generate framing guides based on the position coordinates of the composition frame and the input image, and the framing guides are displayed in real time in the viewfinder of the camera.

[0041] The present invention also provides an electronic device, comprising:

[0042] At least one processor; and

[0043] A memory and a camera communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that are executed by the at least one processor, which enable the at least one processor to execute the Realtime-Transformer-based photographic composition guidance method to generate framing instructions;

[0045] The camera is used to perform framing actions and to receive and display framing instructions sent by the processor in real time.

[0046] Compared with existing technologies, it has the following advantages:

[0047] Breaking Image Boundary Constraints: The system can predict and adjust composition boxes that extend beyond the original image boundaries. Guided by rules and annotations, the model can explore potential compositions that go beyond the initial image constraints, enhancing the flexibility and creativity of the composition process.

[0048] Real-time interaction: Unlike traditional post-processing methods, this invention can provide composition suggestions in real time during the shooting process, helping users to create photographic works that are more in line with compositional aesthetics and reducing the workload of post-processing.

[0049] Multi-label classification: The system supports multiple composition types and can label composition elements in real time, providing explanatory support for the composition category and offering users more professional and comprehensive composition guidance.

[0050] Enhanced User Experience: Real-time Camera Adjustment Guidance provides actionable guidance for real-time camera adjustments based on predicted composition frames. Users receive suggestions to reposition or zoom the camera to capture the most aesthetically pleasing images, allowing them to focus more on the creative process and significantly improving shooting efficiency and image quality. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention, and those skilled in the art can obtain other drawings based on the provided drawings without creative effort.

[0052] Figure 1 Flowchart of a real-time photography composition guidance method based on Realtime-Transformer provided in an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the composition rule categories provided in the embodiments of the present invention;

[0054] Figure 3 A schematic diagram illustrating the principle of generating a composition box based on boundary extension features, as provided in an embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram illustrating the camera viewfinder guidance effect provided in an embodiment of the present invention. Figure 1 ;

[0056] Figure 5 This is a schematic diagram illustrating the camera viewfinder guidance effect provided in an embodiment of the present invention. Figure 2 ;

[0057] Figure 6 This is a schematic diagram illustrating the camera viewfinder guidance effect provided in an embodiment of the present invention. Figure 3 ;

[0058] Figure 7 A comparison image of the viewfinder before and after adjustment according to the instructions, provided in an embodiment of the present invention;

[0059] Figure 8 Comparison of the effects of the composition line marking provided in the embodiments of the present invention Figure 1 ;

[0060] Figure 9 Comparison of the effects of the composition line marking provided in the embodiments of the present invention Figure 2 . Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] This invention combines deep learning models with composition rules to provide users with real-time camera adjustment suggestions, helping them take photos that better meet professional composition standards and overcoming the limitations of existing technologies.

[0063] like Figure 1 As shown, the first aspect of this invention discloses a real-time guidance method for photographic composition based on Realtime-Transformer, comprising the following steps:

[0064] Image acquisition: Real-time acquisition of the camera viewfinder as input image;

[0065] Composition rule classification: The input image is fed into a pre-trained composition rule classification model, multi-scale feature map extraction is performed, and the composition rule category of the input image is predicted based on the multi-scale feature map;

[0066] Boundary box prediction: The input image is fed into a pre-trained RT-DETR model for original feature extraction, and the extracted original features are subjected to stacking and filling prediction to obtain the boundary expansion features of the original image. Based on the original features and the boundary expansion features, the corresponding composition rule category is combined to predict and generate the bounding box of the input image.

[0067] Framing guide generation: A framing guide is generated based on the position coordinates of the composition frame and the input image, and the framing guide is displayed in real time in the camera's viewfinder.

[0068] It should be noted that this method is performed during the camera framing stage before taking a picture. The user adjusts the camera position according to the framing guide. After reaching the position indicated by the composition frame, the user can take a picture of the current frame and obtain a photo with the adjusted composition.

[0069] In one embodiment, such as Figure 2 As shown in (a)-(h), the composition rules are categorized in the following order: central composition, diagonal composition, horizontal composition, symmetrical composition, triangular composition, vanishing point composition, vertical line composition, or rule of thirds composition. The method in this embodiment can flexibly identify one or more composition rules that an input image may conform to.

[0070] In one embodiment, the graph rule classification model includes:

[0071] VGG network is used to extract multi-scale features from input image and generate multiple feature maps of different resolutions;

[0072] CNN convolutional layers are used to extract semantic features from each feature map;

[0073] A global adaptive average pooling layer is used to reduce the dimensionality of the semantic features of each feature map;

[0074] The fully connected layer receives the semantic features of all feature maps after dimensionality reduction and outputs the classification results of the mapping rules.

[0075] This embodiment uses the VGG network as the basic model for feature extraction and combines it with a multi-label classification method to analyze the input image. The specific process is as follows:

[0076] First, the system extracts multi-scale features from the image using the VGG16 model, generating multiple feature maps of different resolutions. These maps are then processed by a convolutional neural network (CNN), including multiple convolutions, batch normalization, and the ReLU activation function, to extract higher-level semantic features. After feature extraction, a global adaptive average pooling layer is used to reduce the dimensionality of the feature maps, and a fully connected layer outputs classification results for eight different mapping rules.

[0077] In one embodiment, the RT-DETR (Real-Time Detection Transformer) model is used to predict the best frame in an image, guiding the user to adjust the camera position in real time to achieve the best photographic composition.

[0078] The input to the RT-DETR model is a preprocessed RGB image, typically a fixed-size image (e.g., 224x224 pixels). After feature extraction via convolutional layers, the image is fed into the Transformer module for global feature analysis and processing.

[0079] The output of the RT-DETR model is a bounding box in the input image and a category label for the composition rule. The bounding box is described by four coordinate points (x, y, w, h), representing the frame in the image that best conforms to the composition rules, helping users adjust the camera's angle, zoom level, etc., based on system suggestions.

[0080] In one embodiment, the RT-DETR model includes a boundary expansion model and a bounding box prediction model, wherein, as Figure 3As shown, the boundary expansion model is used to predict feature representations outside the image boundary. Through a multi-layer self-attention mechanism and a multi-head cross-attention mechanism, it significantly enhances the accuracy and computational efficiency of feature extrapolation. The specific execution process is as follows:

[0081] The input to the boundary expansion model consists of a visible feature V and a learnable query variable m.

[0082] First, a self-attention mechanism is used to calculate the internal correlation of visible features, capturing the global dependencies of the image.

[0083] In the decoder layer, a dynamic weight adjustment mechanism is innovatively introduced, enabling the network to adaptively adjust the weights of features at different levels according to the complexity of image features, thereby improving the accuracy and stability of extrapolated features.

[0084] After each decoder layer, the features are normalized through a phased Add & Norm layer to ensure the stability of the network during training while maintaining efficient feature transfer.

[0085] The boundary expansion model also employs a parallel feedforward network (FFN) structure. After each self-attention and cross-attention calculation, it performs fine-grained feature extraction and fusion on extrapolated features and visible features, which greatly improves the efficiency and quality of feature generation.

[0086] Finally, through multi-layer iterative calculations, a filling feature P is generated. Based on this, multi-head cross-attention is used to interactively calculate the visible feature V with the filling feature P generated in each iteration, inferring the feature representation outside the image boundary. This not only enhances the visual extensibility of the image but also provides more accurate and flexible support for image composition optimization, ensuring the realism and aesthetic effect of the image.

[0087] This embodiment employs a decoder layer for feature stacking and expansion, enabling the predicted bounding box to extend beyond the boundaries of the original image, thus achieving bounding box expansion. The system can predict potential compositions beyond the current field of view of the camera and generate bounding boxes that exceed the hardware framing range.

[0088] In this embodiment, the pre-training steps of the RT-DETR model include:

[0089] Boundary expansion model pre-training:

[0090] Construct a cropping dataset, which includes multiple cropping data pairs. Each cropping data pair includes an original-size image and a local region image cropped from the original-size image.

[0091] The local region image is used as the input to the boundary expansion model, and the original size image is used as the output of the boundary expansion model to train the model weights until the training stopping condition is met, thus obtaining the pre-trained boundary expansion model.

[0092] Pre-training of the bounding box prediction model:

[0093] Construct a bounding box dataset, which includes multiple sets of framed images labeled with bounding boxes;

[0094] The composition image is used as the input to the bounding box prediction model, and the position coordinates of the bounding box are used as the output of the bounding box prediction model. The model weights are trained until the training stopping condition is met, and a pre-trained bounding box prediction model is obtained.

[0095] The loss function in this training process combines positional loss (L1-Loss and IoU-Loss) and cross-entropy loss, training the model to simultaneously optimize the accuracy of bounding boxes and the classification of composition types. The training process uses gradient descent optimization algorithms (such as Adam or SGD) to continuously optimize the model parameters, enabling RT-DETR to effectively predict bounding boxes and composition rule categories.

[0096] In one embodiment, real-time images are captured by a camera, and these images are analyzed. Combined with predicted composition rules, an optimal composition bounding box is generated. Based on the user's current framing content, suggestions for camera movement, zoom, and other adjustments are provided. These suggestions are fed back to the viewfinder in real time, helping the user optimize the camera's angle, position, focal length, etc., before taking a photo to achieve the best composition. The steps for generating framing guidance based on the composition bounding box and the position coordinates of the input image include:

[0097] Obtain the center point coordinates of the composition frame and the input image, calculate the distance vector difference between the two center point coordinates, and generate a lens movement guide symbol based on the distance vector difference, which is displayed in real time in the camera's viewfinder; and / or, obtain the dimensions of the composition frame and the input image, calculate the scaling vector difference between the two dimensions, and generate a lens scaling guide symbol based on the scaling vector difference, which is displayed in real time in the camera's viewfinder.

[0098] The viewfinder provides more flexible shooting guidance by prompting users to move the camera in a certain direction to capture a wider scene or a more aesthetically pleasing composition. This process ensures that users can adjust their perspective based on real-time suggestions before taking the photo, ensuring the composition meets optimal aesthetic standards. Figure 4-6 As shown, framing guidance can take the form of signage, such as arrow symbols. Figure 4 The middle guides the user to move the viewfinder upwards. Figure 5 The middle guides the user to move the viewfinder to the left. Figure 6 The text guides users to zoom in and out of the viewfinder. For example... Figure 7 The image shown is a comparison of the viewfinder before and after adjustments made according to the instructions.

[0099] In one embodiment, the composition box dataset includes multiple sets of images with composition box annotations, composition line annotations, and composition scene object annotations, where the composition scene object is an object element in the image.

[0100] In one embodiment, the method further includes a step of generating icon annotations, wherein the icon annotations include icon lines:

[0101] The input image is processed by depth Hough transform to detect several semantic lines;

[0102] Calculate the contrast of the image pixels on both sides of each semantic line, and filter all semantic lines based on the contrast to obtain the composition lines;

[0103] The composition lines are displayed in real time at designated positions in the camera's viewfinder.

[0104] In practice, the composition lines are detected using an improved depth Hough transform, which includes semantic segmentation, depth Hough variation, and a contrast-based composition line selection method.

[0105] First, semantic segmentation is performed on the input image. Then, deep Hough detection is performed to identify all semantic lines. Finally, the contrast at both ends of each semantic line is calculated to filter all detected semantic lines and obtain the final composition lines.

[0106] In this embodiment, the designated location for displaying the composition line includes the pixel area where the corresponding semantic line is located, and / or, a guide box is displayed at a designated location in the viewfinder, and the guide box displays the corresponding proportional position of the composition line in the viewfinder.

[0107] like Figure 4-6 As shown, a guide box is displayed in the upper right corner of the viewfinder, and the target position of the horizontal composition line is displayed in the guide box at the corresponding proportional height.

[0108] like Figure 8 As shown, the composition lines are in the shape of a diagonal. Figure 8 (a) is the viewfinder of the input image. Figure 8 (b) The viewfinder image is the result of real-time adjustment of the camera position based on the generated diagonal composition lines. During the shooting process, the system analyzes the current perspective in real time and predicts candidate composition frames based on feature prediction. The user adjusts the camera position according to the instructions to capture a photo that conforms to the "diagonal composition" rule, ultimately improving the aesthetic score of the photo. Figure 8 The composition lines marked in real time in (b) provide interpretability, allowing users to understand the logic behind the adjustments.

[0109] like Figure 9 As shown, the composition lines are shaped like horizontal lines. Figure 9 (a) is the viewfinder of the input image. Figure 9 (b) The viewfinder image after real-time adjustment of the camera position based on the generated horizontal line shape composition lines. During the shooting process, the system analyzes key objects and, still combining feature prediction, generates the optimal composition frame, guiding the user to take photos that meet the "horizontal line composition" standard by horizontally moving the camera and appropriately zooming, thus obtaining high-quality visual effects.

[0110] In one embodiment, the method further includes a step of generating compositional annotations, which include markers for compositional scene objects, such as people and buildings.

[0111] The input image is used by the YOLO model to detect the composition scene objects in the image;

[0112] The camera can mark and display the objects in the frame of the scene in real time.

[0113] In practice, compositional elements are detected and labeled using the YOLOv11 model to identify the main content of the image. These annotations help users intuitively see the application of compositional rules when adjusting the shooting angle in real time, thereby guiding them to take photos that meet aesthetic standards.

[0114] In one embodiment, the method further includes an aesthetic scoring step of identifying one or more composition rule categories that the input image may conform to, selecting the composition frame with the highest aesthetic score, and displaying it to the user through the camera viewfinder. Figure 7 As shown, the image before adjustment according to the method of this embodiment has a score of 0.34, and the image after adjustment has a score of 0.53. It can be seen that the image obtained after adjustment is aesthetically superior.

[0115] The second aspect of the embodiment discloses a real-time photography composition guidance system based on Realtime-Transformer, comprising:

[0116] The image acquisition module is used to acquire the camera viewfinder image in real time as the input image;

[0117] The composition rule classification module is used to input the input image into a pre-trained composition rule classification model, extract multi-scale feature maps, and predict the composition rule category of the input image based on the multi-scale feature maps.

[0118] The bounding box prediction module is used to input the input image into the pre-trained RT-DETR model to extract the original features, and perform stacking and filling prediction on the extracted original features to obtain the boundary expansion features of the original image. Based on the original features and the boundary expansion features, the corresponding composition rule category is combined to predict and generate the bounding box of the input image.

[0119] The framing guide generation module is used to generate framing guides based on the position coordinates of the composition frame and the input image. The framing guides are displayed in real time in the camera's viewfinder.

[0120] The second aspect of the present invention can be used to perform all the steps disclosed in the first aspect of the present invention.

[0121] A third aspect of the embodiments discloses an electronic device, comprising:

[0122] At least one processor; and

[0123] A memory and a camera that are communicatively connected to at least one processor; wherein,

[0124] The memory stores instructions that are executed by at least one processor to enable the at least one processor to perform the Realtime-Transformer-based real-time photography composition guidance method disclosed in the first aspect of the embodiment, and to generate framing guidance;

[0125] The camera is used to perform framing actions and to receive and display framing instructions sent by the processor in real time.

[0126] The camera helps users understand the underlying principles behind suggested adjustments by loading a UI with real-time annotated framing guidelines and composition annotations. This real-time feedback mechanism ensures that users can effortlessly adhere to compositional principles, resulting in beautiful photos.

[0127] The foregoing has provided a detailed description of a real-time photography composition guidance method, system, and electronic device based on Realtime-Transformer provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0128] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A real-time guidance method for photographic composition based on Realtime-Transformer, characterized in that, Includes the following steps: Image acquisition: Real-time acquisition of the camera viewfinder as input image; Composition rule classification: The input image is input to a pre-trained composition rule classification model, multi-scale feature map extraction is performed, and the composition rule category of the input image is predicted based on the multi-scale feature map; Constructing bounding box prediction: The input image is input to a pre-trained RT-DETR model for original feature extraction, and the extracted original features are subjected to stacking and filling prediction to obtain the boundary expansion features of the original image. Based on the original features and the boundary expansion features, the corresponding construction rule category is used to predict and generate the construction bounding box of the input image. Framing guide generation: A framing guide is generated based on the position coordinates of the composition frame and the input image, and the framing guide is displayed in real time in the viewfinder of the camera.

2. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that, The composition rules include: central composition, diagonal composition, horizontal composition, symmetrical composition, triangular composition, vanishing point composition, vertical line composition, or regular rule of thirds composition.

3. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that, The composition rule classification model includes: The VGG network is used to extract multi-scale features from the input image and generate multiple feature maps of different resolutions. CNN convolutional layers are used to extract semantic features from each of the feature maps; A global adaptive average pooling layer is used to reduce the dimensionality of the semantic features of each feature map; The fully connected layer receives the semantic features of all feature maps after dimensionality reduction and outputs the classification results of the graph construction rules.

4. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that, The RT-DETR model includes a boundary expansion model and a bounding box prediction model. The pre-training steps of the RT-DETR model include: Boundary expansion model pre-training: Construct a cropping dataset, including multiple cropping data pairs, wherein each cropping data pair includes an original-size image and a local region image cropped from the original-size image; The local region image is used as the input to the boundary expansion model, and the original size image is used as the output of the boundary expansion model for model weight training until the training stopping condition is met, thus obtaining a pre-trained boundary expansion model. Pre-training of the bounding box prediction model: Construct a bounding box dataset, which includes multiple sets of bounding box-labeled images; The composition image is used as the input to the composition box prediction model, and the position coordinates of the composition box are used as the output of the composition box prediction model to train the model weights until the training stopping condition is met, thus obtaining a pre-trained composition box prediction model.

5. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 4, characterized in that, The composition box dataset includes multiple sets of images with composition box annotations and composition object annotations, where the composition object is the object element in the image.

6. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that, The step of generating framing guidance based on the composition frame and the position coordinates of the input image includes: Obtain the center point coordinates of the composition frame and the input image, calculate the distance vector difference between the two center point coordinates, and generate a lens movement guide symbol based on the distance vector difference, which is displayed in real time in the viewfinder of the camera; and / or, The dimensions of the composition frame and the input image are obtained, the scaling vector difference between the two dimensions is calculated, and a lens scaling guide symbol is generated based on the scaling vector difference and displayed in real time in the viewfinder of the camera.

7. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that, It also includes the steps for generating icon labels, which include icon lines: The input image is detected by depth Hough transform to obtain several semantic lines; Calculate the contrast of the image pixels on both sides of each semantic line, and filter all semantic lines based on the contrast to obtain the composition lines; The composition lines are displayed in real time at a designated position in the viewfinder of the camera.

8. The real-time guidance method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that, It also includes the step of generating composition annotations, which include markers for composition scene objects: The input image is used to detect the composition scene objects in the image using the YOLO model; The objects in the framed scene of the camera are marked and displayed in real time.

9. A real-time photography composition guidance system based on Realtime-Transformer, characterized in that, include: The image acquisition module is used to acquire the camera viewfinder image in real time as the input image; The composition rule classification module is used to input the input image into a pre-trained composition rule classification model, extract multi-scale feature maps, and predict the composition rule category of the input image based on the multi-scale feature maps. The bounding box prediction module is used to input the input image into a pre-trained RT-DETR model to extract the original features, and to perform stacking and filling prediction on the extracted original features to obtain the boundary expansion features of the original image. Based on the original features and the boundary expansion features, the module predicts and generates the bounding box of the input image by combining the corresponding composition rule category. The framing guide generation module is used to generate framing guides based on the position coordinates of the composition frame and the input image, and the framing guides are displayed in real time in the viewfinder of the camera.

10. An electronic device, comprising: At least one processor; as well as A memory and a camera communicatively connected to the at least one processor; wherein, The memory stores instructions that are executed by the at least one processor, which enable the at least one processor to execute the Realtime-Transformer-based real-time photography composition guidance method according to any one of claims 1-8 to generate framing guidance; The camera is used to perform framing actions and to receive and display framing instructions sent by the processor in real time.

Images