Realtime-Transform-based photographing composition real-time guiding method and system, and electronic equipment

Abstract

The invention discloses a photographing composition real-time guiding method and system based on Realtime-Transform and electronic equipment. The method comprises the following steps that a view-finding picture of a camera is obtained in real time to serve as an input image; inputting the input image into a pre-trained composition rule classification model, performing multi-scale feature map extraction, and predicting a composition rule category of the input image based on the multi-scale feature map; the method comprises the following steps: inputting an input image into a pre-trained RT-DETR model to carry out original feature extraction, carrying out accumulation filling prediction on the extracted original features to obtain boundary extension features of an original image, and generating a composition frame by combining corresponding composition rule category prediction based on the original features and the boundary extension features; and according to the composition frame and the position coordinate of the input image, generating a view-finding guide, and displaying the view-finding guide in a view-finding picture of the camera in real time. According to the method, the composition principle is integrated into photographing guidance, frame prediction outside the boundary of the original image is realized, camera adjustment suggestions are provided for users in real time, and photos meeting target aesthetic requirements are taken.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and photography guidance, and in particular to a method, system and electronic equipment for real-time guidance of photography composition based on Realtime-Transformer. Background Art

[0002] In the field of photography, the composition of an image plays a vital role in the overall beauty of a photo. Traditional composition optimization methods mainly rely on post-processing techniques, such as image cropping, to highlight the visual focus and improve the composition. However, such methods have certain limitations, mainly manifested in that they can only operate within the boundaries of the original image and cannot be extended beyond the image. This limitation hinders the photographer's opportunity to obtain the optimal composition during the shooting process and increases the burden of post-processing.

[0003] Existing photography composition optimization technologies cannot provide users with real-time guidance when framing a shot, nor can they predict and dynamically adjust the composition boundaries to obtain photos that better meet aesthetic standards. There is an urgent need for a new technical solution that can provide real-time composition suggestions during the shooting process and break the original image boundary limitations.

[0004] Therefore, how to provide a real-time guidance method and system for photographic composition that can provide real-time composition suggestions during the shooting and framing process and break the boundary limitations of the original image is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] In view of the above research status and existing problems, the present invention provides a real-time guidance method, system and electronic device for photographic composition based on Realtime-Transformer, which uses composition rules to guide the RT-DETR model to predict the aesthetic composition of photos. The predicted composition frame is allowed to extend beyond the current framing image range, providing dynamic and real-time guidance for image composition in photography, and taking photos that meet the target aesthetic requirements.

[0006] The present invention provides a real-time guiding method for photographic composition based on Realtime-Transformer, comprising the following steps:

[0007] Image acquisition: real-time acquisition of camera view as input image;

[0008] Composition rule classification: the input image is input into a pre-trained composition rule classification model to extract a multi-scale feature map, and the composition rule category of the input image is predicted based on the multi-scale feature map;

[0009] Composition frame prediction: The input image is input into the pre-trained RT-DETR model to extract original features, and the extracted original features are predicted to be stacked and filled to obtain the boundary extension features of the original image. Based on the original features and the boundary extension features, the corresponding composition rule categories are combined to predict and generate the composition frame of the input image;

[0010] Framing guide generation: generating a framing guide according to the composition frame and the position coordinates of the input image, and the framing guide is displayed in real time in the framing screen of the camera.

[0011] Preferably, the composition rule categories include: central composition, diagonal composition, horizontal line composition, symmetrical composition, triangle composition, vanishing point composition, vertical line composition or regular rule of thirds composition.

[0012] Preferably, the composition rule classification model includes:

[0013] A VGG network is used to extract multi-scale features of the input image and generate multiple feature maps with different resolutions;

[0014] A CNN convolutional layer is used to extract semantic features from each of the feature maps;

[0015] The global adaptive average pooling layer is used to reduce the dimension of the semantic features of each feature map;

[0016] The fully connected layer is used to receive the semantic features of all feature maps after dimensionality reduction and output the composition rule classification results.

[0017] Preferably, the RT-DETR model includes a boundary extension model and a composition frame prediction model, and the pre-training steps of the RT-DETR model include:

[0018] Boundary extension model pre-training:

[0019] Constructing a cropped data set, including a plurality of cropped data pairs, wherein the cropped data pairs include an original size image and a local area image cropped from the original size image;

[0020] The local area image is used as the input of the boundary extension model, and the original size image is used as the output of the boundary extension model to perform model weight training until a training stop condition is met, thereby obtaining a pre-trained boundary extension model;

[0021] Pre-training of the frame prediction model:

[0022] Construct a composition box dataset, including multiple groups of composition images annotated with composition boxes;

[0023] The composition image is used as the input of the composition frame prediction model, and the position coordinates of the composition frame are used as the output of the composition frame prediction model for model weight training until a training stop condition is met, thereby obtaining a pre-trained composition frame prediction model.

[0024] Preferably, the composition frame data set includes a plurality of groups of images with composition frame annotations, composition line annotations and composition scene object annotations, and the composition scene objects are object elements in the image.

[0025] Preferably, the step of generating a framing guide according to the composition frame and the position coordinates of the input image comprises:

[0026] Obtaining the coordinates of the center points of the composition frame and the input image, calculating the distance vector difference between the two center point coordinates, and generating a lens movement guide symbol according to the distance vector difference and displaying it in real time in the camera's viewfinder; and / or,

[0027] The sizes of the composition frame and the input image are obtained, a zoom vector difference between the two sizes is calculated, and a lens zoom guide symbol is generated according to the zoom vector difference and displayed in real time in the viewfinder of the camera.

[0028] Preferably, the method further includes a step of generating a composition annotation, wherein the composition annotation includes composition lines:

[0029] The input image is detected by deep Hough transform to obtain a plurality of semantic lines;

[0030] Calculating the contrast of image pixels on both sides of each semantic line, and screening all semantic lines based on the contrast to obtain the composition line;

[0031] The composition line is displayed in real time at a designated position in the viewfinder image of the camera.

[0032] Preferably, the designated position where the composition line is displayed includes a pixel point area where the corresponding semantic line is located, and / or a guide frame is displayed at the designated position of the framing screen, and the guide frame displays the corresponding proportional position of the composition line in the framing screen.

[0033] Preferably, the method further includes a step of generating a composition annotation, wherein the composition annotation includes a composition scene object mark:

[0034] The input image is detected by the YOLO model to obtain the composition scene object in the image;

[0035] The composition scene objects in the camera's viewfinder are marked and displayed in real time.

[0036] The present invention also provides a real-time guiding system for photographic composition based on Realtime-Transformer, comprising:

[0037] An image acquisition module is used to acquire the camera view picture in real time as an input image;

[0038] A composition rule classification module, used for inputting the input image into a pre-trained composition rule classification model, extracting a multi-scale feature map, and predicting the composition rule category of the input image based on the multi-scale feature map;

[0039] A composition frame prediction module is used to input the input image into a pre-trained RT-DETR model to extract original features, and perform stacking and filling prediction on the extracted original features to obtain boundary extension features of the original image, and generate a composition frame of the input image based on the original features and the boundary extension features in combination with the corresponding composition rule category prediction;

[0040] The framing guide generating module is used to generate the framing guide according to the composition frame and the position coordinates of the input image, and the framing guide is displayed in real time in the framing screen of the camera.

[0041] The present invention also provides an electronic device, comprising:

[0042] at least one processor; and

[0043] A memory and a camera in communication with the at least one processor; wherein,

[0044] The memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time-Transformer-based photography composition real-time guidance method to generate framing guidance;

[0045] The camera is used to perform a framing action, and to obtain the framing guide sent by the processor and display it in real time.

[0046] Compared with the prior art, it has the following beneficial effects:

[0047] Breaking image boundary limitations: The system can predict and adjust composition boxes that exceed the boundaries of the original image. Guided by rules and annotations, the model can explore potential compositions beyond the constraints of the initial image, enhancing the flexibility and creativity of the composition process.

[0048] Real-time interaction: Different from traditional post-processing methods, the present invention can provide composition suggestions in real time during the shooting process, assisting users to complete photographic works that are more in line with composition aesthetics, reducing the workload of post-processing.

[0049] Multi-label classification: The system supports a variety of composition types and can mark composition elements in real time. While providing explanatory support for the composition classification, it also provides users with more professional and rich composition guidance.

[0050] Improved user experience: Real-time camera adjustment guidance The predicted composition frame provides actionable guidance for real-time camera adjustment. Users will receive suggestions for repositioning or zooming the camera to capture the most beautiful image, allowing users to focus more on the creative process, significantly improving photography efficiency and photo quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.

[0052] Figure 1 A flow chart of a real-time guiding method for photographic composition based on Realtime-Transformer provided in an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of composition rule categories provided by an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of generating a composition frame based on boundary extension features provided by an embodiment of the present invention;

[0055] Figure 4 Schematic diagram of the camera framing screen guidance effect provided by an embodiment of the present invention Figure 1 ;

[0056] Figure 5 Schematic diagram of the camera framing screen guidance effect provided by an embodiment of the present invention Figure 2 ;

[0057] Figure 6 Schematic diagram of the camera framing screen guidance effect provided by an embodiment of the present invention Figure 3 ;

[0058] Figure 7 A comparison diagram of the framing images before and after adjustment according to the guidance provided in an embodiment of the present invention;

[0059] Figure 8 Comparison of the composition line marking effect provided by the embodiment of the present invention Figure 1 ;

[0060] Fig. 9 Comparison of the composition line marking effect provided by the embodiment of the present invention Figure 2 . DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The present invention provides users with camera adjustment suggestions in real time by combining deep learning models with composition rules, helping users to take photos that are more in line with professional composition standards, thereby overcoming the limitations of the prior art.

[0063] like Figure 1 As shown, the first aspect of the embodiment of the present invention discloses a real-time guiding method for photographic composition based on Realtime-Transformer, comprising the following steps:

[0064] Image acquisition: real-time acquisition of camera view as input image;

[0065] Composition rule classification: The input image is input into the pre-trained composition rule classification model to extract the multi-scale feature map, and the composition rule category of the input image is predicted based on the multi-scale feature map;

[0066] Composition frame prediction: The input image is input into the pre-trained RT-DETR model for original feature extraction, and the extracted original features are stacked and filled to predict the boundary extension features of the original image. Based on the original features and boundary extension features, the corresponding composition rule category is combined to predict and generate the composition frame of the input image;

[0067] Framing guide generation: Framing guides are generated based on the composition frame and the position coordinates of the input image, and the framing guides are displayed in real time in the camera's viewfinder.

[0068] It should be noted that this method is executed during the camera framing stage before taking a photo. The user adjusts the camera position according to the framing guide. After reaching the position indicated by the composition frame, the user can shoot the current framing screen to obtain a photo with the adjusted composition.

[0069] In one embodiment, Figure 2 As shown in (a)-(h), the composition rule categories are: center composition, diagonal composition, horizontal composition, symmetrical composition, triangle composition, vanishing point composition, vertical line composition or regular rule of thirds composition. The method of this embodiment can flexibly identify one or more composition rules that the input image may comply with.

[0070] In one embodiment, the composition rule classification model includes:

[0071] VGG network, used to extract multi-scale features of the input image and generate multiple feature maps of different resolutions;

[0072] CNN convolutional layer, used to extract semantic features from each feature map;

[0073] The global adaptive average pooling layer is used to reduce the dimension of the semantic features of each feature map;

[0074] The fully connected layer is used to receive the semantic features of all feature maps after dimensionality reduction and output the composition rule classification results.

[0075] This embodiment uses the VGG network as the basic model for feature extraction and combines the multi-label classification method to analyze the input image. The specific process is as follows:

[0076] First, the system extracts multi-scale features of the image through the VGG16 model, generates multiple feature maps of different resolutions, and then processes them through convolutional neural networks (CNNs), which include multi-layer convolution, batch normalization, and ReLU activation functions to extract more advanced semantic features. After feature extraction, the feature map is reduced in dimension using a global adaptive average pooling layer, and the classification results of 8 different composition rules are output through a fully connected layer.

[0077] In one embodiment, the RT-DETR (Real-Time Detection Transformer) model is used to predict the best composition frame in the image and guide the user to adjust the camera position in real time to achieve the best photographic composition effect.

[0078] The input of the RT-DETR model is a preprocessed RGB image, usually an image of a fixed size (such as 224x224 pixels). After the convolutional layer extracts features, the image is sent to the Transformer module for global feature analysis and processing.

[0079] The output of the RT-DETR model is the bounding box in the input image and the composition rule category label. The bounding box is described by four coordinate points (x, y, w, h), which represents the frame in the image that best conforms to the composition rule, helping users adjust the camera's perspective, zoom ratio, etc. according to the system's suggestions.

[0080] In one embodiment, the RT-DETR model includes a boundary extension model and a composition box prediction model, where Figure 3As shown in the figure, the boundary extension model is used to predict the feature representation outside the image boundary. Through the multi-layer self-attention mechanism and the multi-head cross-attention mechanism, the accuracy and computational efficiency of feature extrapolation are significantly enhanced. The specific execution process is as follows:

[0081] The input of the boundary extension model consists of visible features V and a learnable query variable m.

[0082] First, the self-attention mechanism is used to calculate the internal correlation of visible features to capture the global dependencies of the image.

[0083] In the decoder layer, a dynamic weight adjustment mechanism is innovatively introduced, which enables the network to adaptively adjust the weights of features at different levels according to the complexity of image features, thereby improving the accuracy and stability of extrapolated features.

[0084] After each decoder layer, the features are normalized through the staged Add&Norm layers to ensure the stability of the network during training while maintaining efficient feature transfer.

[0085] The boundary extension model also adopts a parallel feedforward network (FFN) structure. After each self-attention and cross-attention calculation, fine-grained feature extraction and fusion are performed on the extrapolated features and visible features respectively, which greatly improves the efficiency and quality of feature generation.

[0086] Finally, through multi-layer iterative calculation, the filling feature P is generated. On this basis, the visible feature V is interactively calculated with the filling feature P generated in each iteration through multi-head cross-attention to infer the feature representation outside the image boundary. This not only enhances the visual scalability of the image, but also provides more accurate and flexible support for image composition optimization, ensuring the authenticity and aesthetic effect of the image.

[0087] This embodiment uses the decoder layer to stack and expand features so that the predicted composition frame can exceed the boundary of the original image, thereby achieving the expansion of the boundary frame. The system can predict potential compositions that exceed the current field of view of the camera and generate composition frames that exceed the hardware framing range.

[0088] In this embodiment, the pre-training steps of the RT-DETR model include:

[0089] Boundary extension model pre-training:

[0090] Constructing a cropping data set, including multiple groups of cropping data pairs, the cropping data pairs including original size images and local area images cropped from the original size images;

[0091] The local area image is used as the input of the boundary extension model, and the original size image is used as the output of the boundary extension model to perform model weight training until the training stop condition is met, thereby obtaining a pre-trained boundary extension model;

[0092] Pre-training of the frame prediction model:

[0093] Construct a composition box dataset, including multiple groups of composition images annotated with composition boxes;

[0094] The composition image is used as the input of the composition frame prediction model, and the position coordinates of the composition frame are used as the output of the composition frame prediction model for model weight training until the training stop condition is met to obtain a pre-trained composition frame prediction model.

[0095] The loss function of this training process combines position loss (L1-Loss and IoU-Loss) and cross entropy loss, and the training model optimizes the accuracy of the composition frame and the classification of the composition type at the same time. The training process uses a gradient descent optimization algorithm (such as Adam or SGD) to continuously optimize the model parameters, so that RT-DETR can effectively predict the composition frame and composition rule category.

[0096] In one embodiment, real-time images are captured by a camera, and based on these framing images, an analysis is performed, and an optimal composition boundary box is generated in combination with the predicted composition rules, and adjustment suggestions such as camera movement and zooming are given in combination with the user's current framing content. These suggestions are fed back to the framing interface in real time to help users optimize the angle, position, focal length, etc. of the camera before taking pictures to obtain the best composition effect. The steps of generating framing guidelines based on the composition frame and the position coordinates of the input image include:

[0097] Obtain the coordinates of the center points of the composition frame and the input image, calculate the distance vector difference between the two center point coordinates, generate a lens movement guide symbol based on the distance vector difference and display it in real time on the camera's viewfinder; and / or obtain the size of the composition frame and the input image, calculate the scaling vector difference of the two sizes, generate a lens scaling guide symbol based on the scaling vector difference and display it in real time on the camera's viewfinder.

[0098] In the framing interface, prompts are provided to guide users to move the camera in a certain direction to capture a wider scene or a more beautiful composition, thus providing more flexible shooting guidance. This process ensures that before taking a photo, users can adjust the perspective based on real-time suggestions to ensure that the composition meets the best aesthetic standards. Figure 4-6 As shown, the framing guide may be in the form of a marking guide, such as an arrow identifier. Figure 4 Instructs the user to move the viewfinder upwards. Figure 5 Instructs the user to move the viewfinder to the left. Figure 6 The user is guided to zoom in or out on the viewfinder screen. Figure 7 Shown is a comparison of the framing images before and after adjustment according to the guidelines.

[0099] In one embodiment, the composition frame dataset includes a plurality of groups of images with composition frame annotations, composition line annotations, and composition scene object annotations, where the composition scene objects are object elements in the images.

[0100] In one embodiment, the step of generating a composition annotation is further included, and the composition annotation includes composition lines:

[0101] The input image is detected through deep Hough transform to obtain several semantic lines;

[0102] Calculate the contrast of the image pixels on both sides of each semantic line, and screen all semantic lines based on the contrast to obtain the composition lines;

[0103] The composition lines are displayed in real time at the specified position in the camera's viewfinder.

[0104] In the specific implementation, the composition line detects the line elements in the image through the improved deep Hough transform. The improved deep Hough transform includes: semantic segmentation, deep Hough change and contrast-based composition line screening method:

[0105] First, the input image is semantically segmented, and then deep Hough is used to detect all semantic lines. Finally, the contrast at both ends of each semantic line is calculated to screen all detected semantic lines to obtain the final composition line.

[0106] In this embodiment, the designated position where the composition line is displayed includes the pixel point area where the corresponding semantic line is located, and / or a guide frame is displayed at the designated position of the framing screen, and the guide frame displays the corresponding proportional position of the composition line in the framing screen.

[0107] like Figure 4-6 As shown in FIG. 1 , a guide frame is displayed in the upper right corner of the viewfinder screen, and the target position of the horizontal composition line is displayed in the guide frame at a corresponding proportional height.

[0108] like Figure 8 As shown, the composition lines of the diagonal shapes are shown. Figure 8 (a) is the framing screen of the input image, Figure 8 (b) is the viewfinder after the camera position is adjusted in real time according to the generated diagonal composition line. During the shooting process, the system analyzes the current viewing angle in real time and predicts the candidate composition frame based on feature prediction. The user adjusts the camera position according to the instructions and captures photos that meet the "diagonal composition" rule, which ultimately improves the aesthetic score of the photo. Figure 8 The composition lines marked in real time in (b) provide explainability, allowing users to understand the logic behind the adjustment.

[0109] like Fig. 9 As shown, the composition lines in the shape of horizontal lines are shown. Fig. 9 (a) is the framing screen of the input image, Fig. 9 (b) is the framing picture after the camera position is adjusted in real time according to the generated horizontal line shape composition line. During the shooting process, the system analyzes the key objects and still combines feature prediction to generate the best composition frame, guiding the user to take photos that meet the "horizontal line composition" standard by moving the camera horizontally and zooming appropriately to obtain high-quality visual effects.

[0110] In one embodiment, the process further includes generating a composition annotation, wherein the composition annotation includes a composition scene object tag, such as a person, a building, etc.:

[0111] The input image is detected by the YOLO model to obtain the composition scene objects in the image;

[0112] The composition scene objects in the camera's viewfinder are marked and displayed in real time.

[0113] In the specific implementation, the composition elements are detected and marked by the YOLOv11 model to obtain the main content of the image. These annotations help users intuitively see the application of composition rules when adjusting the shooting angle in real time, thereby guiding the shooting of photos that meet aesthetic standards.

[0114] In one embodiment, the step of aesthetically grading the one or more composition rule categories that the input image may conform to is also included, selecting the composition frame with the highest aesthetic score, and displaying it to the user through the camera viewfinder. Figure 7 As shown, the score of the image before the guide adjustment according to the method of this embodiment is 0.34 points, and the score of the image after the guide adjustment is 0.53 points. It can be seen that after the guide adjustment, a more aesthetically superior photo is obtained.

[0115] The second aspect of the embodiment discloses a real-time guidance system for photographic composition based on Realtime-Transformer, comprising:

[0116] An image acquisition module is used to acquire the camera view picture in real time as an input image;

[0117] A composition rule classification module, used to input an input image into a pre-trained composition rule classification model, extract a multi-scale feature map, and predict the composition rule category of the input image based on the multi-scale feature map;

[0118] The composition frame prediction module is used to input the input image into the pre-trained RT-DETR model to extract original features, and perform stacking and filling prediction on the extracted original features to obtain the boundary extension features of the original image. Based on the original features and the boundary extension features, the corresponding composition rule category is combined to predict and generate the composition frame of the input image;

[0119] The framing guide generation module is used to generate framing guides according to the composition frame and the position coordinates of the input image, and the framing guides are displayed in real time in the framing screen of the camera.

[0120] The second aspect of the embodiment of the present invention can be used to execute all the steps disclosed in the first aspect of the embodiment of the present invention.

[0121] A third aspect of the embodiment discloses an electronic device, including:

[0122] at least one processor; and

[0123] A memory and a camera in communication with at least one processor; wherein,

[0124] The memory stores instructions executed by at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time-Transformer-based photography composition real-time guidance method disclosed in the first aspect of the embodiment to generate framing guidance;

[0125] The camera is used to perform framing actions, and obtain the framing guidelines sent by the processor and display them in real time.

[0126] The camera helps users understand the rationale behind suggested adjustments by loading a UI interface with real-time annotations of framing guides and composition annotations. This real-time feedback mechanism ensures that users can effortlessly adhere to composition guidelines to produce beautiful photos.

[0127] The above is a detailed introduction to a real-time guidance method, system and electronic device for photographic composition based on Realtime-Transformer provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0128] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A real-time guiding method for photographic composition based on Realtime-Transformer, characterized in that: The steps include: Image acquisition: real-time acquisition of camera view as input image; Composition rule classification: the input image is input into a pre-trained composition rule classification model to extract a multi-scale feature map, and the composition rule category of the input image is predicted based on the multi-scale feature map; Composition frame prediction: The input image is input into the pre-trained RT-DETR model to extract original features, and the extracted original features are predicted to be stacked and filled to obtain the boundary extension features of the original image. Based on the original features and the boundary extension features, the corresponding composition rule categories are combined to predict and generate the composition frame of the input image; Framing guide generation: generating a framing guide according to the composition frame and the position coordinates of the input image, and the framing guide is displayed in real time in the framing screen of the camera.

2. The real-time guiding method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that: The composition rule categories include: central composition, diagonal composition, horizontal line composition, symmetrical composition, triangle composition, vanishing point composition, vertical line composition or regular rule of thirds composition.

3. The real-time guiding method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that: The composition rule classification model includes: A VGG network is used to extract multi-scale features of the input image and generate multiple feature maps with different resolutions; A CNN convolutional layer is used to extract semantic features from each of the feature maps; The global adaptive average pooling layer is used to reduce the dimension of the semantic features of each feature map; The fully connected layer is used to receive the semantic features of all feature maps after dimensionality reduction and output the composition rule classification results.

4. The real-time guiding method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that: The RT-DETR model includes a boundary extension model and a composition box prediction model. The pre-training steps of the RT-DETR model include: Boundary extension model pre-training: Constructing a cropped data set, including a plurality of cropped data pairs, wherein the cropped data pairs include an original size image and a local area image cropped from the original size image; The local area image is used as the input of the boundary extension model, and the original size image is used as the output of the boundary extension model to perform model weight training until a training stop condition is met, thereby obtaining a pre-trained boundary extension model; Pre-training of the frame prediction model: Construct a composition box dataset, including multiple groups of composition images annotated with composition boxes; The composition image is used as the input of the composition frame prediction model, and the position coordinates of the composition frame are used as the output of the composition frame prediction model for model weight training until a training stop condition is met, thereby obtaining a pre-trained composition frame prediction model.

5. The method for real-time guidance of photographic composition based on Realtime-Transformer according to claim 4, characterized in that: The composition frame data set includes a plurality of groups of images with composition frame annotations and composition object annotations, where the composition objects are object elements in the images.

6. The real-time guiding method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that: The step of generating a framing guide according to the composition frame and the position coordinates of the input image comprises: Obtaining the coordinates of the center points of the composition frame and the input image, calculating the distance vector difference between the two center point coordinates, and generating a lens movement guide symbol according to the distance vector difference and displaying it in real time in the camera's viewfinder; and / or, The sizes of the composition frame and the input image are obtained, a zoom vector difference between the two sizes is calculated, and a lens zoom guide symbol is generated according to the zoom vector difference and displayed in real time in the viewfinder of the camera.

7. The real-time guiding method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that: It also includes the steps of generating composition annotations, which include composition lines: The input image is detected by deep Hough transform to obtain a plurality of semantic lines; Calculating the contrast of image pixels on both sides of each semantic line, and screening all semantic lines based on the contrast to obtain the composition line; The composition line is displayed in real time at a designated position in the viewfinder image of the camera.

8. The real-time guiding method for photographic composition based on Realtime-Transformer according to claim 1, characterized in that: It also includes the step of generating composition annotations, which include composition scene object markers: The input image is detected by the YOLO model to obtain the composition scene object in the image; The composition scene objects in the camera's viewfinder are marked and displayed in real time.

9. A real-time guidance system for photographic composition based on Realtime-Transformer, characterized in that: include: An image acquisition module is used to acquire the camera view picture in real time as an input image; A composition rule classification module, used for inputting the input image into a pre-trained composition rule classification model, extracting a multi-scale feature map, and predicting the composition rule category of the input image based on the multi-scale feature map; A composition frame prediction module is used to input the input image into a pre-trained RT-DETR model to extract original features, and perform stacking and filling prediction on the extracted original features to obtain boundary extension features of the original image, and generate a composition frame of the input image based on the original features and the boundary extension features in combination with the corresponding composition rule category prediction; The framing guide generating module is used to generate the framing guide according to the composition frame and the position coordinates of the input image, and the framing guide is displayed in real time in the framing screen of the camera.

10. An electronic device comprising: at least one processor; as well as A memory and a camera in communication with the at least one processor; wherein, The memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time guidance method for photographic composition based on Realtime-Transformer according to any one of claims 1 to 8 to generate framing guidance; The camera is used to perform a framing action, and to obtain the framing guide sent by the processor and display it in real time.

Images