Image output method and device, terminal equipment and storage medium

Abstract

The invention is suitable for the technical field of computers, and provides an image output method and device, terminal equipment and a storage medium, and the image output method comprises the steps: generating an estimated pose parameter of a target camera based on a camera pose estimation model and an input image; triggering the target camera to carry out image acquisition under the pose corresponding to the estimated pose parameter to obtain a target image; obtaining a quality score of the target image; if the quality score is not greater than the first preset threshold value, returning to execute the step of generating the pre-estimated pose parameter of the target camera based on the camera pose pre-estimation model and the input image and the subsequent steps; and if the quality score is greater than a first preset threshold, outputting the target image. According to the image output method provided by the embodiment of the invention, the accuracy of possibly reasonable pose parameters existing in the target camera is improved, the target image meeting the quality scoring requirement is output, and the quality of the output image and the satisfaction degree of a user are improved.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to an image output method, apparatus, terminal device and storage medium. Background Art

[0002] When generating 2D image data in a 3D scene, it is crucial to accurately determine the position and orientation of the camera, which usually involves six-dimensional parameters: the camera coordinate position in 3D space and the 3D rotation angle.

[0003] Currently, there are two main strategies for determining the position and orientation of a camera in a 3D scene. The first strategy is to directly set the position and orientation of the camera by randomly generating six-dimensional parameters. The other strategy relies on manual annotation. Professionals manually select the position and orientation of the camera in the 3D scene, evaluate and select the quality of the resulting image, and finally determine a reasonable camera position. However, the first strategy is difficult to ensure the rationality of the camera position, resulting in the acquisition of low-quality images. The second strategy requires a lot of human resources in scenarios where a large amount of image data needs to be generated, and the efficiency is low. Summary of the invention

[0004] The embodiments of the present application provide an image output method, apparatus, terminal device and storage medium, which can solve the problem that it is difficult to ensure the rationality of the camera posture when determining the position and orientation of the camera in the 3D scene, resulting in the acquisition of low-quality images and low efficiency.

[0005] In a first aspect, an embodiment of the present application provides an image output method, comprising:

[0006] Generate the estimated pose parameters of the target camera based on the camera pose estimation model and the input image;

[0007] Triggering the target camera to perform image acquisition at a posture corresponding to the estimated posture parameter to obtain a target image;

[0008] Obtaining a quality score of the target image;

[0009] If the quality score is not greater than the first preset threshold, returning to execute the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps;

[0010] If the quality score is greater than the first preset threshold, outputting the target image;

[0011] Among them, the camera pose estimation model is a first neural network model trained based on a first training data set. When the input images are the same, the output results of the camera pose estimation model are not all the same. The first training data set includes multiple first training samples, each first training sample includes sample pose parameters and a first sample image captured by the target camera in a pose corresponding to the sample pose parameters, and the quality score of the first sample image is greater than a second preset threshold.

[0012] In a possible implementation manner of the first aspect, obtaining the quality score of the target image includes:

[0013] The target image is input into an image quality assessment model to obtain a quality score of the target image output by the image quality assessment model; wherein the image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes multiple second training samples, each second training sample includes multiple second sample images and annotated quality scores corresponding to each second sample image.

[0014] In a possible implementation manner of the first aspect, before inputting the target image into an image quality assessment model to obtain a quality score of the target image output by the image quality assessment model, the method further includes:

[0015] Acquire the plurality of second training samples;

[0016] Inputting the second sample image in each of the second training samples into the image quality assessment model to obtain a predicted quality score of each of the second sample images output by the image quality assessment model;

[0017] Calculating an error between a predicted quality score of each second sample image and a labeled quality score corresponding to each second sample image;

[0018] The model parameters of the image quality assessment model are iteratively updated according to the error until the image quality assessment model meets a preset second training stop condition.

[0019] In a possible implementation manner of the first aspect, before generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image, the method further includes:

[0020] Acquire a plurality of the first training samples;

[0021] Inputting the first sample image in each of the first training samples into the camera pose estimation model to obtain an estimated pose parameter corresponding to each of the first sample images output by the camera pose estimation model;

[0022] Calculating the loss between the estimated pose parameter corresponding to each of the first sample images and the sample pose parameter corresponding to each of the first sample images;

[0023] Iteratively update model parameters of the camera pose estimation model according to the loss until the camera pose estimation model meets a preset first training stop condition.

[0024] In a possible implementation manner of the first aspect, after generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image, the method further includes:

[0025] Determining whether the estimated pose parameters meet a first preset rule; wherein the first preset rule is used to screen the pose parameters of the target camera;

[0026] If the estimated posture parameters meet the first preset rule, executing the step of triggering the target camera to perform image acquisition at the posture corresponding to the estimated posture parameters to obtain the target image and subsequent steps;

[0027] If the estimated pose parameters do not meet the first preset rule, return to execute the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps.

[0028] In a possible implementation manner of the first aspect, obtaining the quality score of the target image includes:

[0029] Inputting the target image into an image quality assessment model to obtain an image quality score corresponding to the target image output by the image quality assessment model; wherein the image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes a plurality of second training samples, each second training sample includes a plurality of second sample images and annotated quality scores corresponding to each second sample image;

[0030] Performing a posture quality score on the estimated posture parameter according to a second preset rule to obtain a posture quality score corresponding to the estimated posture parameter;

[0031] A quality score of the target image is obtained according to the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameter.

[0032] In a possible implementation manner of the first aspect, obtaining the quality score of the target image according to the image quality score corresponding to the target image and the pose quality score corresponding to the estimated pose parameter includes:

[0033] Obtaining a first weight of an image quality score corresponding to the target image and a second weight of a posture quality score corresponding to the estimated posture parameter;

[0034] Based on the first weight and the second weight, a weighted sum is performed on the image quality score and the posture quality score to obtain a quality score of the target image.

[0035] In a second aspect, an embodiment of the present application provides an image output device, including:

[0036] An estimated pose parameter generation module is used to generate estimated pose parameters of a target camera based on a camera pose estimation model and an input image;

[0037] A target image acquisition module, used for triggering the target camera to acquire an image at a posture corresponding to the estimated posture parameter to obtain a target image;

[0038] A quality score acquisition module, used to acquire the quality score of the target image;

[0039] An execution module, configured to return to the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps if the quality score is not greater than a first preset threshold;

[0040] an output module, configured to output the target image if the quality score is greater than the first preset threshold;

[0041] Among them, the camera pose estimation model is a first neural network model trained based on a first training data set. When the input images are the same, the output results of the camera pose estimation model are not all the same. The first training data set includes multiple first training samples, each first training sample includes sample pose parameters and a first sample image captured by the target camera in a pose corresponding to the sample pose parameters, and the quality score of the first sample image is greater than a second preset threshold.

[0042] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an image output method as described in any one of the first aspects above is implemented.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image output method as described in any one of the first aspects above is implemented.

[0044] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the image output method described in any one of the above-mentioned first aspects.

[0045] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0046] The image output method provided by the embodiment of the present application, first, the camera pose estimation model is a camera pose estimation model obtained by training a first sample image of high quality (quality score greater than the second preset threshold) and a first training sample of the corresponding sample pose parameters. The camera pose estimation model generates the estimated pose parameters of the target camera based on the input image, which replaces the random sampling method in the prior art, and improves the accuracy of the possible reasonable pose parameters of the target camera. Secondly, by adjusting the parameters of the target camera to the estimated pose parameters and triggering the target camera to collect the target image at the corresponding pose, the efficiency and accuracy of image acquisition are improved. Third, by obtaining the quality score of the target image and comparing it with the first preset threshold, the accuracy of the estimated pose parameters can be quantitatively evaluated. When the quality score of the target image is not greater than the preset threshold, it is determined that the estimated pose parameters corresponding to the target image are unreasonable pose parameters of the target camera. At this time, the input image is re-entered into the camera pose estimation model to obtain new estimated pose parameters and the subsequent entire process until the quality score of the collected target image meets the requirements. When the quality score of the target image is greater than a preset threshold, the estimated pose parameters corresponding to the target image are determined to be reasonable pose parameters of the target camera. At this time, the quality of the target image captured by the target camera at the corresponding pose is relatively high, and the target image that meets user requirements or application needs is output, thereby improving the quality of the output image and user satisfaction.

[0047] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 is a flowchart of an image output method provided by an embodiment of the present application;

[0050] Figure 2is a flowchart of the sub-steps of step S101 provided in an embodiment of the present application;

[0051] Figure 3 is a flowchart of another image output method provided by an embodiment of the present application;

[0052] Figure 4 It is a flowchart of a sub-step of step S106 provided in an embodiment of the present application;

[0053] Figure 5 It is a flowchart of another sub-step of step S106 provided in an embodiment of the present application;

[0054] Figure 6 is a flowchart of the sub-steps of step S1067 provided in an embodiment of the present application;

[0055] Figure 7 It is a schematic diagram of the overall process of the image output method provided by an embodiment of the present application;

[0056] Figure 8 is a structural schematic diagram of an image output device provided by an embodiment of the present application;

[0057] Fig. 9 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0060] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0061] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0062] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0063] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0064] Currently, there are two main strategies for determining the position of a camera in a 3D scene. The first strategy is to directly set the position and orientation of the camera by randomly generating six-dimensional parameters. The other strategy relies on manual annotation. Professionals manually select the position and orientation of the camera in the 3D scene, evaluate and select the resulting image quality, and finally determine a reasonable camera position. However, the first strategy is difficult to ensure the rationality of the camera position, resulting in low-quality images. The second strategy requires a lot of human resources and is inefficient in scenarios where a large amount of image data needs to be generated.

[0065] In order to solve the above technical problems, the image output method provided by the embodiment of the present application, first, the camera pose estimation model is a camera pose estimation model obtained by training the first training sample of the first sample image with high quality (quality score greater than the second preset threshold) and the corresponding sample pose parameters, and the camera pose estimation model generates the estimated pose parameters of the target camera based on the input image, replacing the random sampling method in the prior art, and improving the accuracy of the possible reasonable pose parameters of the target camera. Secondly, by adjusting the parameters of the target camera to the estimated pose parameters and triggering the target camera to collect the target image at the corresponding pose, the efficiency and accuracy of image acquisition are improved. Third, by obtaining the quality score of the target image and comparing it with the first preset threshold, the accuracy of the estimated pose parameters can be quantitatively evaluated. When the quality score of the target image is not greater than the preset threshold, it is determined that the estimated pose parameters corresponding to the target image are unreasonable pose parameters of the target camera. At this time, the input image is re-entered into the camera pose estimation model to obtain new estimated pose parameters and the subsequent entire process until the quality score of the collected target image meets the requirements. When the quality score of the target image is greater than a preset threshold, the estimated pose parameters corresponding to the target image are determined to be reasonable pose parameters of the target camera. At this time, the quality of the target image captured by the target camera at the corresponding pose is relatively high, and the target image that meets user requirements or application needs is output, thereby improving the quality of the output image and user satisfaction.

[0066] See also Figure 1 , is a flowchart of an image output method provided in an embodiment of the present application. As an example but not a limitation, the method may include the following steps S102 to S108.

[0067] Step S102: Generate estimated pose parameters of the target camera based on the camera pose estimation model and the input image.

[0068] Among them, the camera pose estimation model is a first neural network model trained based on a first training data set. When the input images are the same, the output results of the camera pose estimation model are not all the same. The first training data set includes multiple first training samples, each of which includes sample pose parameters and a first sample image captured by the target camera at a pose corresponding to the sample pose parameters, and the quality score of the first sample image is greater than a second preset threshold.

[0069] The application scenario of the embodiment of the present application is a 3D scene. In the 3D scene, after the construction of the 3D scene is completed and various 3D objects are placed according to the design requirements, it is necessary to determine the position parameters of the target camera in the 3D scene. The position parameters of the target camera include the position coordinate parameters (x, y, z) and attitude parameters (Yaw, Pitch, Roll) of the target camera in three-dimensional space. Among them, x, y, z represent the coordinate values ​​of the target camera on the x-axis, y-axis and z-axis in three-dimensional space respectively. The position coordinate parameters (x, y, z) determine the spatial position of the target camera in the 3D scene. Yaw, Pitch, Roll represent the rotation angles of the target camera around the z-axis, y-axis and x-axis respectively. Among them, Yaw is also called the azimuth, which determines the direction of the target camera on the horizontal plane, Pitch is also called the pitch angle, which determines the degree of up and down tilt of the camera in the vertical plane, and Roll is also called the roll angle, which determines the rotation of the camera around its own axis. The attitude parameters (Yaw, Pitch, Roll) determine the orientation and rotation direction of the target camera, thereby determining the viewing angle of the target camera. The three position coordinate parameters and the three attitude parameters together determine the position and orientation of the target camera in three-dimensional space.

[0070] It should be noted that the parameters of the target camera include internal parameters and external parameters, among which the internal parameters of the target camera are a set of important parameters that describe the optical properties of the target camera itself, including focal length (fx, fy), principal point position (cx, cy) and distortion coefficient, etc. The internal parameters of the target camera determine how to project 3D points onto the 2D image plane, that is, determine the mapping relationship of the target camera from the three-dimensional world to the two-dimensional image plane, that is, the internal characteristics of the imaging model. The internal parameters of the target camera are obtained through the camera calibration process. After the calibration is completed, the internal parameters of the target camera are fixed. The pose parameters of the target camera in this embodiment refer to the external parameters of the target camera.

[0071] In order to solve the defects of the current method of determining the pose parameters of the target camera in the 3D scene, such as the defect that it is difficult to ensure the rationality of the camera pose, or the defect that the pose parameters of the target camera in the 3D scene are determined by manual annotation, which leads to low efficiency, this embodiment uses a sufficient number of first training samples to train a camera pose estimation model, and the camera pose estimation model is used to learn the complex mapping relationship between the input first sample image and the sample pose parameters corresponding to the input first sample image. Therefore, after the camera pose estimation model is trained, the new input image is input into the camera pose estimation model, and the camera pose estimation model can quickly and accurately generate the estimated pose parameters of the target camera, that is, quickly and accurately predict the position and orientation of the target camera in three-dimensional space.

[0072] In some embodiments, before generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image, the method further includes: Step S101: obtaining a camera pose estimation model through training based on a first training data set. Figure 2 The flowchart of the sub-steps of step S101 provided in this embodiment is shown as follows: Figure 2 As shown, step S101 includes steps S1011 to S1014.

[0073] Step S1011: Acquire multiple first training samples.

[0074] Specifically, before training the camera pose estimation model, multiple first training samples need to be prepared. In the process of preparing the first training sample, first, a large number of randomly generated pose parameters are collected, and the target camera is triggered to perform image acquisition under the randomly generated pose parameters to obtain sample images corresponding to the randomly generated pose parameters. However, since the randomly generated pose parameters may contain many unreasonable or invalid poses, for example, the target camera may be placed in a position where it cannot capture valid information, or the captured image may have extremely poor quality due to factors such as angle and lighting, the quality of the sample images corresponding to the randomly generated pose parameters collected will also be uneven. In order to ensure the accuracy and generalization ability of the trained camera pose estimation model, it is necessary to annotate and screen the sample images corresponding to the randomly generated pose parameters.

[0075] During the annotation process, the quality score of the sample image corresponding to the randomly generated posture parameter is set between 0 and 5 points, and the quality score of each sample image corresponding to the randomly generated posture parameter is manually or automatically scored, and the quality score corresponding to each sample image corresponding to the randomly generated posture parameter is annotated. After the quality scores of all sample images corresponding to the randomly generated posture parameters are annotated, the sample images with a quality score higher than a second preset threshold (for example, 4 points) are screened out, and the sample images with a quality score higher than 4 points are used as the first sample images in the first training samples, and the posture parameters corresponding to the sample images with a quality score higher than 4 points are used as the sample posture parameters in the first training samples, thereby obtaining multiple first training samples.

[0076] A plurality of first training samples constitute a first training data set, and at this time, the first sample images in the first training data set are all high-quality images. Since high-quality images usually have better clarity, contrast, color balance and other features, the accuracy of the sample pose parameters corresponding to the high-quality images is higher. Using more accurate high-quality images and sample pose parameters to train the camera pose estimation model can reduce the possibility of the camera pose estimation model learning incorrect pose parameters, so that the camera pose estimation model generates more accurate estimated pose parameters.

[0077] It should be noted that setting the quality score interval of the sample image between 0-5 points and setting the second preset threshold to 4 points is only a specific example of the quality score interval of the sample image and the second preset threshold. In this embodiment, the quality score interval of the sample image and the value of the second preset threshold can be adjusted according to actual conditions and needs.

[0078] Step S1012: inputting the first sample image in each first training sample into the camera pose estimation model to obtain the estimated pose parameters corresponding to each first sample image output by the camera pose estimation model.

[0079] Among them, the camera pose estimation model is a first neural network model based on training with a first training data set. Variational Autoencoder (VAE for short) is a generative neural network model, and the encoder of VAE is responsible for converting the input image into its feature vector in the latent space, and learning the Gaussian distribution of the feature vector corresponding to the input image in the latent space. After the encoder learns the Gaussian distribution of the feature vector corresponding to the input image in the latent space, a new point in the latent space (i.e., the sampling feature) can be randomly sampled from the Gaussian distribution, and the decoder of VAE is responsible for decoding the sampling features to obtain a new image similar to the input image. Based on the characteristics of VAE, the camera pose estimation model of this embodiment can select VAE as the basic framework of the camera pose estimation model, and use VAE to learn the complex mapping relationship between the first sample image input and the sample pose parameters corresponding to the first sample image input.

[0080] Specifically, in the training process of the camera pose estimation model, for each first training sample, the first sample image in the first training data set is input into the camera pose estimation model, and the encoder of the camera pose estimation model converts the input first sample image into its feature vector in the latent space, and learns the Gaussian distribution of the feature vector corresponding to the input first sample image in the latent space and the complex mapping relationship between the sample pose parameters corresponding to the input first sample image. Therefore, after the first sample image in each first training sample is input into the camera pose estimation model, the camera pose estimation model converts the first sample image into a feature vector in the latent space using the encoder according to the current model parameters, and outputs the estimated pose parameters corresponding to each first sample image through the learned complex mapping relationship between the sample pose parameters corresponding to the input first sample image. However, since the camera pose estimation model is in the early stage of training, or when the number of first training samples is relatively small, the estimated pose parameters output by the camera pose estimation model may not be accurate at this time.

[0081] Step S1013: Calculate the loss between the estimated pose parameters corresponding to each first sample image and the sample pose parameters corresponding to each first sample image.

[0082] Specifically, in order to evaluate the gap between the estimated pose parameters input by the camera pose estimation model and the sample pose parameters in the first training sample, the estimated pose parameters are compared with the sample pose parameters in the first training sample, and the loss between the estimated pose parameters and the sample pose parameters in the first training sample is calculated, for example, the mean square error or root mean square error between the estimated pose parameters and the sample pose parameters in the first training sample is calculated to obtain the difference between the predicted result of the camera pose estimation model and the actual pose.

[0083] Step S1014: Iteratively update the model parameters of the camera pose estimation model according to the loss until the camera pose estimation model meets the preset first training stop condition.

[0084] Specifically, according to the calculated mean square error value or root mean square error value between the estimated pose parameters and the sample pose parameters in the first training sample, the mean square error value or root mean square error value is back-propagated to each layer of the camera pose estimation model through the back propagation algorithm, so as to calculate the gradient of the parameters of each layer. Then, the parameters of the camera pose estimation model are updated according to these gradients using an optimization algorithm (such as Adam, SGD, etc.). The optimization algorithm adjusts the value of the parameter to minimize the loss between the estimated pose parameters and the corresponding sample pose parameters, traverses all the first training samples in the first training data set, and repeats the process from step S202 to step S204 multiple times until the camera pose estimation model meets the preset first training stop condition. Among them, the first training stop condition includes: when the change in the loss between the estimated pose parameters and the corresponding sample pose parameters is less than a certain threshold, that is, the camera pose estimation model has converged and the training stops; or, when the training rounds of the camera pose estimation model in the training process reach the preset maximum training rounds, the training stops.

[0085] This embodiment uses VAE to learn the distribution of reasonable camera positions and orientations, and generates predicted pose parameters of the target camera through VAE, avoiding random sampling of camera positions and orientations from high-dimensional space, saving time for obtaining estimated pose parameters, and the generated estimated pose parameters correspond to the reasonable position and orientation of the camera with higher accuracy.

[0086] In some embodiments, Figure 3 A schematic diagram of a flow chart of another image output method provided by this embodiment is shown, such as Figure 3 As shown, after step S102, step S103 is also included.

[0087] Step S103: Determine whether the estimated pose parameters meet the first preset rule; wherein the first preset rule is used to screen the pose parameters of the target camera.

[0088] Specifically, if there is a first preset rule for filtering the pose parameters of the target camera, it is necessary to filter the estimated pose parameters according to the first preset rule, that is, to determine whether the estimated pose parameters meet the first preset rule. Assuming that the first preset rule is "the height of the target camera must be higher than 1m", after obtaining the estimated pose parameters generated by the camera pose estimation model, it is necessary to determine whether the height in the estimated pose parameters is higher than 1m to determine whether the estimated pose parameters meet the first preset rule.

[0089] If the estimated posture parameters meet the first preset rule, execute step S104 and subsequent steps.

[0090] Specifically, if the height in the estimated pose parameters is higher than 1m, it is judged that the estimated pose parameters meet the first preset rules. At this time, it means that the estimated pose parameters generated by the camera pose estimation model meet the requirements. According to the first preset rules, it can be directly determined that the estimated pose parameters generated by the camera pose estimation model correspond to a reasonable camera position and orientation. At this time, execute step S104 and subsequent steps.

[0091] If the estimated posture parameters do not meet the first preset rule, return to execute step S102 and subsequent steps.

[0092] Specifically, if the height in the estimated pose parameters is not higher than 1m, it is judged that the estimated pose parameters do not meet the first preset rules. At this time, it means that the estimated pose parameters generated by the camera pose estimation model do not meet the requirements. According to the first preset rules, it can be directly determined that the estimated pose parameters generated by the camera pose estimation model correspond to unreasonable camera position and orientation. In order to obtain the estimated pose parameters corresponding to the reasonable camera position and orientation, it is necessary to return to execute step S102 and subsequent steps.

[0093] Return to execute step S102, that is, re-input the input image into the camera pose estimation model again, so that the camera pose estimation model regenerates an estimated pose parameter. It should be noted that when the input image is the same, the output result of the camera pose estimation model is not completely the same, that is, the camera pose estimation model re-samples the feature vector corresponding to the input image from the Gaussian distribution in the latent space to obtain the sampled data, and based on the learned complex mapping relationship between the input image and the pose parameters, a new estimated pose parameter can be generated based on the re-sampled data. After obtaining the new estimated pose parameters, continue to execute the steps after step S102.

[0094] Step S104: triggering the target camera to perform image acquisition at a posture corresponding to the estimated posture parameters to obtain a target image.

[0095] Specifically, after the camera pose estimation model generates the estimated pose parameters, the target camera can be placed in the 3D scene at the position and orientation specified by the estimated pose parameters, and the target camera can be triggered to perform image acquisition at the position and orientation specified by the estimated pose parameters, thereby obtaining the target image, that is, the image acquisition process of the target camera at the specified position and orientation is simulated using the rendering engine to generate a simulated two-dimensional (2D) image. This simulated 2D image is generated based on the fixed internal parameters of the target camera (such as focal length, optical center position, etc.) and the external parameters of the target camera (i.e., the estimated pose parameters). During the rendering process, the rendering engine will consider the internal parameters of the target camera and the position and orientation of the target camera in the 3D scene, project the objects in the 3D scene onto the imaging plane of the target camera, and simulate the effect of the target camera lens and imaging system on light, and finally generate a target image that looks like the target camera captured in the real world.

[0096] Step S106: Obtain the quality score of the target image.

[0097] Specifically, when obtaining the quality score of the target image, there are two main considerations: one consideration is to focus only on the image quality of the target image itself, and the other consideration is to consider both the image quality of the target image and the pose quality of the corresponding estimated pose parameters. In this embodiment, a certain consideration method can be selected to obtain the quality score of the target image according to the specific application scenario and actual needs.

[0098] Specifically, the present embodiment may use a variety of methods to score the quality of the target image to obtain an image quality score of the target image. For example, one scoring method is to score the quality of the target image based on manual experience to obtain an image quality score of the target image. Alternatively, another scoring method is to train a neural network model based on deep learning or machine learning so that the neural network model can learn the complex mapping relationship between the input image and the output image quality score. After the training is completed, the target image is input into the trained neural network model, and the neural network model outputs an image quality score corresponding to the target image. In addition, the present embodiment may also obtain the image quality score of the target image according to other methods, and the specific method for obtaining the image quality score of the target image is not limited herein.

[0099] If the quality score is not greater than the first preset threshold, the process returns to step S102 and subsequent steps.

[0100] Specifically, when the quality score of the target image is not greater than the first preset threshold, that is, when the quality score of the target image is less than or equal to the first preset threshold, it means that the quality of the target image does not meet the requirements, and it is determined that the corresponding estimated pose parameters correspond to an unreasonable target camera position and orientation. In order to obtain the estimated pose parameters corresponding to the reasonable camera position and orientation, it is necessary to return to execute step S102 and subsequent steps, and return to execute step S102, that is, to re-input the input image into the camera pose estimation model, so that the camera pose estimation model regenerates a new estimated pose parameter. After obtaining the estimated pose parameters of the new target camera, continue to execute the steps after step S102.

[0101] If the quality score is greater than the first preset threshold, step S108 is executed: outputting the target image.

[0102] Specifically, when the quality score of the target image is greater than a preset threshold, it means that the quality of the target image meets the requirements, and the estimated pose parameters corresponding to the target image are determined to be reasonable pose parameters of the target camera. At this time, the quality of the target image captured by the target camera at the corresponding pose is relatively high, and the target image that meets the user requirements or application needs is output, thereby improving the quality of the output image and user satisfaction.

[0103] In some embodiments, obtaining a quality score of a target image includes:

[0104] Step S1065: input the target image into the image quality assessment model to obtain a quality score of the target image output by the image quality assessment model.

[0105] The image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes multiple second training samples, each second training sample includes multiple second sample images and annotated quality scores corresponding to each second sample image.

[0106] When the quality score of the target image only needs to consider the image quality of the target image itself, in order to improve the accuracy and efficiency of the quality score of the target image, this embodiment uses a sufficient number of second training samples to train an image quality assessment model, and the image quality assessment model is used to learn the mapping relationship between the input second sample image and the image quality score corresponding to the input second sample image. Therefore, after the image quality assessment model is trained, the target image is input into the image quality assessment model, and the image quality assessment model can quickly and accurately output the image quality score corresponding to the target image.

[0107] In some embodiments, Figure 4 FIG. 4 shows a flow chart of a sub-step of step S106 provided in this embodiment. Figure 4As shown, before step S1065, steps S1061 to S1064 are also included.

[0108] Step S1061: Acquire multiple second training samples.

[0109] Specifically, before training the image quality assessment model, multiple second training samples need to be prepared. In the process of preparing the second training samples, first, a large number of randomly generated pose parameters are collected, and the target camera is triggered to perform image acquisition under the randomly generated pose parameters to obtain a large number of second sample images. Then, the second sample images are annotated. During the annotation process, the image quality score of the second sample image is set between 0 and 5 points. The image quality score of each second sample image is manually or automatically scored to obtain the annotation quality score corresponding to each second sample image. After the image quality scores of all the second sample images are annotated, the second training data set is obtained.

[0110] It should be noted that setting the image quality score interval of the second sample image between 0-5 points is only a specific example of the image quality score interval of the second sample image. In this embodiment, the image quality score interval of the second sample image can be adjusted according to actual conditions and needs.

[0111] Step S1062: input the second sample image in each second training sample into the image quality assessment model to obtain a predicted quality score of each second sample image output by the image quality assessment model.

[0112] Wherein, the image quality assessment model is a second neural network model trained based on the second training data set. Convolutional Neural Networks (CNN) is a feedforward neural network that includes convolution calculations and has a deep structure, and performs very well in the fields of image recognition and natural language processing. When CNN is applied to the field of image quality assessment, the feature representation of image quality can be learned, and the feature representation of the learned image quality can be regressed using a fully connected layer (also called a multilayer perceptron (MLP)) to predict the image quality score. Based on the characteristics of CNN, the image quality assessment model of this embodiment can select CNN as the basic framework of the image quality assessment model, and use CNN to learn the complex mapping relationship between the second sample image input and the annotation quality score corresponding to the second sample image input.

[0113] Specifically, during the training process of the image quality assessment model, after the second sample image in each second training sample is input into the image quality assessment model, the image quality assessment model extracts the quality features of the second sample image according to the current model parameters, and uses the fully connected layer to regress the extracted quality features of the second sample image to obtain the predicted quality score of the second sample image. Exemplarily, the quality score interval of the second sample image is also set between 0 and 5 points.

[0114] Step S1063: Calculate the error between the predicted quality score of each second sample image and the labeled quality score corresponding to each second sample image.

[0115] Specifically, in order to evaluate the gap between the predicted quality score output by the image quality assessment model and the labeled quality score in the corresponding second training sample, the predicted quality score is compared with the labeled quality score in the corresponding second training sample, and the error between the predicted quality score and the labeled quality score in the corresponding second training sample is calculated, for example, the mean square error or root mean square error between the predicted quality score and the labeled quality score in the corresponding second training sample is calculated to obtain the difference between the prediction result of the image quality assessment model and the actual quality score.

[0116] Step S1064: iteratively updating the model parameters of the image quality assessment model according to the error until the image quality assessment model meets a preset second training stop condition.

[0117] Specifically, according to the calculated mean square error or root mean square error between the predicted quality score and the corresponding labeled quality score in the second training sample, the mean square error value or root mean square error value is back-propagated to each layer of the image quality assessment model through the back propagation algorithm, so as to calculate the gradient of the parameters of each layer. Then, the parameters of the image quality assessment model are updated according to these gradients using an optimization algorithm (such as Adam, SGD, etc.). The optimization algorithm adjusts the value of the parameter to minimize the error between the predicted quality score and the labeled quality score in the corresponding second training sample, traverses all the second training samples in the second training data set, and repeats the process from step S10312 to step S10314 multiple times until the image quality assessment model meets the preset second training stop condition. Among them, the second training stop condition includes: when the change in the error between the predicted quality score and the labeled quality score in the corresponding second training sample is less than a certain threshold, that is, the image quality assessment model has converged and the training stops; or, when the training rounds of the image quality assessment model in the training process reach the preset maximum training rounds, the training stops.

[0118] After the image quality assessment model is trained, when the quality score of the target image only needs to consider the image quality of the target image itself, the target image is input into the trained image quality assessment model, and the trained image quality assessment model outputs the quality score of the target image.

[0119] In some embodiments, Figure 5 A schematic flow chart of another sub-step of step S106 provided in this embodiment is shown, as shown in FIG. Figure 5 As shown, after step S1065, step S1066 and step S1067 are also included.

[0120] Step S1066: Perform a posture quality score on the estimated posture parameters according to the second preset rule to obtain a posture quality score corresponding to the estimated posture parameters.

[0121] Specifically, when the quality score of the target image considers both the quality of the target image and the posture quality of the corresponding estimated posture parameters, after obtaining the image quality score corresponding to the target image output by the image quality assessment model, it is also necessary to obtain the posture quality score corresponding to the estimated posture parameters. At this time, there is a second preset rule for the position and orientation of the camera.

[0122] In a specific example, assuming that the second preset rule is that when the camera is at a higher height, the pitch angle should be larger; when the camera is at a lower height, the pitch angle should be smaller, the camera's height and pitch angle are processed according to the second preset rule, and based on the processing results of the camera's height and pitch angle, the posture quality of the estimated posture parameters is scored to obtain a posture quality score corresponding to the estimated posture parameters.

[0123] Specifically, the second preset rule includes restrictions on the height and pitch angle of the camera, and the height of the camera is normalized to obtain a value of 0-1. Assuming that the maximum height in the 3D scene is 6m and the minimum height is 1m, the normalized height of the target camera is: h 归一化 =(h-1) / (6-1), where h is the current height of the target camera. Normalize the pitch angle of the target camera to get a value of 0-1. Assuming that the maximum pitch angle is 90 degrees and the minimum is 0 degrees, the normalized pitch angle of the target camera is: α 归一化 =α / 90, where α is the current pitch angle of the target camera. According to the second preset rule, write a function containing the camera's height and pitch angle f(h, α)=1-α 归一化 *h 归一化At this time, f(h, α) can model the relationship between the height and pitch angle of the target camera in the estimated pose parameters. If the pitch angle is large and the height is also high, the output pose quality score f(h, α) is low. If the pitch angle is small and the height is also low, the output pose quality score f(h, α) is high.

[0124] Step S1067: Obtain a quality score for the target image based on the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameters.

[0125] After obtaining the image quality score corresponding to the target image and the pose quality score corresponding to the estimated pose parameters, the image quality score corresponding to the target image and the pose quality score corresponding to the estimated pose parameters jointly determine the quality score of the target image.

[0126] In some embodiments, Figure 6 The flowchart of the sub-steps of step S1067 provided in this embodiment is shown as follows: Figure 6 As shown, step S1067 includes step S10671 and step S10672.

[0127] Step S10671: Obtain the first weight of the image quality score corresponding to the target image and the second weight of the posture quality score corresponding to the estimated posture parameter.

[0128] Specifically, the image quality score corresponding to the target image is recorded as Q1, the first weight of the image quality score Q1 corresponding to the target image is recorded as λ, the posture quality score corresponding to the estimated posture parameter is recorded as Q2, and the second weight of the posture quality score Q1 corresponding to the estimated posture parameter is recorded as 1-λ. This embodiment determines the specific values ​​of the first weight λ and the second weight 1-λ according to actual conditions and application requirements. If the quality score of the target image pays more attention to the image quality score Q1, a higher weight λ is given to the image quality score Q1. If the quality score of the target image pays more attention to the posture quality score Q2, a higher weight 1-λ is given to the posture quality score Q2.

[0129] Step S10672: Based on the first weight and the second weight, perform weighted summation on the image quality score and the posture quality score to obtain the quality score of the target image.

[0130] Specifically, the product of the image quality score Q1 and the first weight λ is calculated to obtain the first score Q1*λ; the product of the posture quality score Q2 and the second weight 1-λ is calculated to obtain the second score Q2*(1-λ). The quality score of the target image is recorded as Q, then Q=Q1*λ+Q2*(1-λ), that is, based on the first weight and the second weight, the image quality score and the posture quality score are weighted and summed to obtain the quality score of the target image.

[0131] In a specific example, Figure 7 The overall process diagram of the image output method provided by the embodiment of the present application is shown as follows: Figure 7 As shown, after building a specific 3D scene, sampling is performed in the Gaussian distribution of the feature space of the trained camera pose estimation model to generate an estimated pose parameter of a target camera. After generating the estimated pose parameters of the target camera, it is determined whether the estimated pose parameters meet the first preset rule. When it is determined that the estimated pose parameters meet the first preset rule, the target image captured when the parameters of the target camera are the estimated pose parameters is obtained. When it is determined that the estimated pose parameters do not meet the first preset rule, it returns to the step of generating the estimated pose parameters of the camera, and re-samples in the Gaussian distribution of the feature space of the trained camera pose estimation model to generate a new estimated pose parameter of the target camera. Re-determine whether the new estimated pose parameters meet the first preset rule, and continue with the subsequent steps after the new estimated pose parameters meet the first preset rule.

[0132] When it is determined that the estimated pose parameters meet the first preset rule, and the target image captured when the parameters of the target camera are the estimated pose parameters is obtained, the quality of the target image is scored based on the trained image quality assessment model to obtain an image quality score corresponding to the target image. The estimated pose parameters are scored based on the second preset rule to obtain a pose quality score corresponding to the estimated pose parameters.

[0133] After obtaining the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameters, according to the first weight corresponding to the image quality score corresponding to the target image and the second weight corresponding to the posture quality score corresponding to the estimated posture parameters (if the second weight is zero, it means that the quality score of the target image is only related to the image quality of the target image itself), based on the first weight and the second weight, the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameters are weightedly summed to obtain the quality score of the target image.

[0134] After obtaining the quality score of the target image, the quality score of the target image is compared with the first preset threshold. When the quality score of the target image is greater than the first preset threshold, it means that the quality of the target image meets the requirements, and the corresponding estimated pose parameters correspond to a reasonable camera position and orientation. At this time, the target image is output. When the quality score of the target image is not greater than the first preset threshold, that is, the quality score of the target image is less than or equal to the first preset threshold, it means that the quality of the target image does not meet the requirements, and the corresponding estimated pose parameters correspond to an unreasonable camera position and orientation. At this time, return to the step of generating the estimated pose parameters of the camera, re-sample the Gaussian distribution in the feature space of the trained camera pose estimation model, and generate a new estimated pose parameter of the target camera. Re-execute the subsequent steps of obtaining the new estimated pose parameters of the target camera.

[0135] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0136] Corresponding to the image output method described in the above embodiment, Figure 8 : is a structural block diagram of an image output device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0137] Reference Figure 8 , the image output device comprises:

[0138] The estimated pose parameter generation module 81 is used to generate the estimated pose parameters of the target camera based on the camera pose estimation model and the input image.

[0139] The target image acquisition module 82 is used to trigger the target camera to perform image acquisition at the posture corresponding to the estimated posture parameter to obtain the target image.

[0140] The quality score acquisition module 83 is used to acquire the quality score of the target image.

[0141] The first execution module 84 is used to return to the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps if the quality score is not greater than the first preset threshold.

[0142] The output module 85 is configured to output the target image if the quality score is greater than the first preset threshold.

[0143] Among them, the camera pose estimation model is a first neural network model trained based on a first training data set. When the input images are the same, the output results of the camera pose estimation model are not all the same. The first training data set includes multiple first training samples, each first training sample includes sample pose parameters and a first sample image captured by the target camera in a pose corresponding to the sample pose parameters, and the quality score of the first sample image is greater than a second preset threshold.

[0144] In some embodiments, the quality score acquisition module 83 is further used to:

[0145] The target image is input into an image quality assessment model to obtain a quality score of the target image output by the image quality assessment model; wherein the image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes multiple second training samples, each second training sample includes multiple second sample images and annotated quality scores corresponding to each second sample image.

[0146] In some embodiments, the apparatus further comprises:

[0147] A second training sample acquisition module 86, used to acquire the plurality of second training samples;

[0148] A predicted quality score generating module 87, configured to input the second sample image in each of the second training samples into the image quality assessment model to obtain a predicted quality score of each of the second sample images output by the image quality assessment model;

[0149] A second calculation module 88, configured to calculate an error between a predicted quality score of each second sample image and a labeling quality score corresponding to each second sample image;

[0150] The second iteration module 89 is used to iteratively update the model parameters of the image quality assessment model according to the error until the image quality assessment model meets a preset second training stop condition.

[0151] In some embodiments, the apparatus further comprises:

[0152] A first training sample acquisition module 810, configured to acquire a plurality of the first training samples;

[0153] An estimated pose parameter generating module 811 is used to input the first sample image in each of the first training samples into the camera pose estimation model to obtain an estimated pose parameter corresponding to each of the first sample images output by the camera pose estimation model;

[0154] A first calculation module 812, configured to calculate a loss between an estimated pose parameter corresponding to each of the first sample images and a sample pose parameter corresponding to each of the first sample images;

[0155] The first iteration module 813 is used to iteratively update the model parameters of the camera pose estimation model according to the loss until the camera pose estimation model meets a preset first training stop condition.

[0156] In some embodiments, the apparatus further comprises:

[0157] A first judgment module 814 is used to judge whether the estimated posture parameters meet the first preset rule; wherein the first preset rule is used to screen the posture parameters of the target camera;

[0158] A second execution module 815 is used for executing the step of triggering the target camera to acquire an image at a posture corresponding to the estimated posture parameter to obtain a target image and subsequent steps if the estimated posture parameter meets the first preset rule;

[0159] The third execution module 816 is used to return to the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps if the estimated pose parameters do not meet the first preset rule.

[0160] In some embodiments, the quality score acquisition module 83 is further used to:

[0161] Inputting the target image into an image quality assessment model to obtain an image quality score corresponding to the target image output by the image quality assessment model; wherein the image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes a plurality of second training samples, each second training sample includes a plurality of second sample images and annotated quality scores corresponding to each second sample image;

[0162] Performing a posture quality score on the estimated posture parameter according to a second preset rule to obtain a posture quality score corresponding to the estimated posture parameter;

[0163] A quality score of the target image is obtained according to the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameter.

[0164] In some embodiments, the quality score acquisition module 83 is further used to:

[0165] Obtaining a first weight of an image quality score corresponding to the target image and a second weight of a posture quality score corresponding to the estimated posture parameter;

[0166] Based on the first weight and the second weight, a weighted sum is performed on the image quality score and the posture quality score to obtain a quality score of the target image.

[0167] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0168] in addition, Figure 8 The image output device shown may be a software unit, a hardware unit, or a combination of software and hardware, which is built into an existing terminal device, or may be integrated into the terminal device as an independent accessory, or may exist as an independent terminal device.

[0169] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0170] Fig. 9 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Fig. 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Fig. 9 Only one processor is shown in the figure), a memory 91, and a computer program 92 stored in the memory 91 and executable on the at least one processor 90, wherein the processor 90 implements the steps of any of the above-mentioned image output method embodiments when executing the computer program 92.

[0171] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Fig. 9It is only an example of the terminal device 9 and does not constitute a limitation on the terminal device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0172] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0173] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. In other embodiments, the memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 9. Further, the memory 91 may also include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program, etc. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0174] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0175] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0177] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0178] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0179] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image output method, characterized in that: include: Generate the estimated pose parameters of the target camera based on the camera pose estimation model and the input image; Triggering the target camera to perform image acquisition at a posture corresponding to the estimated posture parameter to obtain a target image; Obtaining a quality score of the target image; If the quality score is not greater than the first preset threshold, returning to execute the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps; If the quality score is greater than the first preset threshold, outputting the target image; Among them, the camera pose estimation model is a first neural network model trained based on a first training data set. When the input images are the same, the output results of the camera pose estimation model are not all the same. The first training data set includes multiple first training samples, each first training sample includes sample pose parameters and a first sample image captured by the target camera in a pose corresponding to the sample pose parameters, and the quality score of the first sample image is greater than a second preset threshold.

2. The image output method according to claim 1, characterized in that: The obtaining the quality score of the target image includes: The target image is input into an image quality assessment model to obtain a quality score of the target image output by the image quality assessment model; wherein the image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes multiple second training samples, each second training sample includes multiple second sample images and annotated quality scores corresponding to each second sample image.

3. The image output method according to claim 2, characterized in that: Before inputting the target image into the image quality assessment model to obtain the quality score of the target image output by the image quality assessment model, the method further includes: Acquire the plurality of second training samples; Inputting the second sample image in each of the second training samples into the image quality assessment model to obtain a predicted quality score of each of the second sample images output by the image quality assessment model; Calculating an error between a predicted quality score of each second sample image and a labeled quality score corresponding to each second sample image; The model parameters of the image quality assessment model are iteratively updated according to the error until the image quality assessment model meets a preset second training stop condition.

4. The image output method according to any one of claims 1 to 3, characterized in that: Before generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image, the method further includes: Acquire a plurality of the first training samples; Inputting the first sample image in each of the first training samples into the camera pose estimation model to obtain an estimated pose parameter corresponding to each of the first sample images output by the camera pose estimation model; Calculating the loss between the estimated pose parameter corresponding to each of the first sample images and the sample pose parameter corresponding to each of the first sample images; Iteratively update model parameters of the camera pose estimation model according to the loss until the camera pose estimation model meets a preset first training stop condition.

5. The image output method according to any one of claims 1 to 3, characterized in that: After generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image, the method further includes: Determining whether the estimated pose parameters meet a first preset rule; wherein the first preset rule is used to screen the pose parameters of the target camera; If the estimated posture parameters meet the first preset rule, executing the step of triggering the target camera to perform image acquisition at the posture corresponding to the estimated posture parameters to obtain the target image and subsequent steps; If the estimated pose parameters do not meet the first preset rule, return to execute the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps.

6. The image output method according to claim 1, characterized in that: The obtaining the quality score of the target image includes: Inputting the target image into an image quality assessment model to obtain an image quality score corresponding to the target image output by the image quality assessment model; wherein the image quality assessment model is a second neural network model trained based on a second training data set, the second training data set includes a plurality of second training samples, each second training sample includes a plurality of second sample images and annotated quality scores corresponding to each second sample image; Performing a posture quality score on the estimated posture parameter according to a second preset rule to obtain a posture quality score corresponding to the estimated posture parameter; A quality score of the target image is obtained according to the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameter.

7. The image output method according to claim 6, characterized in that: The obtaining, according to the image quality score corresponding to the target image and the posture quality score corresponding to the estimated posture parameter, a quality score of the target image, comprises: Obtaining a first weight of an image quality score corresponding to the target image and a second weight of a posture quality score corresponding to the estimated posture parameter; Based on the first weight and the second weight, a weighted sum is performed on the image quality score and the posture quality score to obtain a quality score of the target image.

8. An image output device, characterized in that: include: An estimated pose parameter generation module is used to generate estimated pose parameters of a target camera based on a camera pose estimation model and an input image; A target image acquisition module, used for triggering the target camera to acquire an image at a posture corresponding to the estimated posture parameter to obtain a target image; A quality score acquisition module, used to acquire the quality score of the target image; An execution module, configured to return to the step of generating the estimated pose parameters of the target camera based on the camera pose estimation model and the input image and subsequent steps if the quality score is not greater than a first preset threshold; an output module, configured to output the target image if the quality score is greater than the first preset threshold; Among them, the camera pose estimation model is a first neural network model trained based on a first training data set. When the input images are the same, the output results of the camera pose estimation model are not all the same. The first training data set includes multiple first training samples, each first training sample includes sample pose parameters and a first sample image captured by the target camera in a pose corresponding to the sample pose parameters, and the quality score of the first sample image is greater than a second preset threshold.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image output method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image output method according to any one of claims 1 to 7 is implemented.

Images