LED Virtual Shooting Synthesis Method of Cardan Shaft Scenery System

Abstract

The present invention relates to the technical field of image synthesis, and specifically relates to an LED virtual shooting synthesis method for a cardan shaft scenic system. The method first obtains real photography images and virtual photography images; screens out virtual representative images from the virtual photography images; matches the real photography images and the virtual representative images, and analyzes the matching difference degree between the matched real photography images and virtual representative images; uses the proportion of the matching difference degree of each real photography image as an adjustment value to adjust the quantity difference between the real photography images and the virtual photography images, and obtains a sample increment for each real photography image; trains a sample classifier through the set of real photography images and the set of virtual photography images after increasing the samples, and determines whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier. The present invention optimizes the sample classifier by increasing the samples of the real photography images, and improves the visual realism of the virtual image.

Description

Technical Field

[0001] The present invention relates to the technical field of image synthesis, and particularly to an LED virtual shooting synthesis method for a gimbal scenery system. Background Art

[0002] The LED virtual shooting synthesis method for a gimbal scenery system realizes high-quality shooting effects by synchronizing the movements of multiple LED screens with a camera and displaying a virtual background in real time. The position information of the camera is fed back in real time to ensure the dynamic consistency between the background and the shooting object, enhancing the visual realism.

[0003] During the LED virtual shooting process, it is necessary to evaluate the authenticity of the captured images and then adjust various shooting parameters, etc. In existing methods, virtual images and real images are often compared through professional software. When comparing, various technical parameters such as resolution and frame rate are important indicators, but these parameters sometimes cannot fully reflect the complexity of the visual effects, especially in terms of showing delicate features such as layering. Therefore, commonly, a classifier is usually trained to distinguish between real images and virtual images. When the discrimination ability of the classifier is weak, it reflects that real images and virtual images are difficult to distinguish, correspondingly reflecting that the LED virtual shooting is relatively realistic. The training of the classifier is crucial. Usually, the classifier is trained with real images and virtual images, but under normal circumstances, the number of real images is relatively small, which will lead to poor classification effects of the classifier after training. Summary of the Invention

[0004] In order to solve the technical problem that the relatively small number of real images will lead to poor classification effects of the classifier after training, the purpose of the present invention is to provide an LED virtual shooting synthesis method for a gimbal scenery system, and the specific technical solution adopted is as follows:

[0005] In a first aspect, an embodiment of the present invention provides an LED virtual shooting synthesis method for a gimbal scenery system, and the method includes:

[0006] Obtain real photography images and virtual photography images simulated by an LED screen;

[0007] Screen out representative virtual representative images from the virtual photography images; match the real photography images and the virtual representative images, and analyze the matching difference degree between the matched real photography images and virtual representative images;

[0008] Use the proportion of the matching difference degree of each real photography image as an adjustment value to adjust the quantity difference between the real photography images and the virtual photography images, and obtain a sample increment for each real photography image;

[0009] Increase the samples of the real photographic images. The increased samples are the samples obtained by fusing the real photographic images and the virtual photographic images, and the number of the increased samples is the sample increment;

[0010] Train the sample classifier with the set of real photographic images and the set of virtual photographic images after increasing the samples, and judge whether the state of LED virtual shooting is real according to the classification error rate of the sample classifier.

[0011] Further, the screening of representative virtual representative images from the virtual photographic images includes:

[0012] Determine the distances between the virtual photographic images according to the structural differences between the virtual photographic images;

[0013] Perform hierarchical clustering on the virtual photographic images based on the distances between the virtual photographic images to obtain a hierarchical clustering tree;

[0014] Obtain the number of categories in each layer of the hierarchical clustering tree, compare the difference in the number of categories and the number of real photographic images, and use the layer with the smallest difference in quantity as the best layer of the hierarchical clustering tree; Mark all categories in the best layer as representative categories;

[0015] For any representative category, calculate the sum of the structural similarities between each virtual photographic image and other virtual photographic images in the belonging representative category, and use the virtual photographic image corresponding to the maximum value of the sum of the structural similarities as the virtual representative image in the representative category; Among them, the structural similarity characterizes the degree of structural similarity between photographic images.

[0016] Further, the analysis of the matching difference degree between the matched real photographic image and the virtual representative image includes:

[0017] For the virtual representative image, calculate the structural similarity between the virtual representative image and each other virtual photographic image in the belonging category of the virtual representative image, and arrange the obtained structural similarities in descending order to obtain a speculated similarity sequence;

[0018] For the real photographic image, calculate the structural similarity between each real photographic image and each other real photographic image, and arrange the obtained structural similarities in descending order to obtain a real similarity sequence;

[0019] Compare the speculated similarity sequence and the real similarity sequence to obtain the matching difference degree.

[0020] Further, the comparison of the speculated similarity sequence and the real similarity sequence to obtain the matching difference degree includes:

[0021] Compare the sub-sequence numbers of the matched real photographic images and virtual representative images in the real similarity sequence and the inferred similarity sequence that match, to obtain a matching degree; use the negative correlation mapping value of the matching degree as the matching difference degree between the matched real photographic image and the virtual representative image.

[0022] Further, the method for obtaining the sample increment is as follows:

[0023] Take the quantity difference value between the real photographic image and the virtual photographic image as the initial sample increment;

[0024] For each real photographic image, calculate the proportion of the matching difference degree of the real photographic image as a weighting value, and weight the initial sample increment to obtain the final sample increment for each real photographic image; wherein, the proportion of the matching difference degree of the real photographic image is the ratio of the matching difference degree of the current real photographic image to the sum of the matching difference degrees of all real photographic images.

[0025] Further, the method for obtaining the sample of the real photographic image is as follows:

[0026] For each real photographic image, calculate the structural similarity between the real photographic image and other real photographic images, and in descending order, take the real photographic images corresponding to the first sample increment of the structural similarity as the base images; calculate the sum value of the matching difference degrees between each real photographic image and the corresponding base image as the overall difference parameter of each real photographic image;

[0027] For any base image, take the matching difference degree between the base image and the corresponding real photographic image as the numerator, and take the overall difference parameter of the real photographic image as the denominator, and the ratio composed of the numerator and the denominator as the weighting weight of the real photographic image; take the difference between the preset constant threshold and the weighting weight of the real photographic image as the weighting weight of the base image;

[0028] Weight the base images through the weighting weight of the base images, weight the real photographic images through the weighting weight of the real photographic images, and fuse the weighted base images and real photographic images to obtain the sample of the real photographic image.

[0029] Further, the training of the sample classifier using the set of real photographic images and the set of virtual photographic images after increasing the samples includes:

[0030] A set of real photographic images and samples, and a set of real photographic images after adding samples; a set of virtual photographic images composed of virtual photographic images; respectively select a preset proportion of photographic images from the set of real photographic images after adding samples and the set of virtual photographic images as training data to train the SVM classifier to obtain a trained sample classifier.

[0031] Further, determining whether the state of LED virtual shooting is real according to the classification error rate of the sample classifier includes:

[0032] Use the photographic images other than the training data in the set of real photographic images after adding samples and the set of virtual photographic images as verification data; use the verification data as the input of the trained sample classifier, calculate the proportion of the number of verification data with classification errors as the classification error rate; when the classification error rate is greater than the preset error threshold, determine that the state of LED virtual shooting is real; when the classification error rate is less than or equal to the preset error threshold, determine that the state of LED virtual shooting is not real.

[0033] Further, matching the real photographic image and the virtual representative image includes:

[0034] Follow the maximum matching principle, use the structural similarity between the real photographic image and the virtual representative image as the matching edge weight value between the real photographic image and the virtual representative image, and perform one-to-one matching on the real photographic image and the virtual representative image.

[0035] Further, determining the distance between virtual photographic images according to the structural differences between virtual photographic images includes:

[0036] Use the negative correlation mapping value of the SSIM structural similarity between virtual photographic images as the distance between virtual photographic images.

[0037] In a second aspect, an LED virtual shooting synthesis system for a cardan shaft scenic system is provided. The system includes the following modules:

[0038] An image acquisition module for acquiring real photographic images and virtual photographic images simulated by an LED screen;

[0039] A difference analysis module for screening representative virtual representative images from the virtual photographic images; matching the real photographic images and the virtual representative images, and analyzing the matching difference degree between the matched real photographic images and virtual representative images;

[0040] An increment determination module for using the proportion of the matching difference degree of each real photographic image as an adjustment value to adjust the quantity difference between the real photographic images and the virtual photographic images to obtain the sample increment of each real photographic image;

[0041] A sample increasing module, configured to increase samples of the real photographic image, where the increased samples are samples obtained by fusing the real photographic image and the virtual photographic image, and the number of the increased samples is a sample increment;

[0042] A classifier training module, configured to train a sample classifier by using the set of real photographic images and the set of virtual photographic images after the samples are increased, and determine whether the state of LED virtual shooting is real according to the classification error rate of the sample classifier.

[0043] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor. An executable code is stored in the memory. When the processor executes the executable code, the method according to each possible implementation example of the first aspect is implemented.

[0044] In a fourth aspect, an embodiment of the present invention provides a computer program product, including: computer program code. When the computer program code runs on a computer, the computer is caused to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0045] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method according to each possible implementation example of the first aspect.

[0046] The embodiment of the present invention has at least the following beneficial effects:

[0047] The present invention relates to the field of image synthesis technology. In order to increase the samples of real photographic images, the method first screens out virtual representative images from virtual photographic images. Different virtual representative images reflect different scenes. By matching real photographic images and virtual representative images, the complex situation of matching each real photographic image with each virtual photographic image is reduced. Based on the matching difference degree between real photographic images and virtual representative images, the quantity difference between real photographic images and virtual photographic images is adjusted. The sample increment based on the quantity difference between real photographic images and virtual photographic images, with the matching difference degree of each real photographic image as the adjustment value, determines the sample increment corresponding to each real photographic image. Then, using the set of real photographic images and virtual photographic images after increasing the samples, the sample classifier is trained. According to the classification error rate of the sample classifier, it is judged whether the state of LED virtual shooting is real. Increasing the sample quantity of real photographic images to optimize the sample classifier, and using the optimized sample classifier to evaluate the classification and recognition of real photographic images and virtual photographic images. When the classification and recognition of the classifier for the two is low, it indicates that the virtual image has successfully imitated the real image in terms of overall layering and information representation, thus indicating a better virtual effect. This method helps to improve the visual realism of virtual images and greatly enhances the authenticity of virtual images. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 It is a flowchart of the LED virtual shooting synthesis method of the universal joint scenic system provided by an embodiment of the present invention;

[0050] Figure 2 It is a system module diagram of the LED virtual shooting synthesis system of the universal joint scenic system provided by an embodiment of the present invention;

[0051] Figure 3 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of the LED virtual shooting synthesis method of the universal joint scenic system according to the present invention.

[0053] In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0054] Among them, in the description of the embodiments of the present invention, unless otherwise specified, " / " means "or". For example, A / B may represent A or B. The "and / or" in the text is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present invention, "a plurality of" means two or more than two.

[0055] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and should not be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs.

[0057] The embodiments of the present invention will be described below with reference to the accompanying drawings. As those of ordinary skill in the art will know, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

[0058] The specific solution of the LED virtual shooting synthesis method of the cardan shaft scenic system provided by the present invention will be specifically described below with reference to the accompanying drawings.

[0059] Please refer to Figure 1 , which shows a flowchart of the steps of the LED virtual shooting synthesis method of the cardan shaft scenic system provided by an embodiment of the present invention. The method includes the following steps:

[0060] Step S100, obtaining a real photography image and a virtual photography image simulated by an LED screen.

[0061] First, obtain a real image of the image to be photographed, denoted as a real photography image. The real photography image can be obtained from the Internet, a professional database, a material library, or taken on-site.

[0062] Shooting the real scene from different angles can reflect different structural features of the real scene. For example: when shooting a forest land, when shooting from above, the density distribution characteristics of the forest land can be reflected, and when shooting obliquely, the height difference characteristics of the forest land can be reflected.

[0063] Then, by building an LED display system, a real-time rendering engine, and a camera tracking system, project the virtual scene onto the LED wall, and capture the motion data of the camera in real time, and synchronously render the background image that matches the camera's perspective, and finally obtain the virtual photography image simulated by the LED screen combined with the real scene in LED virtual shooting.

[0064] Furthermore, real photography images and virtual photography images are obtained. Among them, LED virtual shooting is to simulate various scenes through an LED screen in a photography studio, and at the same time shoot through a synchronized camera.

[0065] Step S200: Screen out representative virtual representative images from the virtual photography images; match the real photography images and the virtual representative images, and analyze the matching difference degree between the matched real photography images and virtual representative images.

[0066] Since the LED shooting image needs to obtain the image under the corresponding camera perspective according to the movement of the real-time position of the camera, which is often difficult to achieve in the actual scene. For example: shooting a scene where a protagonist is about to fall on the edge of a cliff, etc., it is not realistic to use the actual scene. Therefore, a background like this is created through the LED virtual shooting method, which requires a high similarity between the LED virtual shooting image and the real image.

[0067] Generally, the number of real images is small. Therefore, each real photography image can be regarded as the maximum display of the characteristics of a certain aspect of the real scene, while the number of virtual photography images is large. Therefore, by screening the virtual photography images, representative virtual representative images are screened out, and then subsequent analysis and calculation are carried out.

[0068] In the embodiment of the present invention, the step of screening out representative virtual representative images from the virtual photography images is as follows: Determine the distance between the virtual photography images according to the structural difference between the virtual photography images; perform hierarchical clustering on the virtual photography images based on the distance between the virtual photography images to obtain a hierarchical clustering tree; obtain the number of categories in each layer of the hierarchical clustering tree, compare the difference in the number of categories and the number of real photography images, and take the layer with the smallest number difference as the best layer of the hierarchical clustering tree. And mark all categories in the best layer as representative categories, thus obtaining multiple representative categories. For any representative category, the sum of the structural similarities between each virtual photography image and other virtual photography images in the same representative category, and the virtual photography image corresponding to the maximum value of the sum of the structural similarities is used as the virtual representative image in the representative category; among them, the structural similarity (Structural Similarity, SSIM) characterizes the structural similarity degree between photography images.

[0069] In some embodiments, determining the distance between virtual photography images according to the structural differences between the virtual photography images includes: using the negative correlation mapping value of the SSIM structural similarity between the virtual photography images as the distance between the virtual photography images. Specifically, the reciprocal of the SSIM structural similarity can be used as the distance between the virtual photography images.

[0070] In some embodiments, based on the distance between the virtual photography images, a hierarchical clustering tree is obtained by a bottom-up hierarchical clustering method.

[0071] In some embodiments, the method for obtaining the optimal layer is: calculating the absolute value of the difference between the number of categories in each layer of the hierarchical clustering tree and the real photography image, and the layer with the smallest absolute value of the difference is the layer with the smallest difference in quantity, and the layer corresponding to the smallest absolute value of the difference is used as the optimal layer of the hierarchical clustering tree.

[0072] After screening out representative virtual representative images from the virtual photography images, the real photography image and the virtual representative image are matched, and the matching difference degree between the matched real photography image and virtual representative image is further analyzed.

[0073] In the embodiments of the present invention, the KM matching algorithm is used to implement the matching of the real photography image and the virtual representative image. Among them, the KM matching algorithm is a method of one-to-one matching of all nodes on the left side with all nodes on the right side, and all nodes on the left and right sides belong to two categories.

[0074] In the embodiments of the present invention, the real photography image is used as the left node, and the virtual representative image is used as the right node. Each left node corresponds to a real photography image, and each right node corresponds to a virtual representative image. Each node on the left is connected to the nodes on the right by an edge, following the maximum matching principle, and a one-to-one matching relationship between the left node and the right node is obtained according to the KM matching. More specifically: following the maximum matching principle, the structural similarity between the real photography image and the virtual representative image is used as the matching edge weight between the real photography image and the virtual representative image, and the real photography image and the virtual representative image are matched one by one.

[0075] Since the number of left nodes and the number of right nodes are not necessarily the same, after obtaining the one-to-one matching relationship, additional operations need to be performed on the unmatched nodes. Specifically: if the number of left nodes is greater than or equal to the number of right nodes, that is, the number of real photographic images is greater than or equal to the number of categories, no additional operation is required; if the number of left nodes is less than the number of right nodes, that is, the number of real photographic images is less than the number of categories, then for the unmatched nodes on the right, calculate the structural similarity between each right node and all left nodes, and take the node corresponding to the maximum structural similarity as the corresponding node of this node, thus obtaining the matching relationship between each category and each real photographic image.

[0076] For the virtual representative image, calculate the structural similarity between the virtual representative image and each other virtual photographic image in the category to which the virtual representative image belongs, and arrange the obtained structural similarities in descending order to obtain a speculative similarity sequence. More specifically: for the virtual representative image of the representative category of each category, calculate the structural similarity between the virtual representative image of the representative category of other categories and the virtual representative image of this category, and sort them in descending order of structural similarity to obtain a speculative similarity sequence. Take this speculative similarity sequence as the similarity relationship of the corresponding virtual representative image, that is, the speculation of the similarity relationship of the corresponding virtual representative image.

[0077] For each real photographic image, calculate the structural similarity between each real photographic image and each other real photographic image. Similarly, according to the arrangement order of the speculative similarity sequence, arrange the obtained structural similarities in descending order to obtain a real similarity sequence. It should be noted that when the speculative similarity sequence is arranged in descending order of structural similarity, the corresponding real similarity sequence is also arranged in descending order. This real similarity sequence reflects the similarity between real photographic images.

[0078] Compare the speculative similarity sequence and the real similarity sequence to obtain a matching difference degree. Specifically: compare the sub-sequence numbers of the matched real photographic image and virtual representative image in the real similarity sequence and the speculative similarity sequence corresponding to the matched real photographic image and virtual representative image to obtain a matching degree; take the negative correlation mapping value of the matching degree as the matching difference degree between the matched real photographic image and virtual representative image.

[0079] Each real photographic image has its own corresponding matched virtual representative image. Each element in the real similarity sequence corresponds to a real photographic image, and each element in the speculative similarity sequence corresponds to a virtual representative image. And since each real photographic image has a matched virtual representative image, there are multiple pairs of matched real photographic images and virtual representative images in the real similarity sequence and the speculative similarity sequence.

[0080] Compare the sub-sequence numbers of the matched real photographic images and virtual representative images in the corresponding real similarity sequence and the speculated similarity sequence to obtain the matching degree; use the negative correlation mapping value of the matching degree as the matching difference degree between the matched real photographic images and virtual representative images. Specifically: Take any matched real photographic image and virtual representative image as the target real photographic image and the target virtual representative image; calculate the average of the absolute values of the differences between the sub-sequence numbers of all the matched real photographic images and virtual representative images in the real similarity sequence corresponding to the target real photographic image and the speculated similarity sequence corresponding to the target virtual representative image, and use the obtained average value as the matching degree; perform a negative correlation mapping on the matching degree, and use the negative correlation mapping value of the matching degree as the matching difference degree between the target real photographic image and the target virtual representative image that are matched.

[0081] In some embodiments, use the difference between the preset constant threshold and the normalized matching degree as the matching difference degree. In the embodiments of the present invention, the preset constant threshold is 1, and in other embodiments, the implementer can also set this value according to the actual situation. That is, in the embodiments of the present invention, the negative correlation mapping of the matching degree is achieved by subtracting the normalized matching degree from the constant 1.

[0082] This matching difference degree can represent the consistency between the similarity relationship among real photographic images and the similarity relationship among virtual representative images. The smaller the matching degree, the larger the matching difference degree, that is, the greater the difference between the virtual representative image and the corresponding real photographic image. The recognition ability of the classifier obtained by training can be enhanced by increasing the samples of such real photographic images.

[0083] Step S300: Use the proportion of the matching difference degree of each real photographic image as the adjustment value to adjust the quantity difference between the real photographic image and the virtual photographic image, and obtain the sample increment of each real photographic image.

[0084] First, use the quantity difference value between the real photographic image and the virtual photographic image as the initial sample increment. This initial sample increment is the basic value of the number of samples that need to be added.

[0085] Then, for each real photographic image, calculate the proportion of the matching difference degree of the real photographic image as the weighting value to weight the initial sample increment, and obtain the final sample increment of each real photographic image; wherein, the proportion of the matching difference degree of the real photographic image is the ratio of the matching difference degree of the current real photographic image to the sum of the matching difference degrees of all real photographic images.

[0086] Step S400, increase the samples of the real photographic image. The increased samples are the samples obtained by fusing the real photographic image and the virtual photographic image, and the number of the increased samples is the sample increment.

[0087] Take any real photographic image as the target real photographic image; take the real photographic images with the structural similarity to the target real photographic image ranked Top-k as the base images. More specifically: for each real photographic image, taking any real photographic image as the target real photographic image as an example, calculate the structural similarity between the target real photographic image and other real photographic images, and in the order from large to small, take the real photographic images corresponding to the first sample increment of the structural similarity as the base images; calculate the sum value of the matching difference degrees between each real photographic image and the corresponding base image as the overall difference parameter of each real photographic image. Among them, the value of k is the sample increment; for example, when the value of k is 5, take the first 5 real photographic images with the largest structural similarity to the target real photographic image as the base images.

[0088] For any base image, take the matching difference degree between the base image and the corresponding real photographic image as the numerator, and take the overall difference parameter of the real photographic image as the denominator. The ratio composed of the numerator and the denominator is used as the weighted weight of the real photographic image; take the difference between the preset constant threshold and the weighted weight of the real photographic image as the weighted weight of the base image. In the embodiment of the present invention, the value of the preset constant threshold is 1, and in other embodiments, the implementer adjusts this value according to the actual situation.

[0089] Weight the base images through the weighted weights of the base images, weight the real photographic images through the weighted weights of the real photographic images, and fuse the weighted base images and real photographic images to obtain the samples of the real photographic images. There are as many samples of the real photographic images as there are base images.

[0090] Step S500, train the sample classifier through the set of real photographic images and the set of virtual photographic images after increasing the samples, and judge whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier.

[0091] To improve the performance and generalization ability of the sample classifier, train the classifier by combining the real sample data updated incrementally and the sample data taken in the LED virtual environment. This strategy can not only utilize the diversity and complexity of the real samples, but also rely on the controllability and scalability of the virtual samples, so as to achieve a more accurate classification effect in different application scenarios. In this way, the classifier can learn a more comprehensive feature representation, and can better adapt to the newly emerging data, reduce the risk of overfitting, and improve the stability and reliability of the model in practical applications.

[0092] A set of real photographic images and samples forms a set of real photographic images after adding samples, and a set of virtual photographic images forms a set of virtual photographic images; respectively select a preset proportion of photographic images from the set of real photographic images after adding samples and the set of virtual photographic images as training data to train the SVM classifier to obtain a trained sample classifier. In the embodiment of the present invention, the preset proportion is 70%, and in other embodiments, the implementer can adjust this ratio according to the actual situation.

[0093] Use 70% of the data in the set of real photographic images after adding samples and the set of virtual photographic images as training data to train a sample classifier. The sample classifier in the embodiment of the present invention uses an SVM classifier. In the embodiment of the present invention, 70% of the images can be randomly selected from both the set of real photographic images after adding samples and the set of virtual photographic images as training data.

[0094] By enhancing the classification performance of the sample classifier, if a sample classifier with good performance cannot distinguish real samples and virtual samples well, it indicates that the virtual effect of the virtual samples is good.

[0095] Train the sample classifier through training data. Use the photographic images in the set of real photographic images after adding samples and the set of virtual photographic images other than the training data as verification data. Use the verification data as the input of the trained sample classifier, and calculate the proportion of the number of misclassified verification data as the classification error rate; when the classification error rate is greater than the preset error threshold, judge that the state of LED virtual shooting is real; when the classification error rate is less than or equal to the preset error threshold, determine that the state of LED virtual shooting is not real. In the embodiment of the present invention, the value of the preset error threshold is 0.9, and in other embodiments, the implementer can adjust this value according to the actual situation.

[0096] Use the remaining 30% of the data in the set of real photographic images after adding samples and the set of virtual photographic images as verification data. Use the verification data as the input of the sample classifier, and calculate the ratio of the number of misclassified verification data to the total number of verification data as the classification error rate. The larger the classification error rate, the more difficult it is for the sample classifier to distinguish real samples and virtual samples, and the better the virtual effect of the virtual samples. When the classification error rate is greater than the preset error threshold, it indicates that the current virtual effect is already good and there is no need to adjust the shooting parameters. When the classification error rate is less than or equal to the preset error threshold, it indicates that the authenticity of the simulated virtual photographic images is still poor. At this time, the shooting effect of better virtual shooting images can be obtained by adjusting the LED brightness, optimizing the color temperature, using dynamic light sources, etc.

[0097] Please refer to Figure 2, which shows the system module diagram of the LED virtual shooting synthesis system of the universal shaft scenic system provided by an embodiment of the present invention. The system includes the following modules:

[0098] An image acquisition module 10, configured to acquire real photographic images and virtual photographic images simulated by an LED screen;

[0099] A difference analysis module 20, configured to screen representative virtual representative images from the virtual photographic images; match the real photographic images and the virtual representative images, and analyze the matching difference degree between the matched real photographic images and virtual representative images;

[0100] An increment determination module 30, configured to use the proportion of the matching difference degree of each real photographic image as an adjustment value to adjust the quantity difference between the real photographic images and the virtual photographic images, and obtain the sample increment of each real photographic image;

[0101] A sample addition module 40, configured to add samples of the real photographic images. The added samples are samples obtained by fusing the real photographic images and the virtual photographic images, and the number of the added samples is the sample increment;

[0102] A classifier training module 50, configured to train a sample classifier through the set of real photographic images and the set of virtual photographic images after adding samples, and determine whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier.

[0103] Optionally, the transmission medium may be a wired link, such as but not limited to, coaxial cable, optical fiber, digital subscriber line, etc., or a wireless link, such as but not limited to, Wireless Fidelity (WIFI), Bluetooth, mobile device network, etc.

[0104] It should be noted that: for the device provided in the above embodiment, only the above division of each functional module is used for illustration. In actual applications, the above functions may be allocated to different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above.

[0105] Figure 3 is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Exemplarily, as Figure 3 shown, the computer device 600 includes: a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and running on the processor 620. When the processor 620 executes the computer program 630, the computer device can execute any of the LED virtual shooting synthesis methods of the universal shaft scenic system described above.

[0106] In addition, an embodiment of the present invention also protects a device, which may include a memory and a processor. Among them, executable program code is stored in the memory, and the processor is used to call and execute the executable program code to execute the LED virtual shooting synthesis method of the gimbal scene system provided by the embodiment of the present invention.

[0107] The embodiment of the present invention can divide the functions of the device according to the above method examples. For example, it can correspond to each functional module, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0108] In the case of dividing each module according to each function, the device may further include a signal uploading module, a determination module, an adjustment module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be cited in the function description of the corresponding functional module, and will not be repeated here.

[0109] It should be understood that the device provided by the embodiment of the present invention is used to execute the above-mentioned LED virtual shooting synthesis method of the gimbal scene system, so the same effect as the above implementation method can be achieved.

[0110] In the case of adopting an integrated unit, the device may include a processing module and a storage module. Among them, when the device is applied to a device, the processing module can be used to control and manage the actions of the device. The storage module can be used to support the device to execute mutual program code, etc. Among them, the processing module can be a processor or a controller, which can implement or execute various exemplary logic blocks, modules and circuits described in combination with the disclosure of the present invention. The processor can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory.

[0111] In addition, the device provided by the embodiment of the present invention may specifically be a chip, a component or a module. The chip may include a processor and a memory connected thereto; among them, the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the LED virtual shooting synthesis method of the gimbal scene system provided by the above embodiment.

[0112] An embodiment of the present invention also provides a computer-readable storage medium, in which computer program code is stored. When the computer program code runs on a computer, the computer is enabled to execute the above-related method steps to implement the LED virtual shooting synthesis method of the gimbal scenic system provided in the above embodiment.

[0113] An embodiment of the present invention also provides a computer program product. When the computer program product runs on a computer, the computer is enabled to execute the above-related steps to implement the LED virtual shooting synthesis method of the gimbal scenic system provided in the above embodiment.

[0114] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present invention are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be elaborated here. Through the description of the above embodiments, those skilled in the art can understand that for the convenience and conciseness of description, only the above-mentioned division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by the present invention, it should be understood that the disclosed device and method can be implemented in other ways.

[0115] The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0116] It should also be noted that in this article, the terms "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or terminal device including the said element.

[0117] It should be noted that the above-mentioned order of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0118] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments.

[0119] The above content is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should be covered within the protection scope of the present invention.

Claims

1. A method for LED virtual shooting synthesis of a universal shaft scenic system, characterized in that The method includes the following steps: Obtain a real photographic image and a virtual photographic image simulated by an LED screen; Select representative virtual representative images from the virtual photographic images; match the real photographic images and the virtual representative images, and analyze the matching difference degree between the matched real photographic images and virtual representative images; wherein, the method for obtaining the matching difference degree is as follows: for a virtual representative image, calculate the structural similarity between the virtual representative image and each other virtual photographic image in the category to which the virtual representative image belongs, arrange the obtained structural similarities in descending order to obtain a speculated similarity sequence; for a real photographic image, calculate the structural similarity between each real photographic image and each other real photographic image, arrange the obtained structural similarities in descending order to obtain a real similarity sequence; compare the sub-sequence numbers of the matched real photographic image and virtual representative image in the real similarity sequence and the speculated similarity sequence corresponding to the matched real photographic image and virtual representative image to obtain a matching degree; use the negative correlation mapping value of the matching degree as the matching difference degree between the matched real photographic image and virtual representative image; Use the proportion of the matching difference degree of each real photographic image as an adjustment value to adjust the quantity difference between the real photographic images and the virtual photographic images to obtain a sample increment for each real photographic image; Increase the samples of the real photographic images. The increased samples are samples obtained by fusing the real photographic images and the virtual photographic images, and the number of the increased samples is the sample increment; Train a sample classifier with the set of real photographic images and the set of virtual photographic images after increasing the samples, and judge whether the state of LED virtual shooting is real according to the classification error rate of the sample classifier.

2. The LED virtual shooting synthesis method of the cardan shaft scene production system according to claim 1, wherein The step of selecting representative virtual representative images from the virtual photographic images includes: Determine the distance between virtual photographic images according to the structural differences between virtual photographic images; Perform hierarchical clustering on the virtual photographic images based on the distance between virtual photographic images to obtain a hierarchical clustering tree; Obtain the number of categories in each layer of the hierarchical clustering tree, compare the number difference with the number of real photographic images, and use the layer with the smallest number difference as the best layer of the hierarchical clustering tree; mark all categories in the best layer as representative categories; For any representative category, for each virtual photographic image and the sum of the structural similarities between other virtual photographic images in the representative category to which it belongs, use the virtual photographic image corresponding to the maximum value of the sum of the structural similarities as the virtual representative image in the representative category; wherein, the structural similarity characterizes the structural similarity degree between photographic images.

3. The LED virtual shooting synthesis method of the cardan shaft scene production system according to claim 1, characterized in that, The method for obtaining the sample increment is as follows: Use the quantity difference value between the real photographic images and the virtual photographic images as the initial sample increment; For each real photographic image, calculate the proportion of the matching difference degree of the real photographic image as a weighting value, and weight the initial sample increment to obtain the final sample increment of each real photographic image; wherein, the proportion of the matching difference degree of the real photographic image is the ratio of the matching difference degree of the current real photographic image to the sum of the matching difference degrees of all real photographic images.

4. The LED virtual shooting synthesis method of the cardan shaft scene production system according to claim 1, characterized in that, The method for obtaining samples of real photographic images is as follows: For each real photographic image, calculate the structural similarity between the real photographic image and other real photographic images, and in the order from large to small, take the real photographic images corresponding to the top sample increment of the structural similarity as the base images; Calculate the sum value of the matching difference degrees between each real photographic image and the corresponding base image as the overall difference parameter of each real photographic image; For any base image, take the matching difference degree between the base image and the corresponding real photographic image as the numerator, and take the overall difference parameter of the real photographic image as the denominator. The ratio composed of the numerator and the denominator is used as the weighting weight of the real photographic image; take the difference between the preset constant threshold and the weighting weight of the real photographic image as the weighting weight of the base image; Weight the base image through the weighting weight of the base image, weight the real photographic image through the weighting weight of the real photographic image, and fuse the weighted base image and the real photographic image to obtain the sample of the real photographic image.

5. The LED virtual shooting synthesis method of the universal shaft scene production system according to claim 1, characterized in that Training the sample classifier with the set of real photographic images and the set of virtual photographic images after adding samples includes: Form a set of real photographic images after adding samples from the real photographic images and the samples, and form a set of virtual photographic images from the virtual photographic images; respectively select a preset proportion of the photographic images in the set of real photographic images and the set of virtual photographic images after adding samples as training data, and train the SVM classifier to obtain a trained sample classifier.

6. The LED virtual shooting synthesis method of the universal shaft scene production system according to claim 1, characterized in that, Judging whether the state of LED virtual shooting is real according to the classification error rate of the sample classifier includes: Take the photographic images other than the training data in the set of real photographic images and the set of virtual photographic images after adding samples as verification data; take the verification data as the input of the trained sample classifier, and calculate the proportion of the number of misclassified verification data as the classification error rate; when the classification error rate is greater than the preset error threshold, determine that the state of LED virtual shooting is real; when the classification error rate is less than or equal to the preset error threshold, determine that the state of LED virtual shooting is not real.

7. The LED virtual shooting synthesis method of the universal shaft scene production system according to claim 1, characterized in that Matching the real photographic image and the virtual representative image includes: Following the maximum matching principle, using the structural similarity between the real photographic image and the virtual representative image as the matching edge weight value between the real photographic image and the virtual representative image, and perform one-to-one matching on the real photographic image and the virtual representative image.

8. The LED virtual shooting synthesis method of the cardan shaft scene production system according to claim 2, wherein Determining the distance between virtual photographic images according to the structural difference between virtual photographic images includes: Using the negative correlation mapping value of the SSIM structural similarity between virtual photographic images as the distance between virtual photographic images.

Images