LED virtual shooting synthesis method of cardan shaft scenery system

Abstract

The invention relates to the technical field of image synthesis, in particular to an LED virtual shooting synthesis method of a universal shaft scenery system. The method comprises the following steps: firstly, acquiring a real photographic image and a virtual photographic image; screening out a virtual representative image from the virtual photographic image; matching the real photographic image with the virtual representative image, and analyzing the matching difference degree between the matched real photographic image and virtual representative image; adjusting the number difference between the real photographic images and the virtual photographic images by taking the proportion of the matching difference degree of each real photographic image as an adjustment value to obtain a sample increment of each real photographic image; and training a sample classifier through the real photographic image set and the virtual photographic image set after the samples are added, and judging whether the state of LED virtual shooting is real or not according to the classification error rate of the sample classifier. According to the method, the visual reality sense of the virtual influence is improved by increasing the samples of the real photographic image and optimizing the sample classifier.

Description

Technical Field

[0001] The invention relates to the technical field of image synthesis, and in particular to an LED virtual shooting synthesis method of a universal axis scene making system. Background Art

[0002] The LED virtual shooting synthesis method of the universal axis framing system is to synchronize multiple LED screens with the movement of the camera to display the virtual background in real time, thereby achieving high-quality shooting effects. The position information of the camera is fed back in real time to ensure that the background is consistent with the dynamics of the subject, enhancing the visual realism.

[0003] During the LED virtual shooting process, it is necessary to evaluate the authenticity of the shot images, and then adjust the various shooting parameters. The existing methods often use professional software to compare virtual images and real images. When making comparisons, various technical parameters such as resolution and frame rate are important indicators, but these parameters sometimes cannot fully reflect the complexity of visual effects, especially in terms of expressing delicate features such as layering. Therefore, it is common to distinguish between real images and virtual images by training classifiers. When the classifier's ability to distinguish is weak, it reflects that the real image and the virtual image are difficult to distinguish, which corresponds to the LED virtual shooting being more realistic. The training of the classifier is crucial. Usually, the classifier is trained through real images and virtual images, but under normal circumstances, the number of real images is relatively small, which will lead to poor classification results after the classifier is trained. Summary of the invention

[0004] In order to solve the technical problem that there are relatively few real images, which leads to poor classification effect of the classifier after training, the purpose of the present invention is to provide an LED virtual shooting synthesis method of a universal axis scene making system, and the technical solution adopted is as follows:

[0005] In a first aspect, an embodiment of the present invention provides an LED virtual shooting synthesis method of a gimbal-axis framing system, the method comprising:

[0006] Acquire real photographic images and virtual photographic images simulated by LED screens;

[0007] Selecting a representative virtual representative image from the virtual photographic image; matching the real photographic image and the virtual representative image, and analyzing the matching difference between the matched real photographic image and the virtual representative image;

[0008] Taking the proportion of the matching difference of each real photographic image as an adjustment value, adjusting the quantity difference between the real photographic image and the virtual photographic image to obtain the sample increment of each real photographic image;

[0009] Adding samples of the real photographic image, where the added samples are samples obtained by fusing the real photographic image and the virtual photographic image, and the number of the added samples is the sample increment;

[0010] By adding the real photographic image set and the virtual photographic image set after the sample, the sample classifier is trained, and according to the classification error rate of the sample classifier, it is judged whether the state of the LED virtual shooting is real.

[0011] Furthermore, the step of selecting a representative virtual representative image from the virtual photographic image includes:

[0012] Determining the distance between the virtual photographic images according to the structural differences between the virtual photographic images;

[0013] Based on the distance between virtual photographic images, the virtual photographic images are hierarchically clustered to obtain a hierarchical clustering tree;

[0014] Obtain the number of categories in each layer of the hierarchical clustering tree, compare the number of categories with the number of real photographic images, and take the layer with the smallest number difference as the best layer of the hierarchical clustering tree; mark all categories in the best layer as representative categories;

[0015] For any representative category, the sum of the structural similarities between each virtual photographic image and other virtual photographic images in the representative category to which it belongs is taken, and the virtual photographic image corresponding to the maximum value of the sum of the structural similarities is taken as the virtual representative image in the representative category; wherein the structural similarity represents the degree of structural similarity between photographic images.

[0016] Furthermore, the analyzing and matching the matching difference between the real photographic image and the virtual representative image includes:

[0017] For the virtual representative image, the structural similarity between the virtual representative image and each other virtual photographic image in the category to which the virtual representative image belongs is calculated, and the obtained structural similarities are arranged in order of magnitude to obtain an inferred similarity sequence;

[0018] For real photographic images, the structural similarity between each real photographic image and each other real photographic image is calculated, and the obtained structural similarities are arranged in order of size to obtain a real similarity sequence;

[0019] Compare the inferred similarity sequence and the true similarity sequence to obtain the matching difference.

[0020] Furthermore, the comparing the inferred similarity sequence and the real similarity sequence to obtain the matching difference degree includes:

[0021] Compare the order numbers of the matched real photographic image and the virtual representative image in the real similarity sequence and the inferred similarity sequence corresponding to the matched real photographic image and the virtual representative image to obtain the matching degree; and use the negative correlation mapping value of the matching degree as the matching difference between the matched real photographic image and the virtual representative image.

[0022] Furthermore, the method for obtaining the sample increment is:

[0023] The quantitative difference between the real photographic image and the virtual photographic image is used as the initial sample increment;

[0024] For each real photographic image, the proportion of the matching difference of the real photographic image is calculated as a weighted value, and the initial sample increment is weighted to obtain the final sample increment of each real photographic image; wherein, the proportion of the matching difference of the real photographic image is the ratio of the matching difference of the current real photographic image to the sum of the matching differences of all real photographic images.

[0025] Furthermore, the method for obtaining samples of real photographic images is as follows:

[0026] For each real photographic image, the structural similarity between the real photographic image and other real photographic images is calculated, and the real photographic images corresponding to the structural similarity of the previous sample increment are used as the basic images in descending order; the sum of the matching difference between each real photographic image and the corresponding basic image is calculated as the overall difference parameter of each real photographic image;

[0027] For any basic image, the matching difference between the basic image and the corresponding real photographic image is used as the numerator, the overall difference parameter of the real photographic image is used as the denominator, and the ratio formed by the numerator and the denominator is used as the weighted weight of the real photographic image; the difference between the preset constant threshold and the weighted weight of the real photographic image is used as the weighted weight of the basic image;

[0028] The base image is weighted by the weighted weight of the base image, and the real photographic image is weighted by the weighted weight of the real photographic image. The weighted base image and the real photographic image are fused to obtain a sample of the real photographic image.

[0029] Furthermore, the training of the sample classifier by adding the real photographic image set and the virtual photographic image set after the sample is added includes:

[0030] A real photographic image set after adding samples is composed of real photographic images and samples, and a virtual photographic image set is composed of virtual photographic images; photographic images of preset proportions in the real photographic image set after adding samples and the virtual photographic image set are respectively selected as training data to train the SVM classifier to obtain a trained sample classifier.

[0031] Further, judging whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier includes:

[0032] The photographic images other than the training data in the real photographic image set and the virtual photographic image set after adding samples are used as verification data; the verification data is used as the input of the trained sample classifier, and the proportion of the number of verification data with classification errors is calculated as the classification error rate; when the classification error rate is greater than the preset error threshold, it is determined that the state of the LED virtual photography is real; when the classification error rate is less than or equal to the preset error threshold, it is determined that the state of the LED virtual photography is not real.

[0033] Furthermore, matching the real photographic image and the virtual representative image includes:

[0034] Following the maximum matching principle, the structural similarity between the real photographic image and the virtual representative image is used as the matching edge weight between the real photographic image and the virtual representative image, and a one-to-one matching is performed between the real photographic image and the virtual representative image.

[0035] Furthermore, determining the distance between the virtual photographic images according to the structural differences between the virtual photographic images includes:

[0036] The negative correlation mapping value of the SSIM structural similarity between the virtual photographic images is used as the distance between the virtual photographic images.

[0037] In a second aspect, an LED virtual shooting synthesis system of a gimbal-axis framing system is provided, the system comprising the following modules:

[0038] An image acquisition module, used to acquire real photographic images and virtual photographic images simulated by an LED screen;

[0039] a difference analysis module, used to select a representative virtual representative image from the virtual photographic image; match the real photographic image with the virtual representative image, and analyze the matching difference between the matched real photographic image and the virtual representative image;

[0040] An increment determination module, used to adjust the quantity difference between the real photographic image and the virtual photographic image by taking the proportion of the matching difference degree of each real photographic image as an adjustment value, so as to obtain a sample increment of each real photographic image;

[0041] A sample adding module, used for adding samples of the real photographic image, the added samples are samples obtained by fusing the real photographic image and the virtual photographic image, and the number of added samples is the sample increment;

[0042] The classifier training module is used to train the sample classifier by adding the real photographic image set and the virtual photographic image set after the sample is added, and judge whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier.

[0043] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, various possible embodiments of the first aspect are implemented.

[0044] In a fourth aspect, an embodiment of the present invention provides a computer program product, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0045] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute various possible implementations of the first aspect.

[0046] The embodiments of the present invention have at least the following beneficial effects:

[0047] The present invention relates to the field of image synthesis technology. In order to increase the samples of real photographic images, the method first selects virtual representative images from virtual photographic images, and different virtual representative images reflect different scenes, match the real photographic images and virtual representative images, and reduce the complexity of matching the real photographic images with each virtual photographic image. The number difference between the real photographic image and the virtual photographic image is adjusted by the matching difference between the real photographic image and the virtual representative image, and the sample increment based on the number difference between the real photographic image and the virtual photographic image is determined by the matching difference of each real photographic image as an adjustment value. Then, by increasing the real photographic image set and the virtual photographic image set after the sample is added, the sample classifier is trained, and the state of the LED virtual shooting is judged whether it is real according to the classification error rate of the sample classifier. The number of samples of the real photographic image is increased to optimize the sample classifier, and the optimized sample classifier is used to evaluate the classification and recognition of the real photographic image and the virtual photographic image. When the classifier has a low classification recognition of the two, it means that the virtual image successfully imitates the real image in terms of overall hierarchy and information expression, which indicates that the virtual effect is better. This method helps to improve the visual realism of the virtual image and greatly improves the authenticity of the virtual image. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A method flow chart of an LED virtual shooting synthesis method of a universal axis scene making system provided by an embodiment of the present invention;

[0050] Figure 2 A system module diagram of an LED virtual shooting synthesis system of a universal axis scene making system provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the LED virtual shooting synthesis method of the universal axis scene making system proposed by the present invention, its specific implementation method, structure, characteristics and effects are described in detail as follows in combination with the accompanying drawings and preferred embodiments.

[0053] In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0054] Among them, in the description of the embodiments of the present invention, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" refers to two or more than two.

[0055] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0056] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0057] The embodiments of the present invention are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0058] The specific scheme of the LED virtual shooting synthesis method of the universal axis scene making system provided by the present invention is described in detail below with reference to the accompanying drawings.

[0059] See also Figure 1 , which shows a flowchart of the steps of the LED virtual shooting synthesis method of the universal axis scene making system provided by an embodiment of the present invention, the method comprises the following steps:

[0060] Step S100, acquiring a real photographic image and a virtual photographic image simulated by an LED screen.

[0061] First, a real image to be photographed is obtained, which is recorded as a real photographic image. The real photographic image can be obtained from the Internet, a professional database, a material library, or on-site photography.

[0062] Shooting a real scene from different angles can reflect different structural features of the real scene. For example, when shooting a forest, when shooting from above, the density of the forest can be reflected, and when shooting from an angle, the height difference of the forest can be reflected.

[0063] Then, by building an LED display system, a real-time rendering engine and a camera tracking system, the virtual scene is projected onto the LED wall, and the camera's motion data is captured in real time, and the background picture that matches the camera's perspective is rendered synchronously, finally obtaining a virtual photographic image simulated by the LED screen combined with the real scene in LED virtual shooting.

[0064] Then, real photographic images and virtual photographic images are obtained. Among them, LED virtual shooting is to simulate various scenes through LED screens in a studio and shoot through synchronized cameras at the same time.

[0065] Step S200, selecting a representative virtual representative image from the virtual photographic image; matching the real photographic image and the virtual representative image, and analyzing the matching difference between the matched real photographic image and the virtual representative image.

[0066] Since the LED shooting image needs to be based on the real-time position of the camera to obtain the image under the corresponding camera perspective, it is often difficult to do so in real scenes. For example, it is unrealistic to shoot a scene where a protagonist is about to fall off a cliff, and then create such a background through LED virtual shooting methods, which requires that the LED virtual shooting image is highly similar to the real image.

[0067] Usually, there are fewer real images, so each real photographic image can be regarded as a maximized display of a certain aspect of the real scene. There are more virtual photographic images, so by screening the virtual photographic images, representative virtual representative images are screened out, and then subsequent analysis and calculation are carried out.

[0068] In an embodiment of the present invention, the steps of selecting representative virtual representative images from virtual photographic images are as follows: determining the distance between virtual photographic images based on the structural differences between virtual photographic images; hierarchically clustering the virtual photographic images based on the distance between virtual photographic images to obtain a hierarchical clustering tree; obtaining the number of categories in each layer of the hierarchical clustering tree, comparing the number of categories with the number of real photographic images, and taking the layer with the smallest number difference as the best layer of the hierarchical clustering tree. And marking all categories in the best layer as representative categories, so as to obtain multiple representative categories. For any representative category, the sum of the structural similarities of each virtual photographic image and other virtual photographic images in the representative category to which it belongs is taken, and the virtual photographic image corresponding to the maximum value of the sum of the structural similarities is taken as the virtual representative image in the representative category; wherein the structural similarity (SSIM) characterizes the degree of structural similarity between photographic images.

[0069] In some embodiments, determining the distance between virtual photographic images according to the structural differences between the virtual photographic images includes: using the negative correlation mapping value of the SSIM structural similarity between the virtual photographic images as the distance between the virtual photographic images. Specifically, the inverse of the SSIM structural similarity can be used as the distance between the virtual photographic images.

[0070] In some embodiments, a hierarchical clustering tree is obtained through a bottom-up hierarchical clustering method based on the distance between virtual photographic images.

[0071] In some embodiments, the method for obtaining the optimal layer is: calculate the absolute value of the difference between the number of categories in each layer in the hierarchical clustering tree and the real photographic image, the layer with the smallest absolute value of the difference is the layer with the smallest quantity difference, and the layer corresponding to the smallest absolute value of the difference is taken as the optimal layer of the hierarchical clustering tree.

[0072] After a representative virtual representative image is screened out from the virtual photographic image, the real photographic image and the virtual representative image are matched, and the matching difference between the matched real photographic image and the virtual representative image is further analyzed.

[0073] In the embodiment of the present invention, the KM matching algorithm is used to achieve the matching of the real photographic image and the virtual representative image. The KM matching algorithm is a method of matching all nodes on the left side with all nodes on the right side one-to-one, and all nodes on the left side and all nodes on the right side belong to two categories.

[0074] In the embodiment of the present invention, the real photographic image is used as the left node, and the virtual representative image is used as the right node. Each left node corresponds to a real photographic image, and each right node corresponds to a virtual representative image. Each node on the left is connected to the right node by an edge, and the maximum matching principle is followed, and a one-to-one matching relationship between the left node and the right node is obtained according to KM matching. More specifically: following the maximum matching principle, the structural similarity between the real photographic image and the virtual representative image is used as the matching edge weight between the real photographic image and the virtual representative image, and a one-to-one matching is performed on the real photographic image and the virtual representative image.

[0075] Since the number of nodes on the left and the number of nodes on the right are not necessarily the same, after obtaining a one-to-one matching relationship, additional operations are required for the unmatched nodes. Specifically: if the number of nodes on the left is greater than or equal to the number of nodes on the right, that is, the number of real photographic images is greater than or equal to the number of categories, no additional operations are required; if the number of nodes on the left is less than the number of nodes on the right, that is, the number of real photographic images is less than the number of categories, then for the nodes on the right that have no matches, calculate the structural similarity between each right node and all the nodes on the left, and use the node corresponding to the maximum structural similarity as the corresponding node of the node, thereby obtaining the matching relationship between each category and each real photographic image.

[0076] For the virtual representative image, the structural similarity between the virtual representative image and each other virtual photographic image in the category to which the virtual representative image belongs is calculated, and the obtained structural similarities are arranged in order of magnitude to obtain an inferred similarity sequence. More specifically: for the virtual representative image of the representative category of each category, the structural similarity between the virtual representative images of the representative categories of other categories and the virtual representative image of the category is calculated, and the structural similarities are arranged in order from large to small according to the structural similarity to obtain an inferred similarity sequence. The inferred similarity sequence is used as the similarity relationship of the corresponding virtual representative image, that is, the inference of the similarity relationship of the corresponding virtual representative image.

[0077] For each real photographic image, the structural similarity between each real photographic image and each other real photographic image is calculated, and the obtained structural similarities are arranged in order of size according to the arrangement order of the inferred similarity sequence to obtain a real similarity sequence. It should be noted that when the inferred similarity sequence is arranged in order from large to small according to the structural similarity, the corresponding real similarity sequence is also arranged in order from large to small. The real similarity sequence reflects the similarity between the real photographic images.

[0078] Compare the inferred similarity sequence with the real similarity sequence to obtain the matching difference degree, specifically: compare the order numbers of the matching real photographic image and the virtual representative image in the real similarity sequence and the inferred similarity sequence corresponding to the matching real photographic image and the virtual representative image to obtain the matching degree; and use the negative correlation mapping value of the matching degree as the matching difference degree between the matching real photographic image and the virtual representative image.

[0079] Each real photographic image has its own corresponding virtual representative image. Each element in the real similarity sequence corresponds to a real photographic image, and each element in the inferred similarity sequence corresponds to a virtual representative image. Since each real photographic image has a matching virtual representative image, there are multiple pairs of matching real photographic images and virtual representative images in the real similarity sequence and the inferred similarity sequence.

[0080] Compare the order numbers of the matched real photographic image and virtual representative image in the real similarity sequence and the inferred similarity sequence corresponding to the matched real photographic image and virtual representative image to obtain the matching degree; use the negative correlation mapping value of the matching degree as the matching difference between the matched real photographic image and the virtual representative image. Specifically: use any matched real photographic image and virtual representative image as the target real photographic image and the target virtual representative image; average the absolute values ​​of the difference between the order numbers of all matched real photographic images and virtual representative images in the real similarity sequence corresponding to the target real photographic image and the inferred similarity sequence corresponding to the target virtual representative image, and use the obtained average value as the matching degree; negative correlation mapping is performed on the matching degree, and the negative correlation mapping value of the matching degree is used as the matching difference between the matched target real photographic image and the target virtual representative image.

[0081] In some embodiments, the difference between the preset constant threshold and the normalized matching degree is used as the matching difference. In the embodiment of the present invention, the preset constant threshold is 1, and in other embodiments, the value can be set by the implementer according to the actual situation. That is, in the embodiment of the present invention, the negative correlation mapping of the matching degree is achieved by subtracting the normalized matching degree from the constant 1.

[0082] The matching difference degree can indicate the consistency of the similarity relationship between real photographic images and the similarity relationship between virtual representative images. The smaller the matching degree, the greater the matching difference degree, that is, the greater the difference between the virtual representative image and the corresponding real photographic image. The recognition ability of the trained classifier can be enhanced by increasing the samples of such real photographic images.

[0083] Step S300: Taking the proportion of the matching difference of each real photographic image as an adjustment value, the quantity difference between the real photographic image and the virtual photographic image is adjusted to obtain a sample increment of each real photographic image.

[0084] First, the difference between the real photographic image and the virtual photographic image is used as the initial sample increment, which is the basic value of the number of samples that need to be added.

[0085] Then, for each real photographic image, the proportion of the matching difference of the real photographic image is calculated as a weighted value, and the initial sample increment is weighted to obtain the final sample increment of each real photographic image; wherein, the proportion of the matching difference of the real photographic image is the ratio of the matching difference of the current real photographic image to the sum of the matching differences of all real photographic images.

[0086] Step S400, adding samples of the real photographic image, the added samples are samples obtained by fusing the real photographic image and the virtual photographic image, and the number of added samples is the sample increment.

[0087] Take any real photographic image as the target real photographic image; take the real photographic images with the top-k structural similarity to the target real photographic image as the basic image; more specifically: for each real photographic image, take any real photographic image as the target real photographic image as an example, calculate the structural similarity between the target real photographic image and other real photographic images, and take the real photographic images corresponding to the structural similarity of the first sample increment as the basic image in descending order; calculate the sum of the matching difference between each real photographic image and the corresponding basic image as the overall difference parameter of each real photographic image. Among them, the value of k is the sample increment; for example, when the value of k is 5, the top 5 real photographic images with the largest structural similarity to the target real photographic image are taken as the basic image.

[0088] For any basic image, the matching difference between the basic image and the corresponding real photographic image is used as the numerator, the overall difference parameter of the real photographic image is used as the denominator, and the ratio formed by the numerator and the denominator is used as the weighted weight of the real photographic image; the difference between the preset constant threshold and the weighted weight of the real photographic image is used as the weighted weight of the basic image. In the embodiment of the present invention, the value of the preset constant threshold is 1, and in other embodiments, the implementer adjusts the value according to the actual situation.

[0089] The base image is weighted by the weighted weight of the base image, and the real photographic image is weighted by the weighted weight of the real photographic image. The weighted base image and the real photographic image are fused to obtain a sample of the real photographic image. There are as many samples of the real photographic image as there are base images.

[0090] Step S500, training a sample classifier by adding a real photographic image set and a virtual photographic image set after the sample is added, and judging whether the state of the LED virtual photography is real according to the classification error rate of the sample classifier.

[0091] In order to improve the performance and generalization ability of the sample classifier, the classifier is trained by combining the real sample data that has been incrementally updated with the sample data taken in the LED virtual environment. This strategy can not only take advantage of the diversity and complexity of real samples, but also take advantage of the controllability and scalability of virtual samples to achieve more accurate classification results in different application scenarios. In this way, the classifier can learn a more comprehensive feature representation and can better adapt to new data, reduce the risk of overfitting, and improve the stability and reliability of the model in practical applications.

[0092] The real photographic images and samples constitute a real photographic image set after adding samples, and the virtual photographic images constitute a virtual photographic image set; the photographic images of preset proportions in the real photographic image set after adding samples and the virtual photographic image set are respectively selected as training data to train the SVM classifier to obtain a trained sample classifier. In the embodiment of the present invention, the preset proportion is 70%, and in other embodiments, the implementer can adjust the ratio according to actual conditions.

[0093] 70% of the data in the real photographic image set and the virtual photographic image set after the samples are added are used as training data to train a sample classifier, and the sample classifier in the embodiment of the present invention adopts an SVM classifier. In the embodiment of the present invention, 70% of the images can be randomly selected from the real photographic image set and the virtual photographic image set after the samples are added as training data.

[0094] By enhancing the classification performance of the sample classifier, if a sample classifier with better performance cannot distinguish between real samples and virtual samples well, it means that the virtual effect of the virtual sample is better.

[0095] The sample classifier is trained through training data, and the photographic images other than the training data in the real photographic image set and the virtual photographic image set after adding samples are used as verification data, and the verification data is used as the input of the trained sample classifier, and the proportion of the number of verification data with classification errors is calculated as the classification error rate; when the classification error rate is greater than the preset error threshold, it is judged that the state of the LED virtual shooting is true; when the classification error rate is less than or equal to the preset error threshold, it is judged that the state of the LED virtual shooting is not true. In the embodiment of the present invention, the value of the preset error threshold is 0.9, and in other embodiments, the implementer can adjust the value according to the actual situation.

[0096] The remaining 30% of the data in the real photographic image set and the virtual photographic image set after adding samples is used as verification data, and the verification data is used as the input of the sample classifier. The ratio of the number of verification data with classification errors to the total number of verification data is calculated as the classification error rate. The larger the classification error rate, the more difficult it is for the sample classifier to distinguish between real samples and virtual samples, and the better the virtual effect of the virtual sample. When the classification error rate is greater than the preset error threshold, it means that the current virtual effect is already good and there is no need to adjust the shooting parameters. When the classification error rate is less than or equal to the preset error threshold, it means that the authenticity of the simulated virtual photographic image is still poor. At this time, it can be adjusted by adjusting the LED brightness, optimizing the color temperature, using dynamic light sources, etc. to obtain a better shooting effect of the virtual shooting image.

[0097] See also Figure 2, which shows a system module diagram of an LED virtual shooting synthesis system of a universal axis scene making system provided by an embodiment of the present invention, the system includes the following modules:

[0098] An image acquisition module 10 is used to acquire a real photographic image and a virtual photographic image simulated by an LED screen;

[0099] The difference analysis module 20 is used to select a representative virtual representative image from the virtual photographic image; match the real photographic image and the virtual representative image, and analyze the matching difference between the matched real photographic image and the virtual representative image;

[0100] The increment determination module 30 is used to adjust the quantity difference between the real photographic image and the virtual photographic image by taking the proportion of the matching difference of each real photographic image as an adjustment value, so as to obtain the sample increment of each real photographic image;

[0101] A sample adding module 40 is used to add samples of the real photographic image, where the added samples are samples obtained by fusing the real photographic image and the virtual photographic image, and the number of added samples is a sample increment;

[0102] The classifier training module 50 is used to train the sample classifier by adding the real photographic image set and the virtual photographic image set after the sample is added, and judge whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier.

[0103] Optionally, the transmission medium can be a wired link, such as but not limited to coaxial cable, optical fiber and digital subscriber line, or a wireless link, such as but not limited to Wireless Fidelity (WIFI), Bluetooth and mobile device network.

[0104] It should be noted that the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above.

[0105] Figure 3 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. Figure 3 As shown, the computer device 600 includes: a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and running on the processor 620, wherein when the processor 620 executes the computer program 630, the computer device can execute the LED virtual shooting synthesis method of any universal axis framing system introduced above.

[0106] In addition, an embodiment of the present invention also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute the LED virtual shooting synthesis method of the universal axis framing system provided in the embodiment of the present invention.

[0107] The embodiment of the present invention can divide the functional modules of the device according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0108] In the case of dividing each module according to each function, the device may also include a signal uploading module, a determining module, an adjusting module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0109] It should be understood that the device provided in the embodiment of the present invention is used to execute the LED virtual shooting synthesis method of the universal axis framing system, and thus can achieve the same effect as the above-mentioned implementation method.

[0110] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc. The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0111] In addition, the device provided in the embodiment of the present invention may specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the LED virtual shooting synthesis method of the universal axis scene making system provided in the above embodiment.

[0112] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement the LED virtual shooting synthesis method of the universal axis scene making system provided in the above embodiment.

[0113] The embodiment of the present invention further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the LED virtual shooting synthesis method of the universal axis framing system provided in the above embodiment.

[0114] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present invention are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Through the description of the above implementation methods, technicians in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways.

[0115] The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. There may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0116] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0117] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0118] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0119] The above contents are only specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A method for synthesizing LED virtual shooting of a gimbal-axis scene making system, characterized in that: The method comprises the following steps: Acquire real photographic images and virtual photographic images simulated by LED screens; Selecting a representative virtual representative image from the virtual photographic image; matching the real photographic image and the virtual representative image, and analyzing the matching difference between the matched real photographic image and the virtual representative image; Taking the proportion of the matching difference of each real photographic image as an adjustment value, adjusting the quantity difference between the real photographic image and the virtual photographic image to obtain the sample increment of each real photographic image; Adding samples of the real photographic image, where the added samples are samples obtained by fusing the real photographic image and the virtual photographic image, and the number of the added samples is the sample increment; By adding the real photographic image set and the virtual photographic image set after the sample, the sample classifier is trained, and according to the classification error rate of the sample classifier, it is judged whether the state of the LED virtual shooting is real.

2. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The step of selecting a representative virtual representative image from the virtual photographic image comprises: Determining the distance between the virtual photographic images according to the structural differences between the virtual photographic images; Based on the distance between virtual photographic images, the virtual photographic images are hierarchically clustered to obtain a hierarchical clustering tree; Obtain the number of categories in each layer of the hierarchical clustering tree, compare the number of categories with the number of real photographic images, and take the layer with the smallest number difference as the best layer of the hierarchical clustering tree; mark all categories in the best layer as representative categories; For any representative category, the sum of the structural similarities between each virtual photographic image and other virtual photographic images in the representative category to which it belongs is taken, and the virtual photographic image corresponding to the maximum value of the sum of the structural similarities is taken as the virtual representative image in the representative category; wherein the structural similarity represents the degree of structural similarity between photographic images.

3. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The analyzing and matching matching difference between the real photographic image and the virtual representative image includes: For the virtual representative image, the structural similarity between the virtual representative image and each other virtual photographic image in the category to which the virtual representative image belongs is calculated, and the obtained structural similarities are arranged in order of magnitude to obtain an inferred similarity sequence; For real photographic images, the structural similarity between each real photographic image and each other real photographic image is calculated, and the obtained structural similarities are arranged in order of size to obtain a real similarity sequence; Compare the inferred similarity sequence and the true similarity sequence to obtain the matching difference.

4. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 3, characterized in that: The comparing the inferred similarity sequence and the real similarity sequence to obtain the matching difference degree includes: Compare the order numbers of the matched real photographic image and the virtual representative image in the real similarity sequence and the inferred similarity sequence corresponding to the matched real photographic image and the virtual representative image to obtain the matching degree; and use the negative correlation mapping value of the matching degree as the matching difference between the matched real photographic image and the virtual representative image.

5. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The method for obtaining the sample increment is: The quantitative difference between the real photographic image and the virtual photographic image is used as the initial sample increment; For each real photographic image, the proportion of the matching difference of the real photographic image is calculated as a weighted value, and the initial sample increment is weighted to obtain the final sample increment of each real photographic image; wherein, the proportion of the matching difference of the real photographic image is the ratio of the matching difference of the current real photographic image to the sum of the matching differences of all real photographic images.

6. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The method for obtaining samples of real photographic images is: For each real photographic image, the structural similarity between the real photographic image and other real photographic images is calculated, and the real photographic image corresponding to the structural similarity of the previous sample increment is used as the base image in descending order; Calculate the sum of the matching differences between each real photographic image and the corresponding basic image as the overall difference parameter of each real photographic image; For any basic image, the matching difference between the basic image and the corresponding real photographic image is used as the numerator, the overall difference parameter of the real photographic image is used as the denominator, and the ratio formed by the numerator and the denominator is used as the weighted weight of the real photographic image; the difference between the preset constant threshold and the weighted weight of the real photographic image is used as the weighted weight of the basic image; The base image is weighted by the weighted weight of the base image, and the real photographic image is weighted by the weighted weight of the real photographic image. The weighted base image and the real photographic image are fused to obtain a sample of the real photographic image.

7. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The training of the sample classifier by adding the real photographic image set and the virtual photographic image set after the sample is added includes: A real photographic image set after adding samples is composed of real photographic images and samples, and a virtual photographic image set is composed of virtual photographic images; photographic images of preset proportions in the real photographic image set after adding samples and the virtual photographic image set are respectively selected as training data to train the SVM classifier to obtain a trained sample classifier.

8. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The step of judging whether the state of the LED virtual shooting is real according to the classification error rate of the sample classifier includes: The photographic images other than the training data in the real photographic image set and the virtual photographic image set after adding samples are used as verification data; the verification data is used as the input of the trained sample classifier, and the proportion of the number of verification data with classification errors is calculated as the classification error rate; when the classification error rate is greater than the preset error threshold, it is determined that the state of the LED virtual photography is real; when the classification error rate is less than or equal to the preset error threshold, it is determined that the state of the LED virtual photography is not real.

9. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 1, characterized in that: The matching of the real photographic image and the virtual representative image comprises: Following the maximum matching principle, the structural similarity between the real photographic image and the virtual representative image is used as the matching edge weight between the real photographic image and the virtual representative image, and a one-to-one matching is performed between the real photographic image and the virtual representative image.

10. The LED virtual shooting synthesis method of the universal axis scene making system according to claim 2, characterized in that: The step of determining the distance between the virtual photographic images according to the structural differences between the virtual photographic images comprises: The negative correlation mapping value of the SSIM structural similarity between the virtual photographic images is used as the distance between the virtual photographic images.

Images