Virtual camera validity detection method and device, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for detecting the validity of a virtual camera. The detection method includes: acquiring a first image captured by a real camera and a second image captured by a virtual camera; performing recognition on the first image using multiple preset models to obtain a first recognition result, and performing recognition on the second image to obtain a second recognition result; determining the virtual camera's validity when the similarity value between the first and second recognition results reaches a preset threshold. By using the above method, the validity of the current virtual camera is determined based on the similarity between the recognition results of the image captured by the real camera and the recognition results of the image captured by the virtual camera, thus detecting the validity of the virtual camera's physical model from an algorithm testing perspective.

Description

Technical Field

[0001] This application relates to the field of virtual camera validity detection technology, specifically to a virtual camera validity detection method, device, electronic device, and storage medium. Background Technology

[0002] Researching intelligent driving algorithms requires building a practical virtual camera physical model, generating simulation data for algorithm training and testing, and conducting extensive simulation tests using this data. In this process, the effectiveness of the virtual camera physical model directly impacts the reliability of the intelligent driving algorithm.

[0003] Since the virtual camera physical model ultimately generates simulation data for algorithm training and testing, there is an urgent need for a method to detect the effectiveness of the virtual camera physical model from the perspective of algorithm testing. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for detecting the validity of a virtual camera, enabling the detection of the validity of a virtual camera's physical model from an algorithm testing perspective.

[0005] Firstly, this application provides a method for detecting the validity of a virtual camera, the method comprising:

[0006] Acquire a first image captured by the real camera and a second image captured by the virtual camera, wherein the virtual camera is a simulated camera corresponding to the real camera;

[0007] The first image is identified by multiple preset models to obtain a first identification result, and the second image is identified by multiple preset models to obtain a second identification result.

[0008] When the similarity value between the first recognition result and the second recognition result reaches a preset threshold, the virtual camera is determined to be valid.

[0009] By using the above method, the similarity between the recognition results of images captured by real cameras and the recognition results of images captured by virtual cameras is used to determine whether the current virtual camera is effective, thus realizing the effectiveness of the virtual camera physical model from the perspective of algorithm testing.

[0010] In one possible embodiment, before acquiring the first image captured by the real camera and the second image captured by the virtual camera, the method further includes:

[0011] Construct a virtual scene corresponding to the real-world scene, wherein the virtual scene is a digital twin of the real-world scene;

[0012] Determine the virtual camera in the virtual scene corresponding to the real camera at a preset location in the real scene;

[0013] The parameters of the virtual camera are calibrated to be consistent with those of the real camera.

[0014] Using the methods described above, a virtual camera can be constructed for algorithm testing.

[0015] In one possible embodiment, determining that the virtual camera is valid when the similarity value between the first recognition result and the second recognition result reaches a preset threshold includes:

[0016] Calculate the similarity value between the first recognition result and the second recognition result;

[0017] Determine whether the similarity value is greater than or equal to a preset threshold;

[0018] If so, then the virtual camera is confirmed to be valid;

[0019] If not, the parameters of the virtual camera are adjusted based on the structural similarity between the first image and the second image until the similarity value is greater than or equal to the preset threshold.

[0020] Using the methods described above, the images captured by the virtual camera can be made to be infinitely close to those captured by a real camera.

[0021] In one possible embodiment, adjusting the parameters of the virtual camera based on the structural similarity between the first image and the second image includes:

[0022] Based on the brightness, contrast, and image composition of the first and second images, the structural similarity between the first and second images is calculated using the Structural Similarity Simulation (SSIM) algorithm.

[0023] Based on the structural similarity, the parameters of the virtual camera are adjusted until the similarity value is greater than or equal to a preset threshold.

[0024] By using the methods described above, the virtual camera can be adjusted so that the images captured by the virtual camera can be as close as possible to the images captured by a real camera.

[0025] Secondly, this application provides a virtual camera validity detection device, the device comprising:

[0026] The acquisition module is used to acquire a first image captured by the real camera and a second image captured by the virtual camera, wherein the virtual camera is a simulated camera corresponding to the real camera;

[0027] The recognition module is used to recognize the first image using multiple preset models to obtain a first recognition result, and to recognize the second image using multiple preset models to obtain a second recognition result.

[0028] The first determining module is used to determine that the virtual camera is valid when the similarity value between the first recognition result and the second recognition result reaches a preset threshold.

[0029] In one possible embodiment, the device further includes:

[0030] A construction module is used to construct a virtual scene corresponding to a real-world scene, wherein the virtual scene is a digital twin of the real-world scene;

[0031] The second determining module is used to determine the virtual camera in the virtual scene corresponding to the real camera at the preset position in the real scene.

[0032] The calibration module is used to calibrate the parameters of the virtual camera to be consistent with those of the real camera.

[0033] In one possible embodiment, the first determining module includes:

[0034] A calculation unit is used to calculate the similarity value between the first recognition result and the second recognition result;

[0035] A judgment unit is used to determine whether the similarity value is greater than or equal to a preset threshold.

[0036] A determining unit is configured to determine that the virtual camera is valid if the similarity value is greater than or equal to a preset threshold.

[0037] An adjustment unit is configured to adjust the parameters of the virtual camera based on the structural similarity between the first image and the second image if the similarity value is less than the preset threshold, until the similarity value is greater than or equal to the preset threshold.

[0038] In one possible embodiment, the adjustment unit is specifically used for:

[0039] Based on the brightness, contrast, and image composition of the first and second images, the structural similarity between the first and second images is calculated using the Structural Similarity Simulation (SSIM) algorithm.

[0040] Based on the structural similarity, the parameters of the virtual camera are adjusted until the similarity value is greater than or equal to a preset threshold.

[0041] Thirdly, this application provides an electronic device, comprising:

[0042] Memory, used to store program instructions;

[0043] The processor is configured to invoke program instructions stored in the memory and execute the virtual camera validity detection method described in any one of the first aspects according to the obtained program instructions.

[0044] Fourthly, this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the virtual camera validity detection method described in any one of the first aspects.

[0045] The technical effects of each of the second to fourth aspects mentioned above, as well as the technical effects that each aspect may achieve, are described above with reference to the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect, and will not be repeated here. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0047] Figure 2 A flowchart illustrating a virtual camera validity detection method provided in this application embodiment;

[0048] Figure 3 This is a schematic diagram of the structure of a virtual camera validity detection device provided in an embodiment of this application;

[0049] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0051] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.

[0052] Before introducing the virtual camera validity detection method provided in the embodiments of this application, the technical background of the embodiments of this application will be described in detail below for ease of understanding.

[0053] In the research of intelligent driving algorithms, the effectiveness of the virtual camera physical model directly affects the reliability of the algorithm. Since the virtual camera physical model ultimately generates simulation data for algorithm training and testing, there is an urgent need for a method to detect the effectiveness of the virtual camera physical model from the perspective of algorithm testing.

[0054] To address the aforementioned issues, embodiments of this application provide a method, apparatus, electronic device, and storage medium for detecting the validity of a virtual camera. Based on the similarity between the recognition results of images captured by a real camera and the recognition results of images captured by a virtual camera, the validity of the current virtual camera is determined, thereby detecting the validity of the virtual camera's physical model from an algorithm testing perspective.

[0055] Based on the above technical effects, the preferred embodiments of this application will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for illustration and explanation of this application and are not intended to limit this application. Furthermore, the embodiments of this application and the features in the embodiments can be combined with each other without conflict.

[0056] like Figure 1 The diagram illustrates a possible application scenario provided by an embodiment of this application. This application scenario includes a target terminal (101a, 101b) and a server 102. The target terminal (101a, 101b) and the server 102 can interact via a communication network. The communication network can employ wireless communication or wired communication methods.

[0057] For example, the target terminal (101a, 101b) can access the network and communicate with the server 102 through cellular mobile communication technology, including 5th Generation Mobile Networks (5G) technology.

[0058] For example, the target terminal (101a, 101b) can access the network and communicate with the server 102 through short-range wireless communication, which includes Wireless Fidelity (Wi-Fi) technology.

[0059] This application embodiment does not impose any limitation on the number of the above-mentioned devices, such as Figure 1 As shown, only the target terminal (101a, 101b) and server 102 are described as examples. The following is a brief introduction to each of the above devices and their respective functions.

[0060] The target terminal (101a, 101b) is a device that can provide voice and / or data connectivity to users, including: handheld terminal devices with wireless connectivity, vehicle-mounted terminal devices, etc.

[0061] For example, the target terminals (101a, 101b) include, but are not limited to: mobile phones, tablets, laptops, handheld computers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in autonomous driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0062] Furthermore, the target terminals (101a, 101b) may have a client related to virtual camera validity detection installed. This client can be software (e.g., an app, browser, short video app, etc.), or a webpage, mini-program, etc. In this embodiment, the target terminals (101a, 101b) can use the aforementioned client related to virtual camera validity detection and can interact with the server 102 to exchange information related to the virtual camera validity detection scenario.

[0063] Furthermore, server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0064] Furthermore, in this embodiment of the application, the server 102 may be equipped with a virtual camera validity detection platform corresponding to the client, which is used to perform virtual camera validity detection.

[0065] Based on the above application scenarios, the virtual camera validity detection method provided in the embodiments of this application will be described and explained below with reference to the accompanying drawings. Figure 2 As shown in the figure, this application provides a method for detecting the validity of a virtual camera, which specifically includes the following steps:

[0066] S201, acquire the first image captured by the real camera and the second image captured by the virtual camera;

[0067] S202, the first image is identified using multiple preset models to obtain a first identification result, and the second image is identified to obtain a second identification result;

[0068] S203, when the similarity value between the first recognition result and the second recognition result reaches a preset threshold, the virtual camera is determined to be valid.

[0069] In this embodiment, real-world scenarios frequently used by cameras, such as traffic intersections and gate barriers, are first selected, and a corresponding virtual scenario is constructed. This virtual scenario is a digital twin of the real-world scenario. Furthermore, the virtual camera corresponding to a real camera at a preset location in the real-world scenario is confirmed within the virtual scenario, and its parameters are then calibrated to be consistent with those of the preset real camera. In this way, a virtual camera for algorithm testing can be constructed, where the virtual camera and the real camera are the same camera within the digital twin scenario.

[0070] Since the modeling accuracy of virtual cameras is affected by many factors, such as algorithm accuracy and parameter settings, the effectiveness of the aforementioned virtual cameras needs to be further verified.

[0071] Specifically:

[0072] A first image captured by a real camera in a real scene is acquired, and a second image captured by a virtual camera in a virtual scene is acquired. Specifically, when the first image is captured by the real camera, a virtual camera can be run to simulate the real camera, thereby acquiring the second image, which is the simulated image corresponding to the first image. In this embodiment, the first and second images contain the same number of frames, both being 10,000. Of course, the specific number of frames acquired can be adjusted according to the actual situation, and is not specifically limited here.

[0073] Furthermore, the first image is identified by multiple preset models to obtain a first recognition result, and the second image is identified by multiple preset models to obtain a second recognition result. The preset models can be YOLO Faster-CNN SwinOthers models or other object detection models, which are not specifically limited here. The first recognition result and the second recognition result include at least recognition success and recognition failure.

[0074] Under the condition of meeting the accuracy requirements of virtual camera modeling, if the preset model can recognize image A captured by the real camera, then it can also recognize image B captured by the virtual camera, where image B is the simulated image corresponding to image A under the same conditions. Similarly, if the preset model cannot recognize image A, then it cannot recognize image B either. To ensure the consistency between virtual camera imaging and real camera imaging, to guarantee the accuracy of simulation results, and thus to verify the effectiveness of the virtual camera, it is necessary to calculate the similarity value between the first recognition result and the second recognition result, and ensure that the similarity value reaches a preset threshold. The specific calculation process of the similarity value includes:

[0075] Based on the recognition model information and recognition image information in the first and second recognition results, the recognition results corresponding to the same model and images that are twins are compared, wherein the recognition image information includes the first image or the second image.

[0076] For example, the first image includes image A and image B, the second image includes image a and image b, and the preset models include model 1 and model 2. The results of model 1 and model 2 applying different methods to the first and second images are shown in Table 1.

[0077]

[0078] Table 1

[0079] In Table 1, Model 1's recognition results for the first image are: Image A was successfully recognized; Image B was successfully recognized. Model 2's recognition results for the first image are: Image A was successfully recognized; Image B was not recognized. Model 1's recognition results for the second image are: Image a was successfully recognized; Image b was successfully recognized. Model 2's recognition results for the second image are: Image a was successfully recognized; Image b was not recognized. The recognition result corresponding to the first image is the first recognition result, and the recognition result corresponding to the second image is the second recognition result.

[0080] When comparing the first and second recognition results, the recognition results of the first and second images corresponding to the same model and being twins are compared according to (image information, model information, result). For example, the recognition result of image A corresponding to model 1 is compared with the recognition result of image a corresponding to model 1. According to Table 1, the similarity value between the first and second recognition results is 100%.

[0081] After calculating the similarity value between the first recognition result and the second recognition result, it is further determined whether the similarity value is greater than or equal to a preset threshold.

[0082] If so, it indicates that the consistency between the twin image and the first image meets the requirements, meaning that the virtual camera's imaging realism meets the standard, thus confirming the virtual camera's validity. This method allows for the detection of the validity of a virtual camera.

[0083] If not, the parameters of the virtual camera are adjusted based on the structural similarity between the first and second images. These parameters include at least one or more of the following: intrinsic parameters, extrinsic parameters, motion blur, and white balance, until the similarity value between the first and second recognition results is greater than or equal to a preset threshold. The specific adjustment process includes:

[0084] First, the structural similarity between the first image and the second image is calculated. Structural similarity is an index that measures the similarity between two images, calculated based on the brightness, contrast, and image composition of the first and second images. Here, image composition refers to image content; for example, if the image content of a frame consists of mountains, rivers, and grassland, then the image composition is mountains, rivers, and grassland. In this embodiment, the algorithm for calculating structural similarity is the Structural Similarity (SSIM) algorithm, and the specific calculation formula is as follows:

[0085]

[0086] Here, SSIM represents structural similarity, specifically ranging from [-1, 1]. A value of 1 indicates that the first and second images are completely identical. μ xμ represents the mean of the first image. y δ represents the mean of the second image, which reflects the image brightness; x δ represents the standard deviation of the first image. y The variance of the second image is represented by δ, and the standard deviation is used to reflect image contrast. xy c1 represents the covariance of the first and second images, which is used as a measure of structural similarity; c1 and c2 are constants that maintain the stability of the algorithm, where c1 = (k1L) 2 c1 = (k1L) 2 L is the dynamic range of pixel values, k1 = 0.01, k1 = 0.03.

[0087] Then, based on the calculated structural similarity between the first image and the second image, the parameters of the virtual camera are adjusted until the structural similarity meets the requirements. At this point, the similarity value between the first image and the second image will be greater than or equal to the preset threshold.

[0088] The above methods can make the imaging of virtual cameras infinitely close to that of real cameras, thereby helping to improve the reliability of intelligent driving research.

[0089] Based on the same inventive concept, this application provides a virtual camera validity detection device. Please refer to... Figure 3 The device includes:

[0090] The acquisition module 301 is used to acquire a first image captured by the real camera and a second image captured by the virtual camera, wherein the virtual camera is a simulation camera corresponding to the real camera;

[0091] The recognition module 302 is used to recognize the first image using multiple preset models to obtain a first recognition result, and to recognize the second image to obtain a second recognition result.

[0092] The first determining module 303 is used to determine that the virtual camera is valid when the similarity value between the first recognition result and the second recognition result reaches a preset threshold.

[0093] In one possible embodiment, the device further includes:

[0094] A construction module is used to construct a virtual scene corresponding to a real-world scene, wherein the virtual scene is a digital twin of the real-world scene;

[0095] The second determining module is used to determine the virtual camera in the virtual scene corresponding to the real camera at the preset position in the real scene.

[0096] The calibration module is used to calibrate the parameters of the virtual camera to be consistent with those of the real camera.

[0097] In one possible embodiment, the first determining module 303 includes:

[0098] A calculation unit is used to calculate the similarity value between the first recognition result and the second recognition result;

[0099] A judgment unit is used to determine whether the similarity value is greater than or equal to a preset threshold.

[0100] A determining unit is configured to determine that the virtual camera is valid if the similarity value is greater than or equal to a preset threshold.

[0101] An adjustment unit is configured to adjust the parameters of the virtual camera based on the structural similarity between the first image and the second image if the similarity value is less than the preset threshold, until the similarity value is greater than or equal to the preset threshold.

[0102] In one possible embodiment, the adjustment unit is specifically used for:

[0103] Based on the brightness, contrast, and image composition of the first and second images, the structural similarity between the first and second images is calculated using the Structural Similarity Simulation (SSIM) algorithm.

[0104] Based on the structural similarity, the parameters of the virtual camera are adjusted until the similarity value is greater than or equal to a preset threshold.

[0105] The aforementioned device determines the effectiveness of a virtual camera by comparing the recognition results of images captured by a real camera with those captured by a virtual camera. This allows for the evaluation of the effectiveness of the virtual camera's physical model from an algorithm testing perspective. Furthermore, the similarity value can be used to further adjust the virtual camera's parameters, enabling the virtual camera's imaging to closely approximate that of a real camera, thus improving the reliability of intelligent driving research.

[0106] Based on the same inventive concept, this application also provides an electronic device that can realize the function of the aforementioned virtual camera validity detection method device. (Refer to...) Figure 4 The electronic device includes:

[0107] At least one processor 401 and a memory 402 connected to at least one processor 401. In this embodiment, the specific connection medium between the processor 401 and the memory 402 is not limited. Figure 4 The example shown is the connection between processor 401 and memory 402 via bus 400. Bus 400 is... Figure 4The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 400 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 4 The term is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, processor 401 can also be called a controller; there is no restriction on the name.

[0108] In this embodiment, the memory 402 stores instructions executable by at least one processor 401. By executing the instructions stored in the memory 402, the at least one processor 401 can perform the virtual camera validity detection method described above. The processor 401 can implement... Figure 3 The functions of each module in the device shown.

[0109] The processor 401 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 402 and calling data stored in memory 402, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0110] In one possible embodiment, processor 401 may include one or more processing units. Processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 401. In some embodiments, processor 401 and memory 402 may be implemented on the same chip; in some embodiments, they may be implemented separately on independent chips.

[0111] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the virtual camera validity detection method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0112] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 402 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0113] By designing and programming the processor 401, the code corresponding to the virtual camera validity detection method described in the foregoing embodiments can be embedded into the chip, thereby enabling the chip to execute the code during runtime. Figure 2 The steps of the virtual camera validity detection method in the illustrated embodiment are as follows. How to design and program the processor 401 is a technique well-known to those skilled in the art and will not be described further here.

[0114] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium. The computer program product includes computer program code, which, when executed on a computer, causes the computer to perform any of the virtual camera validity detection methods discussed above. Since the principle by which the above-described computer-readable storage medium solves the problem is similar to that of the virtual camera validity detection method, the implementation of the above-described computer-readable storage medium can be found in the implementation of the method; repeated details will not be elaborated further.

[0115] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable virtual camera validity detection device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable virtual camera validity detection device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable virtual camera validity detection device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the instruction means being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable virtual camera validity detection device, causing a series of user operation steps to be executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting the validity of a virtual camera, characterized in that, The method includes: Acquire a first image captured by a real camera and a second image captured by a virtual camera, wherein the virtual camera is a simulated camera corresponding to the real camera; The first image is identified by multiple preset models to obtain a first identification result, and the second image is identified by multiple preset models to obtain a second identification result; the first identification result and the second identification result include at least identification success and identification failure; Based on the preset model information and recognition image information in the first recognition result and the second recognition result, the recognition results of the first image corresponding to the same preset model and the twin images in the second image are compared to determine whether the recognition results of the first image corresponding to the same preset model and the twin images in the second image are the same, and the similarity value between the first recognition result and the second recognition result is obtained. When the similarity value reaches a preset threshold, the virtual camera is determined to be valid.

2. The method as described in claim 1, characterized in that, Before acquiring the first image captured by the real camera and the second image captured by the virtual camera, the method further includes: Construct a virtual scene corresponding to the real-world scene, wherein the virtual scene is a digital twin of the real-world scene; Determine the virtual camera in the virtual scene corresponding to the real camera at a preset location in the real scene; The parameters of the virtual camera are calibrated to be consistent with those of the real camera.

3. The method as described in claim 1 or 2, characterized in that, Determining the validity of the virtual camera when the similarity value reaches a preset threshold includes: Determine whether the similarity value is greater than or equal to a preset threshold; If so, then the virtual camera is confirmed to be valid; If not, the parameters of the virtual camera are adjusted based on the structural similarity between the first image and the second image until the similarity value is greater than or equal to the preset threshold.

4. The method as described in claim 3, characterized in that, The step of adjusting the parameters of the virtual camera based on the structural similarity between the first image and the second image includes: Based on the brightness, contrast, and image composition of the first and second images, the structural similarity between the first and second images is calculated using the Structural Similarity Simulation (SSIM) algorithm. Based on the structural similarity, the parameters of the virtual camera are adjusted until the similarity value is greater than or equal to a preset threshold.

5. A virtual camera validity detection device, characterized in that, The device includes: The acquisition module is used to acquire a first image captured by a real camera and a second image captured by a virtual camera, wherein the virtual camera is a simulated camera corresponding to the real camera; The recognition module is used to recognize the first image using multiple preset models to obtain a first recognition result, and to recognize the second image using multiple preset models to obtain a second recognition result; the first recognition result and the second recognition result include at least recognition success and recognition failure; The first determining module is used to compare the recognition results of the first image corresponding to the same preset model and the recognition image information in the first recognition result and the second recognition result with the recognition image information, and to determine whether the recognition results of the first image corresponding to the same preset model and the recognition image with the same twin in the second image are the same, and to obtain a similarity value between the first recognition result and the second recognition result; when the similarity value reaches a preset threshold, the virtual camera is determined to be valid.

6. The apparatus as claimed in claim 5, characterized in that, The device further includes: A construction module is used to construct a virtual scene corresponding to a real-world scene, wherein the virtual scene is a digital twin of the real-world scene; The second determining module is used to determine the virtual camera in the virtual scene corresponding to the real camera at the preset position in the real scene. The calibration module is used to calibrate the parameters of the virtual camera to be consistent with those of the real camera.

7. The apparatus as described in claim 5 or 6, characterized in that, The first determining module includes: A judgment unit is used to determine whether the similarity value is greater than or equal to a preset threshold. A determining unit is configured to determine that the virtual camera is valid if the similarity value is greater than or equal to a preset threshold. An adjustment unit is configured to adjust the parameters of the virtual camera based on the structural similarity between the first image and the second image if the similarity value is less than the preset threshold, until the similarity value is greater than or equal to the preset threshold.

8. The apparatus as claimed in claim 7, characterized in that, The adjustment unit is specifically used for: Based on the brightness, contrast, and image composition of the first and second images, the structural similarity between the first and second images is calculated using the Structural Similarity Simulation (SSIM) algorithm. Based on the structural similarity, the parameters of the virtual camera are adjusted until the similarity value is greater than or equal to a preset threshold.

9. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-4.

Images