Camera calibration method, mobile application, cloud software, and camera calibration system

Abstract

The present invention is applicable to the technical field of automatic calibration of image collection devices, and provides a camera calibration method, a mobile application, cloud software, and a camera calibration system. The calibration method applied to a mobile terminal comprises: obtaining an initial image on the basis of an imaging device of the mobile terminal; uploading the initial image to a server, and acquiring prediction calibration parameters provided by a server, the prediction calibration parameters being calibration parameters obtained by the server predicting the initial image on the basis of a neural network model; and calibrating the imaging device on the basis of the prediction calibration parameters. According to the present scheme, the utilization of intelligent prediction can effectively reduce the dependence on manual calibration intervention, thereby reducing the risk of human errors and inconsistency in the calibration process; and a prediction model can perform real-time high-speed calibration on an imaging image, thereby effectively improving the calibration precision and the calibration speed. The present invention is more suitable for a high-frame-rate camera device.

Description

Camera calibration method, mobile application, cloud software and camera calibration system Technical Field

[0001] The present invention belongs to the technical field of automatic calibration of image acquisition equipment, and in particular relates to a camera calibration method, mobile application, cloud software and camera calibration system. Background Art

[0002] For overhead cameras used for reading and writing, the image acquisition modules of these devices need to be calibrated before use. Currently, mainstream calibration techniques typically rely on manual calibration methods, computer vision, or AI technologies.

[0003] Manual calibration involves adjusting various camera parameters such as brightness, contrast, hue, saturation, sharpness, gamma, white balance, exposure, focus, field of view, and image color inversion to achieve the desired image quality. However, this manual process is time-consuming, error-prone, requires specialized technicians, and lacks scalability. Computer vision technologies involve complex mathematical algorithms that require precise measurement and parameter adjustment. However, due to the complexity and computational complexity of parameter adjustment methods, these methods are unable to adapt well to changing environmental conditions and perform low-latency real-time calibration adjustments. AI models, which often focus on image enhancement, also struggle to achieve real-time, fast, and accurate image calibration.

[0004] Therefore, it can be seen that relying on a manual calibration process leads to inefficiency, potential errors, and lack of scalability; computer vision technology, while powerful, can be complex to adjust in real time; and existing AI models often struggle to achieve real-time, fast, and accurate calibration of images. Therefore, improvements are needed in camera calibration technology for high-mounted cameras used for reading and writing purposes.

[0005] Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a camera calibration method that aims to solve the problems of inefficiency, potential errors and lack of scalability caused by the existing reliance on manual calibration process; although computer vision technology is powerful, it may be complex in real-time adjustment; and existing AI models often have difficulty in achieving real-time, fast and accurate calibration of images.

[0007] An embodiment of the present application is implemented as follows: a camera calibration method is provided, which is applied to a mobile terminal. The camera calibration method includes: obtaining an initial image based on an imaging device on the mobile terminal; uploading the initial image to a server terminal, and obtaining predicted calibration parameters provided by the server terminal, wherein the predicted calibration parameters are calibration parameters obtained by the server terminal by predicting the initial image based on a neural network model; and calibrating the imaging device based on the predicted calibration parameters.

[0008] Another object of an embodiment of the present application is to provide a camera calibration method, which is applied to a server side, and the camera calibration method includes: obtaining an initial image uploaded by a mobile terminal; encoding the initial image into a latent vector representation, and the latent vector representation is used to characterize the relevant features in the initial image required for calibrating the imaging device; analyzing the latent vector representation based on a neural network model, and the neural network model is used to predict the calibration parameters required for calibrating the imaging device to obtain predicted calibration parameters; and transmitting the predicted calibration parameters to the mobile terminal.

[0009] Another object of an embodiment of the present application is to provide a mobile application, which includes: an initial image generation module, which is used to obtain an initial image based on an imaging device on a mobile terminal; a predicted calibration parameter acquisition module, which is used to upload the initial image to a server terminal and obtain predicted calibration parameters provided by the server terminal, wherein the predicted calibration parameters are calibration parameters obtained by the server terminal by predicting the initial image based on a neural network model; and a calibration module, which is used to calibrate the imaging device based on the predicted calibration parameters.

[0010] Another object of an embodiment of the present application is to provide a cloud-based software, which includes: an initial image acquisition module for acquiring an initial image uploaded by a mobile terminal; a latent vector representation acquisition module for encoding the initial image into a latent vector representation, wherein the latent vector representation is used to characterize relevant features in the initial image required for calibrating an imaging device; a predicted calibration parameter generation module for analyzing the latent vector representation based on a neural network model, wherein the neural network model is used to predict the calibration parameters required for calibrating the imaging device to obtain predicted calibration parameters; and a predicted calibration parameter transmission module for transmitting the predicted calibration parameters to the mobile terminal.

[0011] Another purpose of an embodiment of the present application is to provide a camera calibration system, which includes: a mobile application as described above, used to execute the camera calibration method as described above; and a cloud software as described above, which interacts with the mobile application for data execution, and is used to execute the camera calibration method as described above.

[0012] A camera calibration method provided in an embodiment of the present application can effectively reduce reliance on manual calibration intervention, thereby reducing the risk of human error and inconsistency in the calibration process. The prediction model can perform real-time and high-speed calibration of the imaging image, effectively improving the calibration accuracy and calibration speed, and is more suitable for high frame rate camera equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG1 is a diagram illustrating an application environment of a camera calibration method provided by an embodiment of the present application;

[0014] FIG2 is a flow chart of a camera calibration method provided in an embodiment of the present application;

[0015] FIG3 is a flow chart of another camera calibration method provided in an embodiment of the present application;

[0016] FIG4 is a schematic diagram of a data interaction step provided in an embodiment of the present application;

[0017] FIG5 is a schematic diagram of another data interaction step provided in an embodiment of the present application;

[0018] FIG6 is a block diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] It is understood that the terms "first," "second," and the like may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are used solely to distinguish a first element from another element. For example, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script, without departing from the scope of this application.

[0021] FIG1 is a diagram of an application environment of a camera calibration method provided in an embodiment of the present application. As shown in FIG1 , the application environment includes a mobile terminal 110 and a server terminal 120 .

[0022] The server end 120 can be an independent physical server or terminal, or a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN.

[0023] The mobile terminal 110 may be, but is not limited to, 3D glasses, AR devices, smartphones, tablet computers, laptop computers, desktop computers, smart watches, etc. The mobile terminal 110 and the server terminal 120 may be connected via a network, and this application does not impose any restrictions thereon.

[0024] As shown in FIG2 , in one embodiment, a camera calibration method is proposed. This embodiment mainly uses the method applied to the mobile terminal 110 in FIG1 as an example. A camera calibration method is applied to the mobile terminal, and the camera calibration method includes:

[0025] Step S202: obtaining an initial image based on the imaging device of the mobile terminal.

[0026] Step S204: uploading the initial image to the server, and obtaining prediction calibration parameters provided by the server, wherein the prediction calibration parameters are calibration parameters obtained by the server by predicting the initial image based on the neural network model.

[0027] Step S206: calibrate the imaging device based on the predicted calibration parameters.

[0028] In one embodiment, the method further includes: obtaining an operating mode of the imaging device selected by the user, the operating mode including a reading mode or a writing mode; and determining an imaging resolution and a shooting frame rate of the imaging device based on the operating mode.

[0029] In one embodiment, the present solution can be used to automatically calibrate an overhead camera for reading and writing purposes, using an artificial intelligence model to implement the calibration process.

[0030] In one embodiment, as shown in Figures 4 and 5, each provides a flow chart of some steps in the method. The system formed based on the method can be divided into three independent parts, each of which contributes to the entire calibration process.

[0031] The first part can be used to acquire the dataset. In this part, when the user starts the camera application, he selects reading mode or writing mode to determine the resolution and frames per second (FPS) of the camera. The system captures the initial image from the camera and starts the calibration process. During the calibration process, the user can adjust various camera parameters, including brightness, contrast, hue, saturation, sharpness, gamma, white balance, exposure, focus, field of view (FoV), and image color inversion. The system allows the user to iterate through parameter adjustments until the desired output quality is achieved. Once the optimal calibration settings are reached, the system captures the final image from the camera, and simultaneously captures the initial image, final image, and corresponding camera parameter values. The collected data, including the initial image, final image, and camera parameter values, are stored on the server and constitute an important part of the data collection phase.

[0032] Part II: The model training phase consists of two stages. In the first stage, an autoencoder model is trained using the initial image as input and the final image as output. This autoencoder model learns to encode the initial image into a latent vector representation that captures the relevant features required for calibration. In the second stage, the trained autoencoder model becomes a component of the inference pipeline. The latent vector obtained from the encoder model is used as input to train a neural network. This neural network takes the latent vector as input and predicts the camera calibration parameters as output. Through this two-stage model training process, the present invention achieves the goal of accurately predicting camera calibration parameters.

[0033] Part 3: The model trained in the second phase can be deployed as a service, providing AI as a service. In the first phase of the inference pipeline, the initial image is fed into the autoencoder model, which extracts a latent vector. This latent vector is then passed to a second neural network model, which predicts camera parameter values ​​as output. These predicted values ​​are returned to the client application, enabling automatic calibration of high-mounted cameras for reading and writing purposes. By leveraging the trained model, the system provides real-time, automated camera calibration, enhancing the user experience.

[0034] The above-described method provided by the embodiments of the present application eliminates the need for manual calibration, thereby reducing manpower and saving time. By automating the calibration process, the system ensures consistent and accurate adjustment of camera parameters, improving image quality and performance. The system's real-time adaptability allows for instant adjustments to environmental changes, ensuring optimal calibration at all times.

[0035] In this embodiment, a user-friendly interface based on the camera application simplifies the calibration process, allowing users to intuitively adjust parameters. The system guides users through the calibration process, provides feedback, and captures the final settings to achieve the desired output quality. By providing a smooth and efficient calibration experience, this embodiment improves overall usability and user satisfaction.

[0036] In this embodiment of the application, the system's ability to capture and store data during the calibration process accumulates a rich dataset, enabling continuous improvement and optimization. The deployed training model can adapt to a wide range of high-mounted cameras used for reading and writing purposes, making the system highly scalable and adaptable to different setups and configurations. This scalability opens up opportunities for integration with a variety of applications and environments, including educational institutions, offices, libraries, and more.

[0037] This embodiment of the application utilizes artificial intelligence technology to reduce the need for manual intervention, thereby reducing the risk of human error and inconsistencies in the calibration process. The accuracy and precision of the model predictions ensure reliable calibration results, allowing users to trust and rely on the calibrated high-mount camera to meet their reading and writing needs.

[0038] By automating the calibration process, the system reduces the need for professionals or external calibration services, saving costs for individuals and organizations. The deployment model eliminates the need for expensive hardware or software investments, making the system accessible and affordable to a wide range of users.

[0039] As shown in FIG3 , in one embodiment, a camera calibration method is proposed. This embodiment is primarily illustrated by applying the method to the server 120 in FIG3 . A camera calibration method, applied to the server, comprises: obtaining an initial image uploaded by a mobile terminal; encoding the initial image into a latent vector representation, wherein the latent vector representation is used to characterize relevant features in the initial image required for calibrating an imaging device; analyzing the latent vector representation based on a neural network model, wherein the neural network model is used to predict calibration parameters required for calibrating the imaging device to obtain predicted calibration parameters; and transmitting the predicted calibration parameters to the mobile terminal.

[0040] In the embodiments of the present application, the description of the operation method and operation process of the above-mentioned model can be found above and will not be repeated here. This solution uses intelligent prediction technology to effectively reduce the reliance on manual calibration intervention, thereby reducing the risk of human error and inconsistency in the calibration process. The prediction model can perform real-time high-speed calibration of the imaging image, effectively improving the calibration accuracy and calibration speed, and is more suitable for high-frame rate camera equipment.

[0041] In one embodiment, the method for encoding the initial image into a latent vector representation is: obtaining an initial image, a final image, and configuration parameters provided by an application front end, wherein the final image is for calibrating the imaging device based on the configuration parameters; obtaining a frame of imaging image captured by the calibrated imaging device, and the camera configuration parameters at this time; constructing an autoencoder model with the initial image as input and the final image as output, wherein the autoencoder model is used to capture relevant features required for camera calibration; and encoding the initial image into a latent vector representation based on the autoencoding model.

[0042] In one embodiment, the method for obtaining the predicted calibration parameters based on the neural network model is: constructing a basic neural network; taking the latent vector representation as input, training the basic neural network to obtain a neural network model; the output of the neural network model is the predicted calibration parameters.

[0043] In one embodiment, the camera calibration method further includes: iteratively calibrating the imaging device based on the configuration parameters until an optimal calibration setting is reached; when the imaging device reaches the optimal calibration setting, obtaining a frame of imaging image captured by the mobile terminal, and using the image as the final image.

[0044] In one embodiment, the configuration parameters include at least one of the following parameters: brightness, contrast, hue, saturation, sharpness, gamma, white balance, exposure, focus, horizontal flip, vertical flip, Y-axis rotation, and field of view.

[0045] In one embodiment, a mobile application is provided, wherein the mobile application includes:

[0046] An initial image generation module, used to obtain an initial image based on an imaging device on a mobile terminal;

[0047] A prediction calibration parameter acquisition module is used to upload the initial image to a server and obtain prediction calibration parameters provided by the server, wherein the prediction calibration parameters are calibration parameters predicted by the server based on a neural network model for the initial image;

[0048] A calibration module is configured to calibrate the imaging device based on the predicted calibration parameters.

[0049] In the embodiments of this application, please refer to the above for the description of the operating methods of each module in the above mobile application, which will not be repeated here. The embodiments of this application utilize artificial intelligence technology to reduce the reliance on manual intervention, thereby reducing the risk of human error and inconsistencies in the calibration process. The accuracy and precision of the model predictions ensure reliable calibration results, allowing users to trust and rely on the calibrated high-mount camera to meet their reading and writing needs.

[0050] By automating the calibration process, the system reduces the need for professionals or external calibration services, saving costs for individuals and organizations. The deployment model eliminates the need for expensive hardware or software investments, making the system accessible and affordable to a wide range of users.

[0051] In one embodiment, a cloud-based software is provided, comprising: an initial image acquisition module for acquiring an initial image uploaded by a mobile terminal; a latent vector representation acquisition module for encoding the initial image into a latent vector representation, wherein the latent vector representation is used to characterize relevant features in the initial image required for calibrating an imaging device; a predicted calibration parameter generation module for analyzing the latent vector representation based on a neural network model, wherein the neural network model is used to predict calibration parameters required for calibrating the imaging device to obtain predicted calibration parameters; and a predicted calibration parameter transmission module for transmitting the predicted calibration parameters to the mobile terminal.

[0052] In the embodiments of this application, the description of the operating methods of each module in the aforementioned cloud software is provided above and will not be repeated here. By utilizing artificial intelligence technology, the system in this embodiment of the application reduces reliance on manual intervention, thereby reducing the risk of human error and inconsistencies during the calibration process. The accuracy and precision of the model predictions ensure reliable calibration results, allowing users to trust and rely on the calibrated high-mount camera to meet their reading and writing needs.

[0053] By automating the calibration process, the system reduces the need for professionals or external calibration services, saving costs for individuals and organizations. The deployment model eliminates the need for expensive hardware or software investments, making the system accessible and affordable to a wide range of users.

[0054] In one embodiment, a camera calibration system is provided, comprising: a mobile application as described above, for executing the camera calibration method as described above; and cloud software as described above, for interacting with the mobile application for executing the camera calibration method as described above.

[0055] In the embodiments of this application, the description of the operating methods of each module in the camera calibration system is provided above and will not be repeated here. By leveraging artificial intelligence technology, this embodiment of the application reduces reliance on manual intervention, thereby reducing the risk of human error and inconsistencies in the calibration process. The accuracy and precision of the model predictions ensure reliable calibration results, allowing users to trust and rely on the calibrated high-mount camera to meet their reading and writing needs.

[0056] By automating the calibration process, the system reduces the need for professionals or external calibration services, saving costs for individuals and organizations. The deployment model eliminates the need for expensive hardware or software investments, making the system accessible and affordable to a wide range of users.

[0057] A camera calibration method provided in an embodiment of the present application can effectively reduce reliance on manual calibration intervention, thereby reducing the risk of human error and inconsistency in the calibration process. The prediction model can perform real-time and high-speed calibration of the imaging image, effectively improving the calibration accuracy and calibration speed, and is more suitable for high frame rate camera equipment.

[0058] FIG6 shows an internal structure diagram of a computer device in one embodiment. The computer device may specifically be the mobile terminal 110 (or the server terminal 120) in FIG1 . As shown in FIG6 , the computer device includes a processor, a memory, a network interface, an input device, and a display screen connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement a camera calibration method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement a camera calibration method. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or may be a key, trackball, or touchpad provided on the computer device housing, or may be an external keyboard, touchpad, or mouse.

[0059] Those skilled in the art will understand that the structure shown in FIG6 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0060] In one embodiment, the mobile application and cloud software provided by this application can be implemented in the form of a computer program, which can be run on a computer device as shown in Figure 6. The memory of the computer device can store various program modules that constitute the mobile application and / or cloud software. The computer program composed of various program modules enables the processor to execute the steps of a camera calibration method in various embodiments of the present application described in this specification.

[0061] It should be understood that, although each step in the flow chart of each embodiment of the present application is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0062] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0063] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0064] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A camera calibration method, applied to a mobile terminal, characterized in that: The camera calibration method comprises: An initial image is obtained based on an imaging device on a mobile terminal; Uploading the initial image to a server, and obtaining prediction calibration parameters provided by the server, wherein the prediction calibration parameters are calibration parameters obtained by the server through prediction of the initial image based on a neural network model; Based on the predicted calibration parameters, the imaging device is calibrated.

2. A camera calibration method according to claim 1, characterized in that: The method further comprises: Acquire a working mode of the imaging device selected by a user, wherein the working mode includes a reading mode or a writing mode; The imaging resolution and shooting frame rate of the imaging device are determined based on the working mode.

3. A camera calibration method, applied to a server, characterized in that: The camera calibration method comprises: Get the initial image uploaded by the mobile terminal; encoding the initial image into a latent vector representation, the latent vector representation being used to characterize relevant features in the initial image required for calibrating an imaging device; Analyzing the latent vector representation based on a neural network model, wherein the neural network model is used to predict calibration parameters required for calibrating an imaging device to obtain predicted calibration parameters; The predicted calibration parameters are transmitted to the mobile terminal.

4. A camera calibration method according to claim 3, characterized in that: The method of encoding the initial image into a potential vector representation is: Get the initial image, final image and configuration parameters provided by the application front end, the final image For example, calibrating the imaging device based on the configuration parameters; Obtain a frame of imaging image captured by the calibrated imaging device and the camera configuration parameters at this time; Taking the initial image as input and the final image as output, constructing an autoencoder model, wherein the autoencoder model is used to capture relevant features required for camera calibration; Based on an auto-encoding model, the initial image is encoded into a latent vector representation.

5. A camera calibration method according to claim 3, characterized in that: The method for obtaining the predicted calibration parameters based on the neural network model is: Build a basic neural network; Taking the latent vector representation as input, training the basic neural network to obtain a neural network model; The output of the neural network model is the predicted calibration parameter.

6. A camera calibration method according to claim 4, characterized in that: The camera calibration method further comprises: Iteratively calibrating the imaging device based on the configuration parameters until an optimal calibration setting is achieved; When the imaging device reaches the optimal calibration setting, a frame of imaging image captured by the mobile terminal is obtained, and the image is used as the final image.

7. According to a camera calibration method according to any one of claims 4 or 6, the configuration parameters include at least one of the following parameters: brightness, contrast, hue, saturation, sharpness, gamma, white balance, exposure, focus, horizontal flip, vertical flip, Y-axis rotation and field of view.

8. A mobile application, characterized in that: The mobile application includes: An initial image generation module, used to obtain an initial image based on an imaging device on a mobile terminal; The prediction calibration parameter acquisition module is used to upload the initial image to the server and obtain the The predicted calibration parameters provided by the server are calibration parameters obtained by the server through prediction of the initial image based on the neural network model; A calibration module is used to calibrate the imaging device based on the predicted calibration parameters.

9. A cloud software, characterized in that: The cloud software includes: An initial image acquisition module is used to acquire the initial image uploaded by the mobile terminal; A latent vector representation acquisition module, used for encoding the initial image into a latent vector representation, wherein the latent vector representation is used for characterizing relevant features in the initial image required for calibrating an imaging device; A predicted calibration parameter generation module, configured to analyze the latent vector representation based on a neural network model, wherein the neural network model is configured to predict calibration parameters required for calibrating an imaging device to obtain predicted calibration parameters; The predicted calibration parameter transmission module is used to transmit the predicted calibration parameter to the mobile terminal.

10. A camera calibration system, characterized in that: The system comprises: A mobile application as claimed in claim 8, used to execute the camera calibration method as claimed in any one of claims 1 to 2; and The cloud software according to claim 9 interacts with the mobile application for executing the camera calibration method according to any one of claims 3 to 7.

Images