A camera calibration method, system, apparatus and electronic device

Abstract

Embodiments of the present application provide a camera calibration method, system, device and electronic equipment. In the embodiments, a host computer in a camera calibration system sends a control instruction to a controller in the camera calibration system, and the controller controls a first motion device to carry a single-sided calibration board to move and controls a second motion device to drive a double-sided calibration board to rotate according to the control instruction. After the host computer detects that the single-sided calibration board moves to a target position and / or the double-sided calibration board rotates to a target angle, the camera can be controlled to collect images of the single-sided calibration board and / or the double-sided calibration board in the field of view of the camera. The automatic collection of calibration images is realized, the problem of inaccurate camera calibration caused by manual operation errors is avoided, and the calibration efficiency is improved compared with conventional manual operation.

Description

Technical Field

[0001] This application relates to the field of camera calibration technology, and in particular to a camera calibration method, system, device and electronic device. Background Technology

[0002] In industrial production environments, camera calibration requires photographing various types of calibration boards (e.g., whiteboard calibration boards, checkerboard calibration boards, etc.) at different distances and angles. The camera is then calibrated based on these images. Currently, common camera calibration methods involve manually switching calibration boards and manually operating the camera or sensor to acquire the necessary images. Errors in this process can lead to inaccurate calibration, and the time-consuming manual switching and operation results in low calibration efficiency. Summary of the Invention

[0003] In view of this, embodiments of this application provide a camera calibration method, system, device, and electronic device to achieve automatic acquisition of calibration images and avoid the problem of inaccurate camera marking caused by manual operation errors.

[0004] According to a first aspect of the embodiments of this application, a camera calibration method is provided, the method being applied to a camera calibration system, the method comprising:

[0005] The host computer in the camera calibration system sends control commands to the controller in the camera calibration system.

[0006] The controller controls the first motion device in the camera calibration system to move, carrying the single-sided calibration plate, according to the control command, and / or controls the second motion device in the camera calibration system to rotate the double-sided calibration plate. The pattern on the single-sided calibration plate is the same as the pattern carried by one of the calibration plates in the double-sided calibration system. When the control command instructs the first motion device to move, carrying the single-sided calibration plate, the control command further carries the target position. When the control command instructs the second motion device to rotate the double-sided calibration plate, the control command further carries the target angle.

[0007] After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it controls the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or double-sided calibration plate within the camera's field of view using different camera acquisition parameters to obtain a calibration image set corresponding to the control command. Each calibration image set includes at least two black and white images and one color image; the calibration image set is used for camera calibration.

[0008] According to a second aspect of the present application, a camera calibration system is provided, comprising: a host computer, a camera, a controller, a calibration plate group, a first motion device, and a second motion device; the calibration plate group comprises: at least one single-sided calibration plate and at least four double-sided calibration plates, wherein the pattern on the single-sided calibration plate is the same as the pattern carried by one of the double-sided calibration plates; the first motion device is used to carry the single-sided calibration plate; the second motion device is used to drive the double-sided calibration plates to rotate; the camera is disposed at one end of the first motion device, and the at least four double-sided calibration plates are disposed at the other end of the first motion device, wherein the single-sided calibration plates and the double-sided calibration plates face the field of view of the camera;

[0009] The host computer is used to send control commands to the controller;

[0010] The controller, based on control commands sent by the host computer, controls the first motion device to move carrying the single-sided calibration plate in the camera calibration system, and / or controls the second motion device to rotate the double-sided calibration plate in the camera calibration system; wherein, when the control command instructs the first motion device to move carrying the single-sided calibration plate in the camera calibration system, the control command further instructs the target position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target angle;

[0011] After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it controls the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or double-sided calibration plate within the camera's field of view using different camera acquisition parameters to obtain a calibration image set corresponding to the control command. Each calibration image set includes at least two black and white images and one color image. The calibration image set is used for camera calibration.

[0012] According to a third aspect of the embodiments of this application, a camera calibration apparatus is provided, the apparatus being applied to a camera calibration system, the apparatus comprising:

[0013] The control command sending module is used to send control commands from the host computer in the camera calibration system to the controller in the camera calibration system.

[0014] The control command sending module is used to send control commands from the host computer in the camera calibration system to the controller in the camera calibration system.

[0015] A motion control module is configured to control a first motion device in the camera calibration system to move a single-sided calibration plate in the camera calibration system according to the control command via the controller, and / or control a second motion device in the camera calibration system to rotate a double-sided calibration plate in the camera calibration system; the pattern on the single-sided calibration plate is the same as the pattern carried by one of the calibration plates in the double-sided calibration system; wherein, when the control command instructs the first motion device to move the single-sided calibration plate in the camera calibration system, the control command further instructs the target position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target angle;

[0016] The calibration image group acquisition module is used to control the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or the double-sided calibration plate within the camera's field of view using different camera acquisition parameters after the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle. The acquired images are used to obtain the calibration image group corresponding to the control command. Each calibration image group includes at least two black and white images and one color image. The calibration image group is used for camera calibration.

[0017] According to a fourth aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor and a memory;

[0018] The memory is used to store machine-executable instructions;

[0019] The processor is configured to read and execute machine-executable instructions stored in the memory to implement the method as described in the first aspect.

[0020] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0021] As can be seen from the above technical solutions, in this embodiment, the host computer in the camera calibration system sends control commands to the controller in the camera calibration system. The controller then controls the first motion device to move the single-sided calibration plate and controls the second motion device to rotate the double-sided calibration plate according to the control commands. After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it can control the camera to acquire images of the single-sided calibration plate and / or the double-sided calibration plate within the camera's field of view. This achieves automatic acquisition of calibration images, avoids the problem of inaccurate camera marking caused by manual operation errors, and improves calibration efficiency compared to conventional manual operation. Attached Figure Description

[0022] Figure 1This is a block diagram of a camera calibration system provided in an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of a single-sided calibration plate provided in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram of the double-sided calibration plate provided in the embodiments of this application.

[0025] Figure 4 This is a flowchart of the camera calibration method provided in the embodiments of this application.

[0026] Figure 5 This is a block diagram of the camera calibration device provided in the embodiments of this application.

[0027] Figure 6 This is a hardware structure diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0031] The embodiments of this application will now be described in detail.

[0032] As described in the background section, the commonly used calibration and testing methods generally involve manually switching calibration boards and manually operating cameras or sensors to collect the images required for calibration and testing. This not only wastes human resources but also results in low calibration efficiency and accuracy. Therefore, this application provides a camera calibration method to achieve automatic acquisition of calibration images, avoiding the problems of inaccurate camera marking and low efficiency caused by manual operation errors.

[0033] See Figure 1 , Figure 1 The camera calibration system provided in the embodiments of this application, such as Figure 1 As shown, the camera calibration system includes: a first motion device, a camera, and a calibration plate assembly; except... Figure 1 The camera calibration system, as shown in the diagram, also includes a host computer, a controller, a second motion device, and a supplementary light.

[0034] The host computer is equipped with calibration and re-verification software, which is used to calibrate the camera and re-verify the calibration results. As an example, the host computer may be a terminal, a server, etc., but this application does not specifically limit the scope.

[0035] In this embodiment, the camera can be a stereo camera, which may include two black-and-white cameras and one color camera. The color camera is positioned between the two black-and-white cameras, and the distance from the color camera to the two black-and-white cameras may be the same or different. This embodiment of the application does not specifically limit the distance.

[0036] In this embodiment, the camera is connected to the host computer. Specifically, the connection between the camera and the host computer can be wired or wireless, and this embodiment of the application does not specifically limit the connection.

[0037] The controller is connected to a host computer, a first motion device, and a second motion device. It receives control commands sent by the host computer and controls the movement of the first motion device and / or the second motion device based on the control commands. As one embodiment, the controller can be a programmable logic controller (PLC), etc., but this application does not specifically limit the scope.

[0038] In this embodiment, the connection between the controller and the host computer, the connection between the controller and the first motion device, and the connection between the controller and the second motion device can all be wired or wireless. This application embodiment does not specifically limit the connection.

[0039] As an example, the host computer and the controller can communicate using the MODBUS TCP protocol.

[0040] In this embodiment, the first motion device can be a linear module, and the second motion device can be a cylinder. Both the first and second motion devices are used to carry the calibration plate in the calibration plate group. As for how the first and second motion devices carry the calibration plate in the calibration plate group, the following embodiments provide examples, which will not be repeated here.

[0041] In this embodiment, the calibration plate group includes: at least one single-sided calibration plate and at least four double-sided calibration plates. The pattern on the single-sided calibration plate is the same as the pattern carried by one of the double-sided calibration plates. Here, the pattern of the single-sided calibration plate can be a checkerboard pattern, or other patterns. This embodiment of the application does not specifically limit the pattern.

[0042] In this embodiment, the other side of the double-sided calibration plate can be a blank plate or a calibration plate with a pattern. The pattern carried can be the same as or different from the pattern on the single-sided calibration plate. This embodiment of the application does not specifically limit the specific pattern.

[0043] Here, the shape of both single-sided and double-sided calibration plates can be square, circular, etc., and the embodiments of this application are not specifically limited.

[0044] This application embodiment uses a single-sided calibration board with a checkerboard pattern and a quantity of one, and four double-sided calibration boards, with one side of the double-sided calibration board being a white calibration board and the other side being a checkerboard calibration board, as an example for illustration. Figure 2 As shown, the single-sided calibration plate is fixedly mounted on the calibration plate fixing fixture, as follows. Figure 3 As shown, two double-sided calibration plates are fixedly mounted on the same fixture, and the patterns of the two double-sided calibration plates mounted on each side of the fixture are the same, for example, as shown. Figure 3 As shown, the front of the fixture has a checkerboard pattern of two double-sided calibration plates, and the back of the fixture has a white background of two double-sided calibration plates. Furthermore, the four double-sided calibration plates are located at the upper left, lower left, upper right, and lower right corners of the single-sided calibration plates, respectively.

[0045] Based on the above description of the calibration plate assembly, the first motion device is used to carry the single-sided calibration plate and control the distance between the single-sided calibration plate and the camera, so as to acquire images of the single-sided calibration plate at different distances from the camera; the second motion device is used to drive the double-sided calibration plate to rotate. Here, the number of cylinders can be 4, of which 2 cylinders are used to drive the fixture of one double-sided calibration plate and the other 2 cylinders are used to drive the fixture of another double-sided calibration plate.

[0046] like Figure 1As shown, the camera is set at one end of the first motion device (i.e., the linear module), and at least four double-sided calibration plates are set at the other end of the first motion device. The single-sided calibration plates and the double-sided calibration plates face the field of view of the camera. Initially, the panel of the double-sided calibration plate facing the stereo camera can be a checkerboard calibration plate or a whiteboard calibration plate. This application embodiment does not specifically limit the specific calibration plate.

[0047] Furthermore, in this embodiment of the application, the camera calibration system may further include: a supplementary light, which may be placed around the stereo camera to provide supplementary lighting for the stereo camera, ensuring that the stereo camera can provide a light source for the camera under poor lighting conditions, so that the acquired image is clearer.

[0048] See Figure 4 , Figure 4 A flowchart illustrating a method provided in this application embodiment. This method is applied to the aforementioned camera calibration system and includes the following steps:

[0049] S410: Sends control commands from the host computer in the camera calibration system to the controller in the camera calibration system.

[0050] For example, in this embodiment, the host computer is used to send a control command to the controller after receiving the image acquisition command for acquiring calibration images. Here, the image acquisition command can be obtained by the user clicking the corresponding button on the operation interface of the calibration and re-examination software of the host computer, or it can be sent by the user to the host computer through its operable terminal. This embodiment of the application is not specifically limited.

[0051] In this embodiment, when the host computer receives an image acquisition command for acquiring calibration images, it sends a control command to the controller based on the configuration file corresponding to the camera's identification information (e.g., camera model). Here, the camera's identification information is obtained by the host computer after power-on and before the image acquisition command is sent. There are many ways to obtain the camera's identification information. For example, after the host computer is powered on, it sends a camera identification information acquisition request to the camera and then receives the camera's identification information based on the acquisition request. Another example is that after the host computer is powered on, the camera actively sends its identification information to the host computer. This embodiment of the application does not specifically limit the methods.

[0052] In this embodiment, the above-mentioned configuration file is predefined and stored in the host computer. It is matched with the camera model, that is, each camera model is matched with one configuration file. The configuration file may include at least one image acquisition configuration information, at least one image re-inspection configuration information, etc. This embodiment of the application does not specifically limit it.

[0053] The image acquisition configuration information is used to acquire calibration image groups. Each image acquisition configuration information includes at least: the target position of the single-sided calibration plate, the target angle between the double-sided calibration plate and the vertical direction, the target calibration plate of the double-sided calibration plate, and the number of image groups, etc. This application embodiment does not specifically limit the scope.

[0054] Image re-inspection configuration information is used to acquire re-inspection images. Each image re-inspection configuration information includes at least: the reference position of the single-sided calibration plate, the reference angle between the double-sided calibration plate and the vertical direction, the target calibration plate of the double-sided calibration plate, and the type of re-inspection image, etc. This application embodiment does not specifically limit the information.

[0055] Of course, the above-mentioned image re-inspection configuration information and image acquisition configuration information may also include: the moving speed of the first motion device, the rotation speed of the second motion device, etc., which are not specifically limited in this application embodiment.

[0056] Based on the above description of the configuration file, sending control commands to the controller according to the configuration file corresponding to the camera's identification information can specifically be as follows: when the camera needs to be calibrated, for each image acquisition configuration information, the host computer sends the control command corresponding to that image acquisition configuration information to the controller.

[0057] It should be noted that, in addition to sending control commands to the controller based on the configuration file as described above, there are other methods to send control commands. For example, when the host computer receives an image acquisition command for acquiring calibration images, it can also receive information manually entered on the operation interface of the calibration and re-inspection software and send control commands to the controller based on the received input information. This application embodiment is not specifically limited.

[0058] S420: The controller controls the first motion device in the camera calibration system to move the single-sided calibration plate in the camera calibration system according to the control command, and / or controls the second motion device in the camera calibration system to rotate the double-sided calibration plate in the camera calibration system.

[0059] For example, in this embodiment, after receiving the control command, the controller stores the control command in the PLC register. The PLC reads the register information to determine whether the control command controls the first motion device to move the single-sided calibration plate in the camera calibration system, controls the second motion device to rotate the double-sided calibration plate in the camera calibration system, or simultaneously controls the first motion device to move the single-sided calibration plate in the camera calibration system and controls the second motion device to rotate the double-sided calibration plate in the camera calibration system.

[0060] In this embodiment, when the control command instructs the first motion device to move the single-sided calibration plate in the camera calibration system, the control command further instructs the target position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target angle.

[0061] Specifically, when the first motion device is a linear module, controlling the first motion device to carry the single-sided calibration plate in the camera calibration system can be done by: sending a first control signal to the module control component (e.g., a motor) used to control the linear module according to the control command, so that the module control component carries the single-sided calibration plate based on the first control signal; when the second motion device is a cylinder, controlling the second motion device to drive the double-sided calibration plate in the camera calibration system to rotate can be done by: sending a second control signal to the cylinder control component (e.g., a pneumatic solenoid valve) of the cylinder according to the controller, so that the cylinder control component drives the double-sided calibration plate to rotate toward the camera based on the second control signal.

[0062] S430: After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it controls the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or double-sided calibration plate within the camera's field of view using different camera acquisition parameters to obtain the calibration image group corresponding to the control command.

[0063] For example, in this embodiment, the controller, according to control instructions, controls the first motion device to move the single-sided calibration plate in the camera calibration system to the target position, and / or controls the second motion device to rotate the double-sided calibration plate in the camera calibration system to the target angle. Then, it modifies the status bits of the register, for example, changing the status flag indicating "in progress" to a status flag indicating "completed". Based on the above description, the host computer detects the single-sided calibration plate moving to the target position and / or the double-sided calibration plate rotating to the target angle by reading the status bits of the controller's register.

[0064] In this embodiment, after the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it controls the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or double-sided calibration plate within the camera's field of view using different camera acquisition parameters. Here, the camera acquisition parameters refer to the camera pitch angle, that is, to acquire multiple sets of calibration image groups with different camera pitch angles.

[0065] For example, in this embodiment, the calibration image group corresponding to the control command is the M calibration image groups that need to be acquired under the control command. Here, the number M of calibration image groups that need to be acquired under each control command can be fixed, for example, 5; the number M of calibration image groups that need to be acquired under each control command can also be variable, that is, the number M of image groups carried by each control command is different, which can be determined according to the image acquisition configuration information corresponding to each control command.

[0066] In this embodiment, any calibration image set includes at least two black-and-white images and one color image. That is, the three cameras of the stereo camera simultaneously acquire images of the single-sided calibration plate and / or double-sided calibration plate within their field of view to obtain a calibration image set.

[0067] In this embodiment, the obtained calibration image set is used for camera calibration. Specifically, feature points of each calibration image in the calibration image set are extracted. For example, when the single-sided calibration board is a checkerboard calibration board, the aforementioned feature points are the corner points of the checkerboard. The camera is calibrated based on the position information of the feature points in the image coordinate system and the position information of the feature points in the world coordinate system. Specifically, the position information of the feature points in the image coordinate system and the position information of the feature points in the world coordinate system are input into the camera calibration algorithm to calculate the camera's internal parameters and external parameters. The camera's internal parameters include: the lens's focal length, optical center coordinates, distortion coefficients, etc., which describe the camera's internal structure and imaging characteristics. The camera's external parameters include: the rotation matrix and translation matrix from the camera coordinate system to the world coordinate system, etc., which describe the camera's position and orientation.

[0068] It should be noted that the specific camera calibration algorithm is a conventional camera calibration algorithm, which will not be described in detail here.

[0069] This concludes the process. Figure 4 The process is shown below.

[0070] pass Figure 4 As can be seen from the process, in this embodiment of the application, the host computer in the camera calibration system sends control commands to the controller in the camera calibration system. The controller then controls the first motion device to move the single-sided calibration plate and controls the second motion device to rotate the double-sided calibration plate according to the control commands. After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it can control the camera to acquire images of the single-sided calibration plate and / or the double-sided calibration plate within the camera's field of view. This achieves automatic acquisition of calibration images, avoids the problem of inaccurate camera marking caused by manual operation errors, and improves calibration efficiency compared to conventional manual operation.

[0071] In this embodiment, the calibration image set used for camera calibration can be acquired at a fixed position on both the single-sided and double-sided calibration plates, or it can be acquired at different target positions on the single-sided calibration plate and at different target angles on the double-sided calibration plate. When the calibration image set is acquired at different target positions on the single-sided calibration plate and at different target angles on the double-sided calibration plate, multiple target positions and multiple target angles can be preset. Image acquisition is performed based on the preset multiple target positions and multiple target angles. That is, when there are multiple sets of image acquisition configuration information, the target position and / or target angle and the number of image sets in each set of image acquisition configuration information are different.

[0072] As a specific embodiment, the image acquisition configuration information can be set according to close distance, long distance, and medium distance. Here, close distance, medium distance, and long distance refer to the distance between the single-sided calibration plate and the camera.

[0073] For example, at close range: target position is A1; target angle is B1; number of image groups M is C1; at medium range: target position is A2; target angle is B2; number of image groups M; at long range: target position is A3; target angle is B3; number of image groups M; then the number of calibration image groups is C3+C2+C1.

[0074] Specifically, sending control commands from the host computer to the controller can be done as follows: for each image acquisition configuration information, the host computer sends the corresponding control command to the controller.

[0075] The acquisition process of the calibration image group is described below using a specific example:

[0076] The number of calibration image groups used for camera calibration is 11; the number of image acquisition configuration information is 3, namely: close-range image acquisition configuration information, medium-range image acquisition configuration information, and long-range image acquisition configuration information. Among them, the number of image groups M in the close-range image acquisition configuration information is 5; the number of image groups M in the medium-range image acquisition configuration information is 5; the number of image groups M in the long-range image acquisition configuration information is 1; the close-range image acquisition configuration information and the medium-range image acquisition configuration information are used to acquire images of the single-sided calibration board, while the long-range image acquisition configuration information acquires images of both the single-sided and double-sided calibration boards.

[0077] For close-range image acquisition configuration information, the single-sided calibration plate moves to the target position. The camera's shooting posture is fine-tuned by changing the tilt angle of the stereo camera, and images are taken of the single-sided calibration plate on the linear module, resulting in 5 sets of calibration images. For mid-range image acquisition configuration information, the acquisition method is the same as that for close-range calibration image acquisition, except that the target position of the single-sided calibration plate is different, which will not be repeated here. For long-range image acquisition configuration information, five calibration plates need to be captured simultaneously, and the angles of the five calibration plates relative to the camera are all different.

[0078] As an optional implementation of this application, when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target calibration plate, which is one of the calibration plates in the double-sided calibration system; controlling the second motion device in the camera calibration system to rotate the double-sided calibration plate in the camera calibration system includes: controlling the second motion device to rotate the target calibration plate indicated by the control command toward the camera.

[0079] For example, in this embodiment, when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further carries the target calibration plate, causing the second motion device to rotate the target calibration plate indicated by the control command toward the camera, so that the target calibration plate is aligned with the camera's field of view.

[0080] Based on the above description, the specific image acquisition steps are as follows:

[0081] When the user clicks the "Automatic Image Acquisition" button on the calibration and re-inspection software on the host computer, the calibration and re-inspection software compiles the target position, target angle, and other information into a control command according to the image acquisition configuration information and sends it to the controller, and begins to monitor the status bits of the controller's registers.

[0082] The controller drives the linear module and cylinder to move according to the control command, moving the single-sided calibration plate to the target position and / or rotating the double-sided calibration plate to the target angle, and then changes the status bit in the controller's register.

[0083] The host computer monitors the status bits of the register in real time. When the status bit indicates "completed", it controls the stereo camera to acquire images with different camera acquisition parameters.

[0084] Repeat the above steps until each image acquisition configuration information has been executed once. Then, transmit the obtained calibration image group and system parameters (e.g., number of calibration boards, pattern of calibration boards, etc.) to the host computer for multi-target calibration to obtain the camera calibration results.

[0085] After calibration is complete, the camera calibration results are saved locally and set in the camera. The camera is then restarted to make the new calibration results take effect.

[0086] After completing the camera calibration, the calibration results need to be rechecked. This can be done through lens sharpness testing, speckle sharpness testing, depth map integrity testing, and calibration accuracy testing. Specifically:

[0087] As an optional implementation of this application, the above camera calibration method further includes:

[0088] First, the calibration results obtained by the host computer from the calibration image group corresponding to each control command of the camera are obtained, and the calibration results are sent to the camera for configuration.

[0089] Secondly, the host computer obtains the re-inspection images acquired after the camera is calibrated based on the configured calibration results. The calibration results are then re-inspected based on the re-inspection images to obtain the re-inspection results.

[0090] For example, in this embodiment, the calibration results obtained by the host computer from the calibration image group corresponding to each control command are obtained. Here, the calibration results can be obtained directly from the host computer or from other electronic devices. This embodiment of the application is not specifically limited and can be determined according to the specific device that performs the camera calibration.

[0091] In this embodiment, after obtaining the calibration result, the calibration result is sent to the camera for configuration. When the camera operates based on the configured calibration result, a re-inspection image is acquired after the camera has been calibrated. Here, the re-inspection image is used to re-inspect the calibration result to obtain the re-inspection result, which is used to determine whether the calibration result is qualified.

[0092] If the re-inspection result is qualified, the calibration result is deemed qualified, and the camera is deemed qualified. If the re-inspection result is unqualified, the calibration result is deemed unqualified, and the camera is calibrated again or the camera is deemed unqualified.

[0093] In this embodiment, the re-inspection results include the above-mentioned lens sharpness test results, speckle sharpness test results, depth map integrity test results, and calibration accuracy test results. If all items of the re-inspection results indicate that they are qualified, then the re-inspection results indicate that they are qualified. If any item of the re-inspection results indicates that they are unqualified, then the re-inspection results indicate that they are unqualified.

[0094] Specifically, for the lens clarity detection, feature points (e.g., checkerboard corner points) are extracted from the re-inspected images, and the position information of each feature point is input into a conventional clarity detection algorithm to output a clarity detection score. For example, 0.6. Based on this score and a preset score threshold (e.g., 0.8), the clarity detection result is obtained. For example, if the output clarity detection score is greater than or equal to the preset score threshold, it is determined that the clarity detection result is qualified; if the output clarity detection score is less than the preset score threshold, it is determined that the clarity detection result is unqualified. Here, six groups of checkerboard images are required for the lens clarity detection, and six calibration image groups can be randomly rotated from the 11 calibration image groups collected during camera calibration for the lens clarity detection.

[0095] For the speckle clarity detection, the clarity of the speckle image is judged by the comparative analysis of the re-inspected image (speckle image) and the standard speckle image, and a score corresponding to each speckle image is obtained. Based on the score of the speckle image, determining whether the speckle clarity detection is qualified is the same as the method of determining whether the lens clarity detection is qualified based on the output clarity detection score, which will not be elaborated here. The comparative analysis of the speckle image can adopt a conventional speckle evaluation algorithm, and the embodiments of the present application do not specifically limit it.

[0096] Here, the standard speckle image is a speckle image taken by a camera that has been determined to be qualified, and this standard speckle image and the re-inspected image are collected by cameras at the same position for a calibration board at the same position.

[0097] For the depth map integrity detection, a depth map including four double-sided calibration boards taken by the left black-and-white camera or the right black-and-white camera is required. The positions of the four double-sided calibration boards need to cover the four corners of the picture. The integrity of the depth map is judged according to the presence or absence of depth data at the four corners of the picture. When depth data exists at all four corners of the picture, it is determined that the depth map integrity detection result is qualified; when depth data does not exist at at least one of the four corners of the picture, it is determined that the depth map integrity detection result is unqualified.

[0098] For the detection of calibration accuracy, a total of four checkerboard calibration images are required, including one rectified image, one depth image, one RGB image at the same spatial position, and one RGB image at a different spatial position. Among them, the detection of calibration accuracy includes absolute accuracy and relative accuracy. For absolute accuracy, based on two RGB images at different spatial positions, calculate the moving distance corresponding to the two RGB images and compare it with the actual moving distance of the single-sided calibration plate to obtain the absolute accuracy detection result. For example, when the two distances are the same, determine that the absolute accuracy detection result is qualified; when they are different, determine that the absolute accuracy detection result is unqualified. For relative accuracy, it is calculated based on the rectified image, depth image, and RGB image collected at the same spatial position. Specifically, based on the rectified image and depth image, obtain the target RGB image in the camera coordinate system, and compare the target RGB image with the actually collected RGB image. According to the corner position deviation, obtain the relative accuracy detection result. For example, when the corner position deviation is less than or equal to the set deviation threshold, determine that the relative accuracy detection result is qualified; otherwise, determine that the relative accuracy detection result is unqualified.

[0099] As an optional implementation manner of an embodiment of the present application, the upper computer obtains the re-inspection images collected after the camera is calibrated based on the configured calibration results, including:

[0100] The upper computer obtains the image re-inspection configuration information for re-inspection.

[0101] Exemplarily, in this embodiment, the number of image re-inspection configuration information is multiple, and each image re-inspection configuration information includes at least a reference position and / or a reference angle, and the type of re-inspection image.

[0102] Here, the type of re-inspection image may include a depth image, an RGB image, a rectified image, a speckle image, etc., which are not specifically limited in the embodiment of the present application.

[0103] For each image re-inspection configuration information, the upper computer sends a control instruction corresponding to the image re-inspection configuration information to the controller.

[0104] The controller controls the first motion device to carry the single-sided calibration plate in the camera calibration system to move, and / or controls the second motion device to drive the double-sided calibration plate in the camera calibration system to rotate according to the control instruction; the control instruction corresponding to any image re-inspection configuration information carries at least the type of re-inspection image in the image re-inspection configuration information; when the control instruction instructs the first motion device to carry the single-sided calibration plate in the camera calibration system to move, the control instruction further carries the reference position; when the control instruction instructs the second motion device to drive the double-sided calibration plate in the camera calibration system to rotate, the control instruction further carries the reference angle;

[0105] After the host computer detects that the single-sided calibration plate has moved to the reference position and / or the double-sided calibration plate has rotated to the reference angle, it controls the camera in the camera calibration system to acquire re-examination images that meet the re-examination image type within the field of view of the single-sided calibration plate and / or double-sided calibration plate, based on the re-examination image type.

[0106] The specific implementation method is the same as the above-described calibration image group acquisition process, which can be found in the description of the above-described calibration image group acquisition process, and will not be repeated here.

[0107] It should be noted that when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command will also further instruct the target calibration plate to rotate toward the camera while controlling the second motion device to move.

[0108] When the re-examination image type is a speckle image type, the camera in the camera calibration system is controlled to acquire re-examination images that meet the re-examination image type within the single-sided calibration plate and / or double-sided calibration plate of the camera's field of view, according to the re-examination image type, including:

[0109] First, for each deployed speckle generator, the speckle generator is activated by the host computer to emit laser light to the single-sided calibration plate and / or double-sided calibration plate.

[0110] Secondly, the host computer controls the camera to acquire speckle images after each speckle generator is activated.

[0111] For example, in this embodiment, at least one speckle generator is deployed. Specifically, the speckle generator can be deployed on the camera. When acquiring speckle images, it is necessary to acquire the speckle image generated after each speckle generator emits a laser. Therefore, in this embodiment, for each deployed speckle generator, the host computer starts the speckle generator to emit a laser to a single-sided calibration plate and / or a double-sided calibration plate. The host computer controls the camera to acquire speckle images after each speckle generator is started, so as to obtain the speckle image corresponding to each speckle generator.

[0112] The following specific example describes the process of acquiring the re-examination image:

[0113] Image re-inspection configuration information may include: image re-inspection configuration information for calibration accuracy detection and image re-inspection configuration information for speckle sharpness detection. The re-inspection images used for lens sharpness detection can reuse the calibration image set acquired during camera calibration, and the depth maps used for depth map integrity detection can reuse the depth maps used for calibration accuracy detection.

[0114] 1. When the camera is running based on the calibration results, the host computer edits a control command based on the first reference position in the image re-inspection configuration information used for calibration accuracy detection and sends it to the PLC.

[0115] 2. After the PLC controls the linear module to move the single-sided calibration plate to the first reference position based on the control instructions, the host computer controls the camera to acquire an RGB image.

[0116] 3. After the first reference position in the image re-inspection configuration information for calibration accuracy detection is acquired, the linear module is controlled to move to the next reference position in the image re-inspection configuration information for calibration accuracy detection, and an RGB image is acquired. Then, the double-sided calibration board is flipped so that the white board faces the camera, the working mode is switched, and a correction image at the same position is acquired. Finally, the working mode is switched again to acquire a depth image.

[0117] 4. Keep the whiteboard facing the camera, switch the working mode to perform image capture for whiteboard sharpness detection.

[0118] 5. After collecting all the speckle patterns, send control commands to the PLC to control the single-sided calibration board and the double-sided calibration board to return to their original positions.

[0119] 6. Input the re-inspection image into the corresponding algorithm module to obtain the re-inspection result.

[0120] Corresponding to the embodiments of the aforementioned methods, embodiments of the present application also provide embodiments of the apparatus and the terminal to which it is applied.

[0121] like Figure 5 As shown, Figure 5 This is a block diagram of a camera calibration device provided in an embodiment of this application. The camera calibration device is applied to, for example... Figure 1 The camera calibration system shown includes:

[0122] The control command sending module is used to send control commands from the host computer in the camera calibration system to the controller in the camera calibration system.

[0123] The motion control module is used to control a first motion device in the camera calibration system to move a single-sided calibration plate in the camera calibration system according to control instructions via a controller, and / or to control a second motion device in the camera calibration system to rotate a double-sided calibration plate in the camera calibration system; the pattern on the single-sided calibration plate is the same as the pattern carried by one of the calibration plates in the double-sided calibration system; wherein, when the control instruction instructs the first motion device to move the single-sided calibration plate in the camera calibration system, the control instruction further instructs the target position; when the control instruction instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control instruction further instructs the target angle;

[0124] The calibration image group acquisition module is used to control the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or the double-sided calibration plate within the camera's field of view using different camera acquisition parameters after the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle. The resulting calibration image group corresponds to the control command. Each calibration image group includes at least two black and white images and one color image. The calibration image group is used for camera calibration.

[0125] As an optional implementation of this application embodiment, the above-mentioned control command sending module is specifically used for:

[0126] The image acquisition configuration information for camera calibration is obtained through a host computer; the image acquisition configuration information includes at least: target position and / or target angle, and number of image groups;

[0127] For each image acquisition configuration, the host computer sends the corresponding control command to the controller.

[0128] As an optional implementation of this application, the control command further carries the number of image groups M;

[0129] The number of calibration image groups corresponding to the control commands is M.

[0130] As an optional implementation of this application, the camera calibration device further includes:

[0131] The calibration result sending module is used to obtain the calibration results of the camera by the calibration image group corresponding to each control command through the host computer, and send the calibration results to the camera for configuration;

[0132] The re-inspection module is used to obtain re-inspection images from the camera after calibration based on the configured calibration results via the host computer, and to re-inspect the calibration results based on the re-inspection images to obtain the re-inspection results; the re-inspection results are used to determine whether the calibration results are qualified.

[0133] As an optional implementation of this application, the above-mentioned re-inspection module is specifically used for:

[0134] The image re-inspection configuration information for re-inspection is obtained through the host computer; the image re-inspection configuration information includes at least the reference position and / or reference angle, and the type of image to be re-inspected;

[0135] For each image re-inspection configuration information, the host computer sends the corresponding control command to the controller.

[0136] The controller controls the first motion device to move the single-sided calibration plate in the camera calibration system according to the control command, and / or controls the second motion device to rotate the double-sided calibration plate in the camera calibration system; the control command corresponding to any image re-examination configuration information carries at least the re-examination image type in the image re-examination configuration information; wherein, when the control command instructs the first motion device to move the single-sided calibration plate in the camera calibration system, the control command further carries the reference position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further carries the reference angle;

[0137] After the host computer detects that the single-sided calibration plate has moved to the reference position and / or the double-sided calibration plate has rotated to the reference angle, it controls the camera in the camera calibration system to acquire re-examination images that meet the re-examination image type within the field of view of the single-sided calibration plate and / or double-sided calibration plate, based on the re-examination image type.

[0138] As an optional implementation of this application, when the re-examination image type is a speckle image type, the camera in the camera calibration system is controlled to acquire re-examination images that meet the re-examination image type within the single-sided calibration plate and / or double-sided calibration plate of the camera's field of view, according to the re-examination image type, including:

[0139] For each deployed speckle generator, the speckle generator is activated by the host computer to emit laser light to the single-sided calibration plate and / or double-sided calibration plate.

[0140] The host computer controls the camera to acquire speckle images after each speckle generator is activated.

[0141] As an optional implementation of this application, when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target calibration plate, which is one of the double-sided calibration plates;

[0142] Controlling the second motion device in the camera calibration system to rotate the double-sided calibration plate in the camera calibration system includes: controlling the second motion device to rotate the target calibration plate indicated by the control command toward the camera.

[0143] As an optional embodiment of this application, the first motion device is a linear module and the second motion device is a cylinder;

[0144] Controlling the first motion device to carry the single-sided calibration plate in the camera calibration system to move according to the control command by the controller includes: sending a first control signal to the module control component for controlling the linear module according to the control command by the controller, so that the module control component carries the single-sided calibration plate to move based on the first control signal;

[0145] Controlling the second motion device to rotate the target calibration plate indicated by the control command toward the camera includes: sending a second control signal to the cylinder control component of the cylinder through the controller, so that the cylinder control component rotates the target calibration plate toward the camera based on the second control signal.

[0146] This completes the work on... Figure 5 Description of the block diagram shown.

[0147] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0148] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0149] Correspondingly, embodiments of this application also provide Figure 5 The hardware structure diagram of the device shown is as follows: Figure 6 As shown, the electronic device can be a device implementing the above-described method. Figure 6 As shown, the hardware architecture includes a processor and memory.

[0150] The memory is used to store machine-executable instructions;

[0151] The processor is configured to read and execute machine-executable instructions stored in the memory to implement the embodiment of the corresponding camera calibration method shown above.

[0152] As one embodiment, the memory can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, the memory can be volatile memory, non-volatile memory, or similar storage media. Specifically, the memory can be RAM (Random Access Memory), flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0153] This concludes the process. Figure 6 Description of the electronic device shown.

[0154] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0155] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0156] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0157] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A camera calibration method, characterized in that, The method is applied to a camera calibration system. The method includes: sending a control command corresponding to image acquisition configuration information to a controller in the camera calibration system via a host computer in the camera calibration system; the image acquisition configuration information is information obtained by the host computer for camera calibration; controlling a first motion device in the camera calibration system to move a single-sided calibration plate according to the control command, and / or controlling a second motion device in the camera calibration system to rotate a double-sided calibration plate; the pattern on the single-sided calibration plate is the same as the pattern carried by one side of the double-sided calibration plate; wherein, when the control command instructs the first motion device to move the single-sided calibration plate, the control command further carries a target position; when the control command instructs the second motion device to rotate the double-sided calibration plate, the control command further carries a target angle. After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it controls the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or double-sided calibration plate within the camera's field of view using different camera acquisition parameters to obtain a calibration image set corresponding to the control command. Each calibration image set includes at least two black and white images and one color image; the calibration image set is used for camera calibration.

2. The method according to claim 1, characterized in that, The step of sending control commands corresponding to the image acquisition configuration information from the host computer in the camera calibration system to the controller in the camera calibration system includes: The host computer obtains image acquisition configuration information for camera calibration; the image acquisition configuration information includes at least: the target position and / or target angle, and the number of image groups; For each image acquisition configuration, the host computer sends the corresponding control command to the controller.

3. The method according to claim 1 or 2, characterized in that, The control command further carries the number of image groups, M; The number of calibration image groups corresponding to the control command is M.

4. The method according to claim 1, characterized in that, The method further includes: The host computer obtains the calibration results of the camera by calibrating it according to the calibration image group corresponding to each control command, and sends the calibration results to the camera for configuration. The host computer obtains the re-inspection image acquired after the camera has been calibrated based on the configured calibration results. The calibration results are then re-inspected based on the re-inspection image to obtain the re-inspection result. The re-inspection result is used to determine whether the calibration results are qualified.

5. The method according to claim 4, characterized in that, The step of obtaining the re-inspection image acquired by the camera after calibration based on the configured calibration results through the host computer includes: The host computer obtains image re-inspection configuration information for re-inspection; the image re-inspection configuration information includes at least a reference position and / or a reference angle, and the type of image to be re-inspected; For each image re-inspection configuration information, the host computer sends the corresponding control command to the controller. The controller controls the first motion device to move carrying the single-sided calibration plate in the camera calibration system according to the control command, and / or controls the second motion device to rotate the double-sided calibration plate in the camera calibration system; the control command corresponding to any image re-examination configuration information carries at least the re-examination image type in the image re-examination configuration information; wherein, when the control command instructs the first motion device to move carrying the single-sided calibration plate in the camera calibration system, the control command further carries a reference position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further carries a reference angle; After the host computer detects that the single-sided calibration plate has moved to the reference position and / or the double-sided calibration plate has rotated to the reference angle, it controls the camera in the camera calibration system to acquire re-inspection images that meet the re-inspection image type within the field of view of the single-sided calibration plate and / or double-sided calibration plate according to the re-inspection image type.

6. The method according to claim 5, characterized in that, When the re-examination image type is a speckle image type, the step of controlling the camera in the camera calibration system to acquire re-examination images that satisfy the re-examination image type within the single-sided calibration plate and / or double-sided calibration plate of the camera's field of view based on the re-examination image type includes: For each deployed speckle generator, the host computer activates the speckle generator to emit a laser beam toward the single-sided calibration plate and / or the double-sided calibration plate. The host computer controls the camera to acquire speckle images after each speckle generator is activated.

7. The method according to claim 1 or 5, characterized in that, When the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target calibration plate, which is one of the double-sided calibration plates; The control of the second motion device in the camera calibration system to rotate the double-sided calibration plate in the camera calibration system includes: controlling the second motion device to rotate the target calibration plate indicated by the control command toward the camera.

8. The method according to claim 7, characterized in that, The first motion device is a linear module, and the second motion device is a cylinder; Controlling the first motion device to carry the single-sided calibration plate in the camera calibration system to move according to the control command by the controller includes: sending a first control signal to the module control component for controlling the linear module according to the control command by the controller, so that the module control component carries the single-sided calibration plate to move based on the first control signal; The method of controlling the second motion device to rotate the target calibration plate toward the camera as indicated by the control command includes: sending a second control signal to the cylinder control component of the cylinder through the controller, so that the cylinder control component rotates the target calibration plate toward the camera based on the second control signal.

9. A camera calibration system, characterized in that, The camera calibration system includes: a host computer, a camera, a controller, a calibration plate assembly, a first motion device, and a second motion device; the calibration plate assembly includes: at least one single-sided calibration plate and at least four double-sided calibration plates, wherein the pattern on the single-sided calibration plate is the same as the pattern carried by one of the double-sided calibration plates; the first motion device is used to move the single-sided calibration plate; the second motion device is used to drive the double-sided calibration plates to rotate; the camera is disposed at one end of the first motion device, and the at least four double-sided calibration plates are disposed at the other end of the first motion device, with the single-sided calibration plates and the double-sided calibration plates facing the field of view of the camera; The host computer is used to send control commands to the controller corresponding to the image acquisition configuration information; the image acquisition configuration information is information obtained by the host computer for camera calibration. The controller, based on control commands sent by the host computer, controls the first motion device to move carrying the single-sided calibration plate in the camera calibration system, and / or controls the second motion device to rotate the double-sided calibration plate in the camera calibration system; wherein, when the control command instructs the first motion device to move carrying the single-sided calibration plate in the camera calibration system, the control command further instructs the target position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further instructs the target angle; After the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, it controls the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or double-sided calibration plate within the camera's field of view using different camera acquisition parameters to obtain a calibration image set corresponding to the control command. Each calibration image set includes at least two black and white images and one color image. The calibration image set is used for camera calibration.

10. A camera calibration device, characterized in that, The device is applied to a camera calibration system. The device includes: a control command sending module, used to send control commands corresponding to image acquisition configuration information to a controller in the camera calibration system via a host computer in the camera calibration system; the image acquisition configuration information is information obtained by the host computer for camera calibration; and a motion control module, used by the controller to control a first motion device in the camera calibration system to carry a single-sided calibration plate in the camera calibration system to move according to the control commands, and / or to control a second motion device in the camera calibration system to drive a double-sided calibration plate in the camera calibration system to rotate; the pattern on the single-sided calibration plate is the same as the pattern carried by one of the calibration plates in the double-sided calibration system; wherein, when the control command instructs the control of the single-sided calibration plate to move, the control device can control the single-sided calibration plate to move. When the first motion device carries the single-sided calibration plate in the camera calibration system, the control command further carries the target position; when the control command instructs the second motion device to rotate the double-sided calibration plate in the camera calibration system, the control command further carries the target angle; the calibration image group acquisition module is used to control the camera in the camera calibration system to acquire images of the single-sided calibration plate and / or the double-sided calibration plate within the camera's field of view using different camera acquisition parameters after the host computer detects that the single-sided calibration plate has moved to the target position and / or the double-sided calibration plate has rotated to the target angle, thereby obtaining the calibration image group corresponding to the control command. Each calibration image group includes at least two black and white images and one color image; the calibration image group is used for camera calibration.

11. An electronic device, characterized in that, Electronic devices include: processors and memory; The memory is used to store machine-executable instructions; The processor is configured to read and execute machine-executable instructions stored in the memory to implement the method as described in any one of claims 1-8.

Images