Calibration method based on double shooting and imaging equipment

Abstract

The invention belongs to the technical field of equipment calibration, and discloses a double-shooting-based calibration method and imaging equipment, through a first shooting part and a second shooting part included in the imaging equipment, a first image containing a starry sky area can be identified from all first images collected by the first shooting part, and a second image containing a starry sky area can be identified from all second images collected by the second shooting part; a second shooting part is controlled to point to a starry sky area in the first image, and a mapping relation between the imaging device and the space-time coordinate system is obtained according to all second images collected by the second shooting part and corresponding collection moments; in the whole process, the imaging equipment can automatically avoid obstacles in any posture of any placement, manual intervention is not needed, and the calibration calculation efficiency and precision can be effectively guaranteed.

Description

Technical Field

[0001] The present application belongs to the technical field of device calibration, and in particular, relates to a dual-camera-based calibration method and imaging device. Background Art

[0002] With the increase in the number of astronomy enthusiasts and the continuous development of astronomical telescope technology, people's demand for observing the starry sky is growing. Due to the influence of the earth's rotation, the stars are moving relative to us at all times. Therefore, in order to accurately shoot the starry sky and celestial bodies in astronomical observation, it is necessary to adjust the position of the camera in real time to make it still relative to the stars. For example, when using an equatorial mount, after the polar axis is aligned, the rotation axis is controlled to rotate at the same speed as the earth's rotation to maintain relative stillness. Polar axis alignment is to coincide the tilt angle and tilt azimuth of the current device with the polar axis of the earth's rotation; for example, when using a theodolite, it is necessary to know the tilt angle and tilt azimuth of the current device. The subsequent shooting process can adjust the rotation of the theodolite in real time to track the target celestial body. Here, the polar axis alignment of the equatorial mount and the acquisition of the device tilt parameters of the theodolite are both calibration processes.

[0003] There are two commonly used calibration schemes. One is to manually observe the position of the North Star and manually adjust the direction of the device to point to the North Star. However, in actual operation, this calibration method cannot guarantee the calibration accuracy, and the manual calibration process is time-consuming and error-prone. The other is to use an equatorial mount or theodolite to control the device to automatically acquire images and use an algorithm to calculate the current tilt angle of the device, but both require manual intervention. In complex observation environments, such as when there are clouds blocking the sky or buildings around, it is also necessary to manually aim at the area with stars. If the captured image is blocked by clouds or buildings, it will also cause calibration failure. Summary of the invention

[0004] In order to solve the above-mentioned technical defects of low efficiency, cumbersome operation and easy error in the manual calibration process, and the need to manually point the lens to the sky area with stars when using an equatorial mount or theodolite, if there is interference from clouds or buildings, it will cause calibration failure, etc., a dual-camera calibration method and imaging device are proposed, and the technical solution is as follows: In a first aspect, an embodiment of the present application provides a dual-camera based calibration method, the method is applied to an imaging device, the imaging device includes a first shooting component and a second shooting component, the method includes: In at least one first image captured by the first photographing component, a first image including a starry sky area is identified; Controlling the second shooting component to point to the starry sky area in the first image; At least one second image captured by the second shooting component is acquired, and a mapping relationship between the imaging device and the space-time coordinate system is obtained according to all the second images and the acquisition time of each second image.

[0005] In an optional solution of the first aspect, the method further comprises: Acquire the field of view angle and attitude angle of the first shooting component, and plan a scanning path according to the field of view angle and the attitude angle; The first shooting component is rotated according to the scanning path, and at least one first image captured by the first shooting component after the rotation processing is acquired.

[0006] In another optional solution of the first aspect, the attitude angle includes an initial pitch angle and an initial yaw angle; The scanning path is planned according to the field of view angle and attitude angle, including: When the attitude angle is consistent with the preset initial value, the preset first angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; or When the attitude angle is inconsistent with the preset initial value, the initial pitch angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; Based on a preset angle rotation relationship, a first yaw attitude angle corresponding to the initial pitch angle is calculated, and at least one first yaw angle is obtained according to the first yaw attitude angle; Obtaining a corresponding first scanning position according to the starting pitch angle and each first yaw angle; Obtaining at least one first pitch angle in sequence according to the field of view angle and the starting pitch angle, and calculating a second yaw attitude angle corresponding to each first pitch angle based on a preset angle rotation relationship; Obtain at least one second yaw angle according to each second yaw attitude angle, and obtain a second scanning position based on each first pitch angle and each corresponding second yaw angle; The starting scanning position, all the first scanning positions and all the second scanning positions are arranged in a preset order, and a scanning path is planned according to the arrangement result.

[0007] In another optional solution of the first aspect, the scanning path includes at least three target scanning positions, each target scanning position is a starting scanning position, any one of the first scanning positions or any one of the second scanning positions; Rotating the first shooting component according to the scanning path and acquiring at least one first image captured by the first shooting component after the processing, including: According to each target scanning position in the scanning path, the first shooting component is rotated in turn, and a first image captured by the first shooting component when it is rotated to each target scanning position is acquired.

[0008] In another optional solution of the first aspect, identifying a first image containing a starry sky area in at least one first image captured by the first shooting component includes: In at least one first image captured by the first shooting component, recognition processing is performed on each captured first image based on a preset first algorithm to obtain a corresponding starry sky area; When the area of ​​any starry sky region exceeds a preset area threshold, the corresponding first image is used as the first image containing the starry sky region.

[0009] In yet another optional solution of the first aspect, in at least one first image captured by the first photographing component, identifying a first image containing a starry sky area, further comprising: Extracting a first star point position from each starry sky region according to a preset second algorithm; The first star point position is matched based on a preset star body coordinate library, and when the first star point position is matched successfully, the corresponding first image is used as the first image containing the starry sky area.

[0010] In yet another optional solution of the first aspect, controlling the second photographing component to point to a starry sky area in the first image includes: Acquire a third image captured by the second shooting component, and identify a first pixel coordinate of a fixed point of the third image in the first image containing a starry sky area; Constructing a rotation matrix according to the camera parameters of the first shooting component and the first pixel coordinates, and obtaining a first coordinate position based on the rotation matrix and the first pixel coordinates; In the first image including the starry sky region, a second pixel coordinate of a fixed point in the starry sky region is identified, and a second coordinate position is obtained according to the rotation matrix and the second pixel coordinate; The difference between the first coordinate position and the second coordinate position is used as the positional relationship between the first photographing component and the second photographing component, and the second photographing component is controlled to point to the starry sky area in the first image according to the positional relationship.

[0011] In yet another optional solution of the first aspect, acquiring at least one second image captured by the second shooting component includes: Controlling the exposure time of the second shooting component to be a preset exposure time, and acquiring a second image captured by the second shooting component; or Acquire a second image captured by a second photographing component, and rotate the second photographing component according to a preset second angle; The fourth image collected by the second shooting component after each rotation process is acquired, and the second image and all the fourth images are used as at least two second images.

[0012] In yet another optional solution of the first aspect, obtaining a mapping relationship between an imaging device and a spatiotemporal coordinate system according to all second images and the acquisition time of each second image includes: Identify the second star point position from each second image, and query the space coordinates corresponding to the second star point position in a preset star database; According to the spatial coordinates corresponding to the second star point position and the acquisition time of the corresponding second image, the coordinates of the star point in the space-time coordinate system are obtained, and based on the coordinates of the star point corresponding to at least one second image in the space-time coordinate system, the mapping relationship between the imaging device and the space-time coordinate system is obtained.

[0013] In yet another optional solution of the first aspect, obtaining a mapping relationship between an imaging device and the space-time coordinate system based on coordinates of a star point corresponding to at least one second image in the space-time coordinate system includes: Based on the coordinates of the star point corresponding to each second image in the space-time coordinate system, determining the coordinates of the corresponding image fixed point in the space-time coordinate system; The coordinates of the center of the circle are calculated according to the coordinates of each fixed point of the image in the space-time coordinate system and the preset equation; The coordinates of the center of the circle are used as the mapping relationship between the imaging device and the space-time coordinate system; or Substitute the coordinates of the circle center into the preset rotation matrix expression to obtain the mapping relationship between the imaging device and the space-time coordinate system.

[0014] In another optional solution of the first aspect, after acquiring at least one second image captured by the second shooting component, the method further includes: Acquire two second images, and calculate the rotation center position according to the two second images; The rotation center position is converted into coordinates in the space-time coordinate system, and according to the coordinates of the rotation center position in the space-time coordinate system, a mapping relationship between the imaging device and the space-time coordinate system is obtained.

[0015] In another optional solution of the first aspect, acquiring two second images, and calculating the rotation center position according to the two second images specifically includes: The pixel coordinates of the same points in the two second images are obtained, and the rotation center position is calculated according to the pixel coordinates of the same points.

[0016] In another optional solution of the first aspect, after acquiring at least one second image captured by the second shooting component, the method further includes: Obtaining the star point trajectory position in the second image, and calculating the trajectory center position according to the star point trajectory position; The center position of the trajectory is converted into coordinates in the space-time coordinate system, and the mapping relationship between the imaging device and the space-time coordinate system is obtained according to the coordinates of the center position of the trajectory in the space-time coordinate system.

[0017] In yet another optional solution of the first aspect, the camera type of the first shooting component is a wide-angle camera, and the camera type of the second shooting component is a telephoto camera.

[0018] In a second aspect, an embodiment of the present application provides an imaging device, the imaging device comprising a first shooting component and a second shooting component, the imaging device further comprising: An image recognition module, configured to recognize a first image containing a starry sky region in at least one first image captured by the first shooting component; A position control module, used for controlling the second shooting component to point to the starry sky area in the first image; The data calculation module is used to obtain at least one second image captured by the second shooting component, and obtain a mapping relationship between the imaging device and the space-time coordinate system according to all the second images and the capture time of each second image.

[0019] In a third aspect, an embodiment of the present application further provides an imaging device, including a processor and a memory; The processor is connected to the memory; A memory for storing executable program codes; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the dual-camera based calibration method provided by the first aspect of the embodiment of the present application or any one of the implementations of the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the dual-camera based calibration method provided by the first aspect of the embodiment of the present application or any one of the implementations of the first aspect can be executed.

[0021] Beneficial effects of this application: When performing automatic calibration processing, the first shooting component and the second shooting component included in the imaging device can be used to identify the first image containing the starry sky area from all the first images captured by the first shooting component, and then the second shooting component can be controlled to point to the starry sky area in the first image, and the mapping relationship between the imaging device and the space-time coordinate system can be obtained based on all the second images captured by the second shooting component and the corresponding capture time; the whole process can enable the imaging device to automatically avoid obstacles in any posture where it is placed, without the need for human intervention, and can effectively ensure the efficiency and accuracy of the calibration calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 An overall flow chart of a dual-camera based calibration method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of an imaging device provided in an embodiment of the present application; Figure 3 A schematic diagram of a preset angle rotation relationship provided in an embodiment of the present application; Figure 4 A schematic diagram of the positional relationship between a first photographing component and a second photographing component provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of another imaging device provided in an embodiment of the present application; Figure 6 A schematic structural diagram of another imaging device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0025] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, although the embodiment may not be clearly recorded in the following text.

[0026] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present application. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described in some examples may be combined in other examples.

[0027] See also Figure 1 , Figure 1 The figure shows an overall flow chart of a dual-camera based calibration method provided in an embodiment of the present application.

[0028] like Figure 1 As shown, the dual-camera based calibration method may include at least the following steps: Step 102: Identify a first image containing a starry sky area in at least one first image captured by a first photographing component.

[0029] In the embodiment of the present application, the calibration method based on dual-camera can be applied to, but is not limited to, an imaging device (or a processor of an imaging device). The imaging device can automatically avoid obstacles without human intervention in any posture at any placement position to complete the calibration process of the device. The imaging device can include at least a first shooting component and a second shooting component. The first shooting component can be, but is not limited to, composed of a first camera and a first rotatable component (the first rotatable component can adjust the attitude angle of the first camera, such as the pitch angle and the yaw angle), and the second shooting component can be, but is not limited to, composed of a second camera and a second rotatable component (the second rotatable component can adjust the attitude angle of the second camera, such as the pitch angle and the yaw angle). The first rotatable component and the second rotatable component can also be, but are not limited to, coaxially arranged, that is, the first rotatable component and the second rotatable component can include the same two rotation axes to achieve synchronous adjustment of the attitude angles of the first camera and the second camera.

[0030] Here, the first camera can be used to obtain images with a larger field of view angle to preliminarily identify the location of the starry sky area; the second camera can be used to obtain images with a smaller field of view angle to obtain corresponding clear starry sky images according to the starry sky area determined by the first camera, that is, the field of view angle of the first camera is greater than the field of view angle of the second camera (for example, the horizontal field of view angle of the first camera is greater than the horizontal field of view angle of the second camera, and the vertical field of view angle of the first camera is greater than the vertical field of view angle of the second camera), and in the embodiment of the present application, the camera type of the first camera may be a wide-angle camera, and the camera type of the second camera may be a telephoto camera, but is not limited to this.

[0031] Of course, in addition to the first shooting component and the second shooting component mentioned above, the imaging device may also include but is not limited to conventional structures and functions such as a housing, a support base, a lens and a processor, which will not be elaborated here.

[0032] Also see here Figure 2 The structural diagram of an imaging device provided by an embodiment of the present application is shown in FIG. Figure 2 As shown, the imaging device may at least include a processor (not shown in the figure), a first camera 1, a second camera 2, a first rotation axis 3 and a second rotation axis 4 connected to the processor, the first camera 1 may be a wide-angle camera, the second camera 2 may be a telephoto camera, the first rotation axis 3 may be used to synchronously adjust the pitch angle of the first camera 1 and the second camera 2, and the second rotation axis 4 may be used to synchronously adjust the yaw angle of the first camera 1 and the second camera 2. Of course, Figure 2 The first rotation axis 3 shown can also be used as a universal joint to adjust the pitch angle and yaw angle of the first camera 1 alone, and the second rotation axis 4 can also be used as a universal joint to adjust the pitch angle and yaw angle of the second camera 2 alone, but is not limited thereto.

[0033] It can be understood that the imaging device can identify the first image containing the starry sky area from all the first images captured by the first shooting component, and then use the positional relationship between the first shooting component and the second shooting component to control the second shooting component to point to the starry sky area, and obtain the mapping relationship between the imaging device and the space-time coordinate system based on all the second images captured by the second shooting component and the corresponding capture time; the whole process can enable the imaging device to automatically avoid obstacles in any posture under any placement without human intervention, and can effectively ensure the efficiency and accuracy of the calibration calculation.

[0034] Specifically, when performing the automatic calibration process, the imaging device may, but is not limited to, determine whether there is a valid starry sky area from each first image sequentially collected by the first shooting component. The valid starry sky area can be understood as a starry sky area whose size meets the specified requirements, and the first image corresponding to the starry sky area that meets the specified requirements can be used as the first image containing the starry sky area, so as to automatically adjust the attitude angle of the second shooting component according to the first image containing the starry sky area. It can be understood that when the first shooting component collects each first image, the corresponding attitude angle is different (that is, the shooting angle of each first image or the sky area captured is different). For example, when collecting the first first image, the corresponding pitch angle can be A1 and the yaw angle can be B1; when collecting the second first image, the corresponding pitch angle can be A1 (of course, it can also be A2), and the yaw angle can be B2. After identifying a first image containing the starry sky area, the imaging device can stop adjusting the attitude angle of the first shooting component, that is, no longer controlling the first shooting component to collect the first image.

[0035] As an option of the embodiment of the present application, the method further includes: Acquire the field of view angle and attitude angle of the first shooting component, and plan a scanning path according to the field of view angle and the attitude angle; The first shooting component is rotated according to the scanning path, and at least one first image captured by the first shooting component after the rotation processing is acquired.

[0036] Specifically, when acquiring each first image captured by the first shooting component, the imaging device can also acquire the field of view angle and attitude angle of the first shooting component, the field of view angle can be understood as the horizontal field of view angle and the vertical field of view angle of the camera included in the first shooting component (for example, it can be directly obtained from the specification parameters of the camera, or calculated from the focal length of the camera and the sensor size), and the attitude angle can be understood as the pitch angle and yaw angle of the rotating component currently located in the first shooting component (which can be collected by the sensor set by the rotating component), and the scanning path corresponding to the first shooting component when capturing each first image can be planned in advance according to the field of view angle and attitude angle. Here, the scanning path can include but is not limited to multiple scanning positions arranged in sequence, and the pitch angle and yaw angle corresponding to each scanning position are different (for example, the pitch angle is different, the yaw angle is different, or the pitch angle and yaw angle are different).

[0037] Then, after planning the scanning path, the imaging device can control the first shooting component to rotate according to the pitch angle and yaw angle corresponding to each scanning position in turn according to the arrangement order of the scanning positions in the scanning path, so that the pitch angle and yaw angle of the first shooting component after rotation are consistent with the pitch angle and yaw angle corresponding to the corresponding scanning position, and can control the first shooting component to capture the first image of the corresponding scanning position, and can further determine whether there is a starry sky area that meets the specified requirements in the first image.

[0038] It should be noted that when the first shooting component captures the first image corresponding to any scanning position, which contains a starry sky area that meets the specified requirements, there is no need to rotate the first shooting component according to the next scanning position of the corresponding scanning position in the scanning path, that is, the image acquisition task of the first shooting component is completed (or the scan is completed).

[0039] As another option of the embodiment of the present application, the attitude angle includes an initial pitch angle and an initial yaw angle; The scanning path is planned according to the field of view angle and attitude angle, including: When the attitude angle is consistent with the preset initial value, the preset first angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; or When the attitude angle is inconsistent with the preset initial value, the initial pitch angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; Based on a preset angle rotation relationship, a first yaw attitude angle corresponding to the initial pitch angle is calculated, and at least one first yaw angle is obtained according to the first yaw attitude angle; Obtaining a corresponding first scanning position according to the starting pitch angle and each first yaw angle; Obtaining at least one first pitch angle in sequence according to the field of view angle and the starting pitch angle, and calculating a second yaw attitude angle corresponding to each first pitch angle based on a preset angle rotation relationship; Obtain at least one second yaw angle according to each second yaw attitude angle, and obtain a second scanning position based on each first pitch angle and each corresponding second yaw angle; The starting scanning position, all the first scanning positions and all the second scanning positions are arranged in a preset order, and a scanning path is planned according to the arrangement result.

[0040] Specifically, when planning the scanning path of the first shooting component, the imaging device can also determine whether the imaging device has performed scanning before based on the initial pitch angle and initial yaw angle of the first shooting component. For example, when the initial pitch angle is consistent with the corresponding preset initial value, and the initial yaw angle is consistent with the corresponding preset initial value, it indicates that the imaging device has not performed scanning before, and then the preset first angle can be used as the starting pitch angle, and the starting pitch angle and the initial yaw angle are used as the starting scanning position (that is, the first scanning position) in the scanning path. It should be noted that when the first shooting component performs the first rotation process, the corresponding pitch angle and yaw angle are consistent with the starting scanning position. Here, the preset first angle can be but is not limited to 80 degrees, that is, it is preset to a higher pitch angle, which can make the yaw angle step size in the subsequent generated scanning position larger, thereby improving the execution efficiency of the scanning path.

[0041] It can be understood that when the initial pitch angle is inconsistent with the corresponding preset initial value, or the initial yaw angle is inconsistent with the corresponding preset initial value, it indicates that the imaging device has performed a scan before, and the initial pitch angle can be used as the starting pitch angle, and the starting pitch angle and the initial yaw angle are used as the starting scanning position in the scanning path. It should be noted that when the first shooting component performs the first rotation processing, the corresponding pitch angle and yaw angle are consistent with the next scanning position calculated from the starting scanning position.

[0042] Then, after determining the starting scanning position, the imaging device can also calculate the first yaw attitude angle corresponding to the starting pitch angle based on a preset angle rotation relationship. The first yaw attitude angle can be understood as a single yaw angle rotation step corresponding to the initial pitch angle, so that among the multiple first yaw angles determined according to the first yaw attitude angle, the angle difference between any two adjacent first yaw angles is the first yaw attitude angle (and the sum of all angle differences is greater than or equal to 360 degrees to achieve 360-degree full scanning coverage), and the starting pitch angle can be combined with each first yaw angle to obtain the corresponding first scanning position. For example, taking the starting pitch angle of the starting scanning position as A and the initial yaw angle as B, when the corresponding first yaw attitude angle is determined to be 90 degrees according to the starting pitch angle, four first scanning positions can be obtained in sequence, which are respectively represented as the pitch angle A and the yaw angle B+90, the pitch angle A and the yaw angle B+180, the pitch angle A and the yaw angle B+270, and the pitch angle A and the yaw angle B+360.

[0043] Here, the preset angle rotation relationship may be but is not limited to the following: In the above formula, alt can be the starting pitch angle, FOV can be the horizontal field of view angle of the first shooting component, It can be a first yaw attitude angle.

[0044] Also see here Figure 3 A schematic diagram of a preset angle rotation relationship provided by an embodiment of the present application is shown, such as Figure 3 As shown, point A and point B can be represented as two adjacent first scanning positions with a higher pitch angle, and the angle between the two first scanning positions is 80 degrees (that is, the first yaw attitude angle is 80 degrees); point C and point D can be represented as two adjacent first scanning positions with a lower pitch angle, and the angle between the two first scanning positions is 50 degrees (that is, the first yaw attitude angle is 50 degrees). It can be seen that when the pitch angle is higher, the corresponding single yaw angle rotation step is larger; when the pitch angle is lower, the corresponding single yaw angle rotation step is smaller.

[0045] Then, after obtaining multiple first scanning positions, the imaging device can also calculate multiple first pitch angles in sequence according to the above-mentioned field of view angle and starting pitch angle, in order from high to low pitch angles, so that the angle difference between any two adjacent first pitch angles is half of the vertical field of view angle. For example, taking the starting pitch angle as A and half of the vertical field of view angle as B as an example, the first pitch angles calculated in sequence can be expressed as AB, A-2B, A-3B, etc., respectively, and the last first pitch angle must be guaranteed to be greater than 0 degrees.

[0046] Then, after obtaining multiple first pitch angles, the imaging device can also calculate the second yaw attitude angle corresponding to each first pitch angle based on the preset angle rotation relationship mentioned above. The second yaw attitude angle can be understood as a single yaw angle rotation step corresponding to the corresponding first pitch angle, so that among the corresponding multiple second yaw angles determined according to each second yaw attitude angle, the angle difference between any two adjacent second yaw angles is the second yaw attitude angle (and the sum of all angle differences is greater than or equal to 360 degrees to achieve 360-degree full scanning coverage), and each first pitch angle can be combined with each corresponding second yaw angle to obtain the corresponding second scanning position. Here, the first second yaw angle corresponding to each first pitch angle corresponds to the last second yaw angle corresponding to the previous pitch angle, that is, after determining all the second scanning positions corresponding to any pitch angle in the yaw direction, the next pitch angle of the pitch angle and the last second yaw angle corresponding to the pitch angle can be used as the next scanning position of all the second scanning positions, and also as the first second scanning position corresponding to the next pitch angle in the yaw direction.

[0047] Then, after all the second scanning positions are determined, the imaging device may also arrange the starting scanning position, all the first scanning positions, and all the second scanning positions in a preset order, and plan a scanning path according to the arrangement result. Here, the preset order may be, but is not limited to, the order from large to small pitch angles, and all scanning positions corresponding to any pitch angle may be in the order from small to large yaw angles, and is not limited thereto.

[0048] As another option of the embodiment of the present application, the scanning path includes at least three target scanning positions, each target scanning position is a starting scanning position, any first scanning position or any second scanning position; Rotating the first shooting component according to the scanning path and acquiring at least one first image captured by the first shooting component after the processing, including: According to each target scanning position in the scanning path, the first shooting component is rotated in turn, and a first image captured by the first shooting component when it is rotated to each target scanning position is acquired.

[0049] Specifically, after obtaining the scanning path, the imaging device can also rotate the first shooting component according to each target scanning position arranged in sequence in the scanning path, so as to obtain the first image captured by the first shooting component after the first shooting component is rotated to the corresponding pitch angle and yaw angle, and after identifying that there is no starry sky area that meets the specified requirements in the corresponding first image, the first shooting component can be rotated again according to the next target scanning position of the corresponding target scanning position, and the above steps are repeated until a starry sky area that meets the specified requirements is identified in a certain first image; when it is identified that there is a starry sky area that meets the specified requirements in any first image, the imaging device does not need to rotate the first shooting component according to the next scanning position of the corresponding scanning position in the scanning path.

[0050] It can be understood that when the first shooting component is rotated according to the last target scanning position in the scanning path, it is assumed that after the first shooting component is rotated to the corresponding pitch angle and yaw angle, the first image captured is the first image containing the starry sky area.

[0051] As another option of the embodiment of the present application, in at least one first image captured by the first shooting component, identifying a first image containing a starry sky area includes: In at least one first image captured by the first shooting component, recognition processing is performed on each captured first image based on a preset first algorithm to obtain a corresponding starry sky area; When the area of ​​any starry sky region exceeds a preset area threshold, the corresponding first image is used as the first image containing the starry sky region.

[0052] Specifically, when determining whether there is a valid starry sky area in each first image, the imaging device can also perform recognition processing on each first image based on a preset first algorithm, and the preset first algorithm can be a starry sky segmentation algorithm based on deep learning (of course, it can also be a traditional image processing algorithm or a machine learning algorithm), such as but not limited to a UNet-based image semantic segmentation model, which is obtained by training with multiple sky image data samples marked with starry sky area labels, and after the image semantic segmentation model training is completed, each first image can be input into the trained image semantic segmentation model to obtain the corresponding starry sky area. It can be understood that the starry sky area can be understood as an area in the night sky where stars can be observed without any obstructions.

[0053] Then, after obtaining the starry sky area corresponding to the first image, the imaging device can also judge whether the specified requirements are met based on the area of ​​the starry sky area, that is, whether there is a valid starry sky area. For example, when it is detected that the area of ​​any starry sky area exceeds the preset area threshold, it indicates that there is a valid starry sky area, and the first image corresponding to the starry sky area can be used as the first image containing the starry sky area.

[0054] As another option of the embodiment of the present application, in at least one first image captured by the first shooting component, identifying a first image containing a starry sky area, further comprising: Extracting a first star point position from each starry sky region according to a preset second algorithm; The first star point position is matched based on a preset star body coordinate library, and when the first star point position is matched successfully, the corresponding first image is used as the first image containing the starry sky area.

[0055] Specifically, when determining whether there is a valid starry sky region in each first image, the imaging device may also perform recognition processing on each first image based on a preset first algorithm to obtain a corresponding starry sky region. Here, the recognition processing method can refer to the above embodiment, and will not be described in detail here.

[0056] Then, after obtaining the starry sky area in each first image, the imaging device can also extract the first star point position from each starry sky area according to a preset second algorithm. The first star point position can be understood as at least two star point pixel coordinates in the starry sky area (preferably five star point pixel coordinates). The preset second algorithm can be used to extract the pixel coordinates of one or more star points in the image, which can include but is not limited to processing algorithms such as image scaling, denoising, binarization, dilation and erosion, and connected domain extraction, and these various types of processing algorithms are all technical means well known in the art.

[0057] Next, after extracting the first star point position in each starry sky area, the imaging device may also perform matching processing on the first star point position in each starry sky area based on the preset star coordinate library, and when the matching result of the first star point position in any starry sky area is a successful match, it indicates that there are multiple stars in the starry sky area that are fully distributed, and the corresponding first image may be used as the first image containing the starry sky area. Here, the preset star coordinate library may be obtained by analyzing the star coordinate big data well known in the art, which includes at least two star point pixel coordinate sets and multiple star point pixel coordinates corresponding to each star point pixel coordinate set, and when there are star point pixel coordinates with matching positional relationships between any star point pixel coordinate set and the first star point position in any starry sky area, it may indicate that the matching result of the first star point position in the starry sky area is a successful match.

[0058] Step 104: Control the second photographing component to point to the starry sky area in the first image.

[0059] Specifically, after identifying the first image containing the starry sky area, the imaging device can, but is not limited to, determine the positional relationship of the third image in the first image containing the starry sky area based on the third image currently captured by the second shooting component, and use the positional relationship to control the second shooting component to rotate so that the second shooting component rotates to point the lens to the starry sky area. Here, the positional relationship can be understood as the difference in pitch angle and yaw angle between the area photographed by the second shooting component and the starry sky area photographed by the first shooting component, and the imaging device can rotate the current pitch angle of the second shooting component according to the difference in pitch angle, and can also rotate the current yaw angle of the second shooting component according to the difference in yaw angle.

[0060] As another option of the embodiment of the present application, controlling the second shooting component to point to the starry sky area in the first image includes: Acquire a third image captured by the second shooting component, and identify a first pixel coordinate of a fixed point of the third image in the first image containing a starry sky area; Constructing a rotation matrix according to the camera parameters of the first shooting component and the first pixel coordinates, and obtaining a first coordinate position based on the rotation matrix and the first pixel coordinates; In the first image including the starry sky region, a second pixel coordinate of a fixed point in the starry sky region is identified, and a second coordinate position is obtained according to the rotation matrix and the second pixel coordinate; The difference between the first coordinate position and the second coordinate position is used as the positional relationship between the first photographing component and the second photographing component, and the second photographing component is controlled to point to the starry sky area in the first image according to the positional relationship.

[0061] Specifically, after identifying the first image containing the starry sky area, the imaging device may also obtain the third image captured by the second shooting component at the current attitude angle, so as to use the image feature matching algorithm to identify the area where the third image is located in the first image containing the starry sky area, and combine the distance from the edge of the first image to any vertex of the area where the third image is located, and the size of the area where the third image is located to obtain the position information of the third image. Here, the position information of the third image may be, but is not limited to, the pixel coordinates (i.e., the first pixel coordinates) of a fixed point of the third image (which may be, but is not limited to, the center of the image) in the first image. It is understandable that the image feature matching algorithm may be an image processing method well known in the art, and in the embodiment of the present application, it may also be, but is not limited to, taking the pixel coordinates of any vertex or other specified position of the third image in the first image as the position information of the third image, and is not limited to this.

[0062] Also see here Figure 4 A schematic diagram showing the positional relationship between a first photographing component and a second photographing component provided in an embodiment of the present application is shown. Figure 4 As shown, taking the fixed point as the center of the image as an example, the third image captured by the second shooting component is shown on the right, and the first image containing the starry sky area is shown on the left. A certain area in the first image (length is represented by w, width is represented by h) corresponds to the area where the third image is located. Since the distance from the edge of the first image to the upper left corner vertex of the area where the third image is located is represented by dx (distance in the length direction) and dy (distance in the width direction), the pixel coordinates of the center of the third image in the first image (i.e., the first pixel coordinates) can be obtained as (dx+w / 2, dy+h / 2).

[0063] Next, after obtaining the first pixel coordinates, the imaging device may also construct a rotation matrix according to the camera parameters of the first shooting component and the first pixel coordinates. The expression of the rotation matrix may be, but is not limited to, as follows: In the above formula, It can be expressed as a rotation matrix, It can be expressed as the camera parameter of the first shooting component (it can also be understood as focal length), It can be expressed as the horizontal coordinate in the first pixel coordinate, Can be expressed as the vertical coordinate in the first pixel coordinate.

[0064] Next, the imaging device may also convert the first pixel coordinates mentioned above into homogeneous coordinates, and obtain the device coordinate position (i.e., the first coordinate position) of the fixed point of the third image in the first image according to the homogeneous coordinates, the rotation matrix mentioned above, and the coordinate conversion relationship determined according to the camera calibration parameters of the first shooting component. It can be understood that the coordinate conversion relationship determined according to the camera calibration parameters of the first shooting component can be understood as a conversion relationship from the camera coordinate system of the first shooting component to the device coordinate system of the imaging device, which belongs to the coordinate system conversion means well known in the art and will not be elaborated here.

[0065] Here, the expression of the first coordinate position may be, but is not limited to, as follows: In the above formula, It can be expressed as the device coordinate position of the fixed point of the third image in the first image (that is, the first coordinate position), It can be expressed as the inverse matrix of the coordinate transformation relationship determined according to the camera calibration parameters of the first shooting component, can be expressed as the inverse matrix of the rotation matrix, It can be represented as the homogeneous coordinate transformed from the first pixel coordinate.

[0066] Then, after obtaining the first position coordinates, the imaging device can also identify the pixel coordinates of the fixed point of the starry sky area in the first image mentioned above that contains the starry sky area, that is, the second pixel coordinates, and can combine the homogeneous coordinates converted from the second pixel coordinates, the rotation matrix mentioned above, and the coordinate transformation relationship determined according to the camera calibration parameters of the first shooting component to obtain the device coordinate position (that is, the second coordinate position) of the center of the starry sky area in the first image.

[0067] Here, the expression of the second coordinate position may be, but is not limited to, as follows: In the above formula, It can be expressed as the device coordinate position of the fixed point in the starry sky area in the first image (that is, the second coordinate position), It can be expressed as the inverse matrix of the coordinate transformation relationship determined according to the camera calibration parameters of the first shooting component, can be expressed as the inverse matrix of the rotation matrix, can be expressed as the homogeneous coordinate transformed from the second pixel coordinate, can be expressed as the horizontal coordinate in the second pixel coordinate, Can be expressed as the ordinate in the second pixel coordinates.

[0068] Next, after respectively obtaining the first coordinate position and the second coordinate position, the imaging device may further perform a difference calculation between the first coordinate position and the second coordinate position, and use the difference result as the positional relationship between the first photographing component and the second photographing component.

[0069] Of course, in the embodiment of the present application, the positional relationship between the first shooting component and the second shooting component can be determined by a calibration method based on the camera calibration parameters of the first shooting component and the camera calibration parameters of the second shooting component, but it will not be elaborated here.

[0070] Step 106: Acquire at least one second image captured by the second shooting component, and obtain a mapping relationship between the imaging device and the space-time coordinate system according to all the second images and the capture time of each second image.

[0071] Specifically, after controlling the second shooting component to point to the starry sky area, the imaging device can, but is not limited to, acquiring one or more second images including the starry sky area captured by the second shooting component. Each of the multiple second images can be captured by the second shooting component when it is rotated to a different attitude angle. The number of the multiple second images can be specifically divided into two, three or more than three. For different numbers of second images, the mapping relationship between the imaging device and the space-time coordinate system can be obtained based on all the second images and the corresponding capture times.

[0072] It can be understood that the space-time coordinate system mentioned in the embodiments of the present application may include, but is not limited to, space coordinates and time coordinates. The space coordinates can be used to represent the position of an object, and the time coordinates can be used to represent a certain moment. For example, taking the space-time coordinate system as an hour angle coordinate system, the hour angle coordinate system may include hour angle coordinates (hour angle, ha) and declination coordinates (declination, dec), wherein the hour angle coordinates can be calculated from the corresponding moment. On this basis, the mapping relationship between the imaging device and the space-time coordinate system can also be expressed as a mapping relationship from the device coordinate system of the imaging device to the hour angle coordinate system, which can be specifically the hour angle coordinates and declination coordinates of the imaging device in the hour angle coordinate system, or a conversion matrix generated based on the hour angle coordinates and declination coordinates in the hour angle coordinate system, but is not limited thereto.

[0073] As another option of the embodiment of the present application, acquiring at least one second image captured by the second shooting component includes: Controlling the exposure time of the second shooting component to be a preset exposure time, and acquiring a second image captured by the second shooting component; or Acquire a second image captured by a second photographing component, and rotate the second photographing component according to a preset second angle; The fourth image collected by the second shooting component after each rotation process is acquired, and the second image and all the fourth images are used as at least two second images.

[0074] Specifically, when acquiring the second image captured by the second shooting component, the imaging device can set the exposure time of the second shooting component to a preset exposure time. The preset exposure time can be but is not limited to a value greater than 10 seconds to ensure that a second image captured by the second shooting component has a clear and effective star point trajectory.

[0075] In addition, the imaging device can also obtain a second image captured by the second shooting component at the current attitude angle, and rotate the second shooting component according to a preset second angle, the preset second angle can be but not limited to a yaw angle rotation of 5 degrees, that is, the yaw angle of the second shooting component is rotated by 5 degrees (can be reduced by 5 degrees, or increased by 5 degrees), and after each rotation process of the second shooting component, the fourth image captured by the second shooting component can be obtained, so that all the fourth images and the second images are used as at least two second images. It can be understood that when the number of the at least two second images is two, the at least two second images may include one second image and one fourth image; when the number of the at least two second images is three, the at least two second images may include one second image and two fourth images; when the number of the at least two second images exceeds three, the at least two second images may include one second image and at least three fourth images.

[0076] As another option of the embodiment of the present application, obtaining a mapping relationship between an imaging device and a spatiotemporal coordinate system according to all second images and the acquisition time of each second image includes: Identify the second star point position from each second image, and query the space coordinates corresponding to the second star point position in a preset star database; According to the spatial coordinates corresponding to the second star point position and the acquisition time of the corresponding second image, the coordinates of the star point in the space-time coordinate system are obtained, and based on the coordinates of the star point corresponding to at least one second image in the space-time coordinate system, the mapping relationship between the imaging device and the space-time coordinate system is obtained.

[0077] Specifically, the imaging device may also extract the second star point position from each second image based on the preset second algorithm mentioned in the above embodiment, the second star point position may be understood as one or more star point pixel coordinates in the second image, and the spatial coordinates corresponding to the second star point position may be queried in the preset star database (the spatial coordinates may be understood as equatorial coordinate system coordinates, that is, equatorial coordinate system coordinates corresponding to one or more star point pixel coordinates). Here, the preset star database may be obtained by analyzing the star coordinate big data well known in the art, which includes at least two star point pixel coordinate sets, at least two star point pixel coordinates corresponding to each star point pixel coordinate set, and star point equatorial coordinate system coordinates (equatorial coordinate system coordinates may include right ascension coordinates and declination coordinates). When the star point pixel coordinates between any star point pixel coordinate set and the second star point position in any second image have a high degree of matching, it may be indicated that the equatorial coordinate system coordinates corresponding to the second star point position in the second image are consistent with the star point equatorial coordinate system coordinates corresponding to the corresponding star point pixel coordinate set.

[0078] Then, after querying the spatial coordinates corresponding to the second star point position, the imaging device can also convert the coordinates of the corresponding star point in the space-time coordinate system according to the spatial coordinates corresponding to each second image and the acquisition time (that is, the time angle coordinates and the declination coordinates in the time angle coordinate system, and the coordinates of the star point in the space-time coordinate system can be understood as the coordinates of at least two star points in the corresponding second image in the space-time coordinate system respectively), and can combine the coordinates of the star points contained in at least one second image in the space-time coordinate system to obtain the mapping relationship between the imaging device and the space-time coordinate system.

[0079] Here, when converting the coordinates of the star point in the space-time coordinate system, the local sidereal time (i.e., LST, reflecting the position of the earth's rotation relative to the stars) can be calculated according to the collection time corresponding to the space coordinates, but is not limited to it, and then the hour angle coordinate is calculated according to the local sidereal time and the right ascension coordinate in the space coordinate, so as to use the hour angle coordinate and the declination coordinate in the space coordinate as the coordinates of the corresponding star point in the space-time coordinate system (i.e., the hour angle coordinate and the declination coordinate in the time-space coordinate system). Of course, reference can also be made to the coordinate conversion means well known in the art, which will not be elaborated here.

[0080] It can be understood that, taking the number of all second images as three as an example, when obtaining the mapping relationship between the imaging device and the space-time coordinate system based on the coordinates of the star points corresponding to at least one second image in the space-time coordinate system, the imaging device can convert the coordinates of the corresponding image fixed points in the space-time coordinate system according to the pixel coordinate position relationship between the star points contained in each second image and the image fixed points, combined with the coordinates of the star points in the space-time coordinate system. The method of obtaining the pixel coordinate position relationship between the star points contained in each second image and the image fixed points may be an image processing method well known in the art, which will not be elaborated here.

[0081] Then, the imaging device may convert the coordinates of the image fixed point of each second image in the space-time coordinate system to obtain the coordinates of each image fixed point in the rectangular coordinate system, and substitute the coordinates of all the image fixed points in the rectangular coordinate system into the preset equation, and obtain the coordinates of the center of the circle in the rectangular coordinate system by solving the center of the circle between the three points. It can be understood that when converting the coordinates of the image fixed point in the space-time coordinate system into the coordinates in the rectangular coordinate system, taking the image fixed point as the image center as an example, it can be but not limited to combining the preset altitude angle calculation formula and the preset azimuth angle calculation formula, converting the coordinates of the image center in the space-time coordinate system into the coordinates in the horizontal coordinate system, and then converting the coordinates of the image center in the horizontal coordinate system into the coordinates in the rectangular coordinate system, and the coordinate conversion means are technical means well known in the art, but will not be elaborated here.

[0082] Here, the preset equation may be but is not limited to the following: In the above formula, the first two equations can be understood as the mid-perpendicular plane equations, and the last equation can be understood as the plane equation. , as well as can be respectively the coordinates of the three image centers in the rectangular coordinate system (all known parameters), can be the coordinates of the center of the circle in the rectangular coordinate system (that is, the parameter to be solved), , as well as They can be the distances from the centers of the three images to the center of the circle (the three distances are equal), and A, B, C and D can be the parameters of the plane equation (all of which can be calculated by substituting the coordinates of the centers of the three images in the rectangular coordinate system).

[0083] Next, after obtaining the coordinates of the center of the circle in the rectangular coordinate system, the imaging device may also perform a conversion process on the coordinates of the center of the circle in the rectangular coordinate system to obtain the coordinates of the center of the circle in the time-space coordinate system, that is, the coordinates of the center of the circle. Here, the conversion process may be, but is not limited to, as follows: In the above formula, ha and dec are the coordinates of the center of the circle in the time-space coordinate system (ha is the hour angle coordinate in the hour angle coordinate system, and dec is the declination coordinate in the hour angle coordinate system), that is, the hour angle coordinate and declination coordinate of the imaging device in the hour angle coordinate system. The y and x in the last formula are the coordinates of the center of the circle in the rectangular coordinate system. as well as .

[0084] Of course, after obtaining the coordinates of the center of the circle in the space-time coordinate system, the imaging device can also substitute the coordinates of the center of the circle in the space-time coordinate system into a preset rotation matrix expression, so as to use the preset rotation matrix expression after substitution as a mapping relationship between the imaging device and the space-time coordinate system. Here, the preset rotation matrix expression can be, but is not limited to, as follows: In the above formula, It can be a preset rotation matrix expression, and ha and dec can be the coordinates of the center of the circle in the space-time coordinate system.

[0085] It can also be understood that when the number of at least one second image exceeds three, taking the image fixed point as the image center as an example, the imaging device can also, but is not limited to, refer to the above embodiment, convert the coordinates of each image center in the space-time coordinate system into coordinates in the rectangular coordinate system, and use the preset center of circle equation to perform least squares solution on the coordinates of all image centers in the rectangular coordinate system, calculate the coordinates of the center of the circle corresponding to all image centers in the rectangular coordinate system, and then convert the coordinates of the center of the circle in the rectangular coordinate system into coordinates in the space-time coordinate system, and the mapping relationship between the imaging device and the space-time coordinate system can be obtained according to the coordinates of the center of the circle in the space-time coordinate system, but it will not be elaborated here.

[0086] Here, the preset circle center equation may be but is not limited to the following: In the above formula, x, y and z may be the coordinates of the center of each image in the rectangular coordinate system, h, k and l may be the coordinates of the center of the circle in the rectangular coordinate system, and r may be the distance from the center of each image to the center of the circle; by substituting the coordinates of the centers of at least three images in the rectangular coordinate system into the preset circle center equation, the coordinates of the center of the circle in the rectangular coordinate system can be solved using the least squares method.

[0087] As another option of the embodiment of the present application, after acquiring at least one second image captured by the shooting component, the method further includes: Acquire two second images, and calculate the rotation center position according to the two second images; The rotation center position is converted into coordinates in the space-time coordinate system, and according to the coordinates of the rotation center position in the space-time coordinate system, a mapping relationship between the imaging device and the space-time coordinate system is obtained.

[0088] As another option of the embodiment of the present application, acquiring two second images and calculating the rotation center position according to the two second images specifically includes: The pixel coordinates of the same points in the two second images are obtained, and the rotation center position is calculated according to the pixel coordinates of the same points.

[0089] Specifically, when the number of at least one second image is two, the imaging device can also obtain the pixel coordinates corresponding to the same point of each second image in the equatorial coordinate system (that is, the same point pixel coordinates), and use a preset numerical solution method to calculate the same point pixel coordinates to obtain the rotation center coordinates of the two second images (that is, the rotation center position), and convert the rotation center coordinates into coordinates in the space-time coordinate system, thereby obtaining the mapping relationship between the imaging device and the space-time coordinate system. Here, the processing method of the rotation center coordinates and the coordinate conversion method can refer to the technical means known in the art, but will not be elaborated here.

[0090] As another option of the embodiment of the present application, after acquiring at least one second image captured by the second shooting component, the method further includes: Obtaining the star point trajectory position in the second image, and calculating the trajectory center position according to the star point trajectory position; The center position of the trajectory is converted into coordinates in the space-time coordinate system, and the mapping relationship between the imaging device and the space-time coordinate system is obtained according to the coordinates of the center position of the trajectory in the space-time coordinate system.

[0091] Specifically, when the number of at least one second image is one, the imaging device can also identify at least three star point trajectory coordinates (i.e., star point trajectory positions) in the second image, and use the preset circle equation to perform least squares solution on all star point trajectory coordinates, calculate the center coordinates of all star point trajectories (i.e., trajectory center positions), and then convert the center coordinates into coordinates in the space-time coordinate system, and the mapping relationship between the imaging device and the space-time coordinate system can be obtained based on the coordinates of the center in the space-time coordinate system, but no further details will be given here.

[0092] Here, the preset circle equation may be but is not limited to the following: In the above formula, x and y can be the coordinates of each star point trajectory, as well as can be the coordinates of the center of the circle, and r can be the distance between each star point trajectory and the center of the circle; by substituting at least three star point trajectory coordinates into the preset circle equation, the coordinates of the center of the circle can be solved using the least squares method.

[0093] In one or more embodiments of the present application, the following effects are achieved: (1) First use the wide field of view camera to scan the starry sky area, and then use the narrow field of view camera to obtain images to complete the calibration. This not only realizes fully automated equipment calibration, but also overcomes the low efficiency of traditional manual operation. (2) The deep learning-based star segmentation algorithm can accurately identify the starry sky area in the image and avoid obstacles, overcoming the problem of calibration failure of traditional equatorial mount boxes and theodolites when there are clouds or other obstacles; (3) Scanning can be controlled by scanning path planning, which can fully cover the sky area and quickly search for starry sky areas, overcoming the shortcomings of traditional telescopes that require manual search for starry sky areas; (4) By obtaining the mapping relationship between the imaging device and the space-time coordinate system and quickly adjusting the imaging device to align with the polar axis, not only can the field rotation problem of ordinary theodolites and telescopes be solved, but the observation performance of the system can also be significantly improved.

[0094] See next Figure 5 , Figure 5 A schematic structural diagram of another imaging device provided in an embodiment of the present application is shown.

[0095] Figure 5 The imaging device shown may include at least a first shooting component and a second shooting component. In addition, the imaging device may also include an image recognition module 501, a position control module 502 and a data calculation module 503, wherein: An image recognition module 501 is used to recognize a first image containing a starry sky area in at least one first image captured by the first shooting component; A position control module 502, used to control the second shooting component to point to the starry sky area in the first image; The data calculation module 503 is used to obtain at least one second image captured by the second shooting component, and obtain a mapping relationship between the imaging device and the space-time coordinate system according to all the second images and the capture time of each second image.

[0096] In some possible embodiments, the imaging device further includes: Acquire the field of view angle and attitude angle of the first shooting component, and plan a scanning path according to the field of view angle and the attitude angle; The first shooting component is rotated according to the scanning path, and at least one first image captured by the first shooting component after the rotation processing is acquired.

[0097] In some possible embodiments, the attitude angle includes an initial pitch angle and an initial yaw angle; The scanning path is planned according to the field of view angle and attitude angle, including: When the attitude angle is consistent with the preset initial value, the preset first angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; or When the attitude angle is inconsistent with the preset initial value, the initial pitch angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; Based on a preset angle rotation relationship, a first yaw attitude angle corresponding to the initial pitch angle is calculated, and at least one first yaw angle is obtained according to the first yaw attitude angle; Obtaining a corresponding first scanning position according to the starting pitch angle and each first yaw angle; Obtaining at least one first pitch angle in sequence according to the field of view angle and the starting pitch angle, and calculating a second yaw attitude angle corresponding to each first pitch angle based on a preset angle rotation relationship; Obtain at least one second yaw angle according to each second yaw attitude angle, and obtain a second scanning position based on each first pitch angle and each corresponding second yaw angle; The starting scanning position, all the first scanning positions and all the second scanning positions are arranged in a preset order, and a scanning path is planned according to the arrangement result.

[0098] In some possible embodiments, the scanning path includes at least three target scanning positions, each target scanning position being a starting scanning position, any first scanning position or any second scanning position; Rotating the first shooting component according to the scanning path and acquiring at least one first image captured by the first shooting component after the processing, including: According to each target scanning position in the scanning path, the first shooting component is rotated in turn, and a first image captured by the first shooting component when it is rotated to each target scanning position is acquired.

[0099] In some possible embodiments, identifying a first image containing a starry sky area in at least one first image captured by the first photographing component includes: In at least one first image captured by the first shooting component, recognition processing is performed on each captured first image based on a preset first algorithm to obtain a corresponding starry sky area; When the area of ​​any starry sky region exceeds a preset area threshold, the corresponding first image is used as the first image containing the starry sky region.

[0100] In some possible embodiments, in at least one first image captured by the first photographing component, identifying a first image containing a starry sky area, further comprising: Extracting a first star point position from each starry sky region according to a preset second algorithm; The first star point position is matched based on a preset star body coordinate library, and when the first star point position is matched successfully, the corresponding first image is used as the first image containing the starry sky area.

[0101] In some possible embodiments, controlling the second photographing component to point to the starry sky area in the first image includes: Acquire a third image captured by the second shooting component, and identify a first pixel coordinate of a fixed point of the third image in the first image containing a starry sky area; Constructing a rotation matrix according to the camera parameters of the first shooting component and the first pixel coordinates, and obtaining a first coordinate position based on the rotation matrix and the first pixel coordinates; In the first image including the starry sky region, a second pixel coordinate of a fixed point in the starry sky region is identified, and a second coordinate position is obtained according to the rotation matrix and the second pixel coordinate; The difference between the first coordinate position and the second coordinate position is used as the positional relationship between the first photographing component and the second photographing component, and the second photographing component is controlled to point to the starry sky area in the first image according to the positional relationship.

[0102] In some possible embodiments, acquiring at least one second image captured by the second shooting component includes: Controlling the exposure time of the second shooting component to be a preset exposure time, and acquiring a second image captured by the second shooting component; or Acquire a second image captured by a second photographing component, and rotate the second photographing component according to a preset second angle; The fourth image collected by the second shooting component after each rotation process is acquired, and the second image and all the fourth images are used as at least two second images.

[0103] In some possible embodiments, obtaining a mapping relationship between an imaging device and a spatiotemporal coordinate system according to all second images and the acquisition time of each second image includes: Identify the second star point position from each second image, and query the space coordinates corresponding to the second star point position in a preset star database; According to the spatial coordinates corresponding to the second star point position and the acquisition time of the corresponding second image, the coordinates of the star point in the space-time coordinate system are obtained, and based on the coordinates of the star point corresponding to at least one second image in the space-time coordinate system, the mapping relationship between the imaging device and the space-time coordinate system is obtained.

[0104] In some possible embodiments, obtaining a mapping relationship between an imaging device and the space-time coordinate system based on coordinates of a star point corresponding to at least one second image in the space-time coordinate system includes: Based on the coordinates of the star point corresponding to each second image in the space-time coordinate system, determining the coordinates of the corresponding image fixed point in the space-time coordinate system; The coordinates of the center of the circle are calculated according to the coordinates of each fixed point of the image in the space-time coordinate system and the preset equation; The coordinates of the center of the circle are used as the mapping relationship between the imaging device and the space-time coordinate system; or Substitute the coordinates of the circle center into the preset rotation matrix expression to obtain the mapping relationship between the imaging device and the space-time coordinate system.

[0105] In some possible embodiments, after acquiring at least one second image captured by the second shooting component, the method further includes: Acquire two second images, and calculate the rotation center position according to the two second images; The rotation center position is converted into coordinates in the space-time coordinate system, and according to the coordinates of the rotation center position in the space-time coordinate system, a mapping relationship between the imaging device and the space-time coordinate system is obtained.

[0106] In some possible embodiments, acquiring two second images and calculating the rotation center position according to the two second images specifically includes: The pixel coordinates of the same points in the two second images are obtained, and the rotation center position is calculated according to the pixel coordinates of the same points.

[0107] In some possible embodiments, after acquiring at least one second image captured by the second shooting component, the method further includes: Obtaining the star point trajectory position in the second image, and calculating the trajectory center position according to the star point trajectory position; The center position of the trajectory is converted into coordinates in the space-time coordinate system, and the mapping relationship between the imaging device and the space-time coordinate system is obtained according to the coordinates of the center position of the trajectory in the space-time coordinate system.

[0108] In some possible embodiments, the camera type of the first shooting component is a wide-angle camera, and the camera type of the second shooting component is a telephoto camera.

[0109] See next Figure 6 , Figure 6 A schematic structural diagram of another imaging device provided in an embodiment of the present application is shown.

[0110] like Figure 6 As shown, the imaging device 600 may include a processor 601 , a network interface 604 , a user interface 603 , a memory 605 , a communication bus 602 , a first photographing component 606 , and a second photographing component 607 .

[0111] The communication bus 602 may be used to realize the connection and communication among the above-mentioned components.

[0112] The user interface 603 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0113] The network interface 604 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0114] The first shooting component 606 may be, but is not limited to, a rotatable component including a wide-angle camera, the second shooting component 607 may be, but is not limited to, a rotatable component including a telephoto camera, and the two rotatable components may be, but are not limited to, coaxially rotating.

[0115] Among them, the processor 601 may include one or more processing cores. The processor 601 uses various interfaces and lines to connect various parts in the imaging device 600, and executes various functions and processes data of the routing imaging device 600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 605, and calling data stored in the memory 605. Optionally, the processor 601 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 601 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 601, and it can be implemented separately through a chip.

[0116] The memory 605 may include a RAM or a ROM. Optionally, the memory 605 includes a non-transitory computer-readable medium. The memory 605 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may optionally be at least one storage device located away from the aforementioned processor 601. As Figure 6 As shown, the memory 605 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a dual-camera based calibration application.

[0117] Specifically, the processor 601 may be used to call a dual-camera based calibration application stored in the memory 605, and specifically perform the following operations: In at least one first image captured by the first photographing component, a first image including a starry sky area is identified; Controlling the second shooting component to point to the starry sky area in the first image; At least one second image captured by the second shooting component is acquired, and a mapping relationship between the imaging device and the space-time coordinate system is obtained according to all the second images and the acquisition time of each second image.

[0118] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0119] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0120] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

Claims

1. A dual-camera calibration method, characterized in that: The method is applied to an imaging device, the imaging device comprising a first shooting component and a second shooting component, and the method comprises: In at least one first image captured by the first photographing component, identifying the first image containing a starry sky area; Controlling the second photographing component to point to the starry sky area in the first image; At least one second image captured by the second shooting component is acquired, and a mapping relationship between the imaging device and the space-time coordinate system is obtained according to all the second images and the acquisition time of each second image.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the field of view angle and the attitude angle of the first shooting component, and plan a scanning path according to the field of view angle and the attitude angle; The first shooting component is rotated according to the scanning path, and at least one first image captured by the first shooting component after the rotation processing is acquired.

3. The method according to claim 2, characterized in that The attitude angle includes an initial pitch angle and an initial yaw angle; The step of planning a scanning path according to the field of view angle and the attitude angle includes: When the attitude angle is consistent with the preset initial value, the preset first angle is used as the starting pitch angle, and the starting scanning position is obtained according to the starting pitch angle and the initial yaw angle; or When the attitude angle is inconsistent with the preset initial value, taking the initial pitch angle as the starting pitch angle, and obtaining a starting scanning position according to the starting pitch angle and the initial yaw angle; Based on a preset angle rotation relationship, calculating a first yaw attitude angle corresponding to the initial pitch angle, and obtaining at least one first yaw angle according to the first yaw attitude angle; Obtaining a corresponding first scanning position according to the starting pitch angle and each of the first yaw angles; Obtaining at least one first pitch angle in sequence according to the field of view angle and the starting pitch angle, and calculating a second yaw attitude angle corresponding to each first pitch angle based on the preset angle rotation relationship; Obtain at least one second yaw angle according to each of the second yaw attitude angles, and obtain a second scanning position based on each of the first pitch angles and each of the corresponding second yaw angles; The starting scanning position, all the first scanning positions and all the second scanning positions are arranged in a preset order, and a scanning path is planned according to the arrangement result.

4. The method according to claim 3, characterized in that The scanning path includes at least three target scanning positions, each of which is the starting scanning position, any one of the first scanning positions, or any one of the second scanning positions; The rotating process of the first photographing component according to the scanning path and acquiring at least one first image captured by the first photographing component after the processing includes: According to each of the target scanning positions in the scanning path, the first photographing component is rotated in sequence, and a first image captured by the first photographing component when it is rotated to each of the target scanning positions is acquired.

5. The method according to any one of claims 1 to 4, characterized in that: The step of identifying the first image containing a starry sky area in at least one first image captured by the first shooting component comprises: In at least one first image captured by the first shooting component, recognition processing is performed on each of the first images captured based on a preset first algorithm to obtain a corresponding starry sky area; When the area of ​​any one of the starry sky regions exceeds a preset area threshold, the corresponding first image is used as the first image containing the starry sky region.

6. The method according to claim 5, characterized in that The step of identifying the first image containing a starry sky area in at least one first image captured by the first photographing component further includes: Extracting a first star point position from each of the starry sky regions according to a preset second algorithm; The first star point position is matched based on a preset star coordinate library, and when the first star point position is matched successfully, the corresponding first image is used as the first image containing the starry sky area.

7. The method according to claim 1, characterized in that The controlling the second photographing component to point to the starry sky area in the first image comprises: Acquire a third image captured by the second photographing component, and identify a first pixel coordinate of a fixed point of the third image in the first image containing a starry sky area; constructing a rotation matrix according to the camera parameters of the first shooting component and the first pixel coordinates, and obtaining a first coordinate position based on the rotation matrix and the first pixel coordinates; In the first image including the starry sky region, identifying the second pixel coordinates of the fixed point of the starry sky region, and obtaining the second coordinate position according to the rotation matrix and the second pixel coordinates; The difference between the first coordinate position and the second coordinate position is used as the positional relationship between the first shooting component and the second shooting component, and the second shooting component is controlled to point to the starry sky area in the first image according to the positional relationship.

8. The method according to claim 1, characterized in that: The acquiring of at least one second image captured by the second shooting component comprises: Controlling the exposure time of the second shooting component to be a preset exposure time, and acquiring a second image captured by the second shooting component; or Acquire a second image captured by the second photographing component, and rotate the second photographing component according to a preset second angle; Acquire the fourth image collected by the second shooting component after each rotation process, and use the second image and all the fourth images as at least two second images.

9. The method according to claim 1, characterized in that: The obtaining, according to all the second images and the acquisition time of each second image, a mapping relationship between the imaging device and the spatiotemporal coordinate system comprises: Identifying a second star point position from each of the second images, and querying a preset star database for the space coordinates corresponding to the second star point position; According to the spatial coordinates corresponding to the position of the second star point and the acquisition time of the corresponding second image, the coordinates of the star point in the space-time coordinate system are obtained, and based on the coordinates of the star point in the space-time coordinate system corresponding to at least one of the second images, the mapping relationship between the imaging device and the space-time coordinate system is obtained.

10. The method according to claim 9, characterized in that The obtaining, based on the coordinates of the star point corresponding to at least one of the second images in the space-time coordinate system, a mapping relationship between the imaging device and the space-time coordinate system comprises: Based on the coordinates of the star point corresponding to each second image in the space-time coordinate system, determining the coordinates of the corresponding image fixed point in the space-time coordinate system; Calculate the coordinates of the center of the circle according to the coordinates of each fixed point of the image in the space-time coordinate system and a preset equation; Using the circle center coordinates as the mapping relationship between the imaging device and the space-time coordinate system; or Substituting the coordinates of the circle center into a preset rotation matrix expression, a mapping relationship between the imaging device and the space-time coordinate system is obtained.

11. The method according to claim 1, characterized in that: After acquiring at least one second image captured by the second shooting component, the method further includes: Acquire two second images, and calculate the rotation center position according to the two second images; The rotation center position is converted into coordinates in a space-time coordinate system, and a mapping relationship between the imaging device and the space-time coordinate system is obtained according to the coordinates of the rotation center position in the space-time coordinate system.

12. The method according to claim 11, characterized in that Acquiring two second images and calculating the rotation center position according to the two second images specifically includes: The pixel coordinates of the same points in the two second images are obtained, and the rotation center position is calculated according to the pixel coordinates of the same points.

13. The method according to claim 1, characterized in that After acquiring at least one second image captured by the second shooting component, the method further includes: Acquire the star point trajectory position in the second image, and calculate the trajectory center position according to the star point trajectory position; The center position of the trajectory is converted into coordinates in a space-time coordinate system, and a mapping relationship between the imaging device and the space-time coordinate system is obtained according to the coordinates of the center position of the trajectory in the space-time coordinate system.

14. The method according to claim 1, characterized in that The camera type of the first shooting component is a wide-angle camera, and the camera type of the second shooting component is a telephoto camera.

15. An imaging device, characterized in that: The imaging device comprises a first shooting component and a second shooting component, and the imaging device further comprises: An image recognition module, configured to recognize, from the at least one first image captured by the first shooting component, the first image containing a starry sky area; A position control module, used for controlling the second shooting component to point to the starry sky area in the first image; A data calculation module is used to obtain at least one second image captured by the second shooting component, and obtain a mapping relationship between the imaging device and the space-time coordinate system based on all the second images and the capture time of each second image.

16. An imaging device, characterized in that: including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 14.

Images