Information acquisition system, and calibration method, device and computer readable storage medium thereof

Abstract

The application provides an information collection system and a calibration method, device and readable storage medium thereof. A display interface includes at least three non-collinear and non-overlapping display areas. A display device is controlled to sequentially display a plurality of target images (102) on the display interface. The sequence of pixel values displayed by any one of the at least three display areas is different from the sequence of pixel values displayed by any other display area. During the sequential display of the plurality of target images, a shooting device is controlled to shoot the display interface, and a plurality of images (104) are sequentially recorded. The pixel values of the same pixel position in the plurality of images are obtained, and the sequence of pixel values of the pixel position is generated (106). Based on the sequence of pixel values of the pixel position of the recorded plurality of images and the sequence of pixel values of the at least three display areas, the correspondence between the pixel position of the image and the pose of the display area of the shooting device is determined (108).

Description

Technical Field

[0001] This application relates to the field of information processing technology, and more specifically, to an information acquisition system and its calibration method, apparatus and computer-readable storage medium. Background Technology

[0002] In technical fields such as image measurement or machine vision, the calibration of the imaging equipment is a crucial step, and the accuracy of the calibration results directly affects the subsequent operation of the imaging equipment. Summary of the Invention

[0003] In view of this, embodiments of this application provide an information acquisition system and its calibration method, apparatus and computer-readable storage medium to solve the problem of cumbersome and complicated calibration process of information acquisition systems in related technologies.

[0004] In a first aspect, a calibration method for an information acquisition system is provided, the information acquisition system including a display device and a shooting device;

[0005] The shooting device faces the display interface of the display device; the display interface includes at least three display areas, which are non-collinearly arranged and do not overlap with each other;

[0006] The method includes:

[0007] The display device is controlled to sequentially display multiple target images on the display interface. During the sequential display of the target images, the sequence of pixel values ​​displayed sequentially in any one of the at least three display areas is different from the sequence of pixel values ​​displayed sequentially in any other display area of ​​the at least three display areas.

[0008] During the process of sequentially displaying multiple target images, the capturing device is controlled to capture images of the display interface, and multiple images are sequentially recorded.

[0009] Obtain pixel values ​​at the same pixel location from multiple images, and generate a sequence of pixel values ​​at that pixel location;

[0010] Based on the sequence of pixel values ​​at the pixel positions of multiple recorded images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

[0011] Secondly, a calibration method for an information acquisition system is provided, the system including a display device and a shooting device;

[0012] The shooting device faces the display interface of the display device; the display interface includes at least M display areas, which are non-collinearly arranged and do not overlap with each other;

[0013] The method includes:

[0014] The display device is controlled to display i target images on the display interface. During the display of the target images, the sequence of pixel values ​​displayed in any display area of ​​the M display areas is different from the sequence of pixel values ​​displayed in any other display area of ​​the at least M display areas. The number of types of pixel values ​​is N, where N raised to the power of i is greater than or equal to M, and i is greater than or equal to 1.

[0015] During the display of i target images, the camera is controlled to capture images of the display interface, and i images are recorded.

[0016] Based on the sequence of pixel values ​​at each pixel position in the i images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

[0017] Thirdly, a calibration device for an information acquisition system is provided, the information acquisition system including a display device and a shooting device;

[0018] The calibration device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor. When the processor executes the computer program, it implements the steps of the calibration method described in the first aspect.

[0019] Fourthly, a calibration device for an information acquisition system is provided, the information acquisition system including a display device and a shooting device; the shooting device faces the display interface of the display device;

[0020] The calibration device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor. When the processor executes the computer program, it implements the steps of the calibration method described in the second aspect.

[0021] Fifthly, an information acquisition system is provided, the information acquisition system comprising: a shooting device and a display device; and the calibration device described in the third aspect and / or the calibration device described in the fourth aspect.

[0022] In a sixth aspect, a computer-readable storage medium is provided, wherein a plurality of computer instructions are stored thereon, and when executed, the computer instructions implement the steps of the calibration method described in the first aspect.

[0023] In a seventh aspect, a computer-readable storage medium is provided, wherein a plurality of computer instructions are stored thereon, and when executed, the computer instructions implement the steps of the calibration method described in the second aspect.

[0024] Using the solution provided in this application, the display interface of the display device includes at least three non-collinear and non-overlapping display areas. By displaying multiple target images on the display device, the sequence of pixel values ​​displayed sequentially in the display areas can be determined. The imaging device can capture multiple images of the display interface, and the sequence of pixel values ​​at each pixel position can be obtained from these multiple images. Based on this, if the sequence of pixel values ​​at the pixel position of the image captured by the imaging device is the same as the sequence of pixel values ​​of the display area, the pixel position of the image captured by the imaging device corresponds to the position of the display area on the display interface, thereby determining the pose correspondence between the two. This embodiment can directly determine the pose correspondence between the imaging device and the display device, and the calibration process is very flexible and does not require many restrictions. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1A This is a schematic diagram of a calibration scenario for an information collection system in one embodiment of this application.

[0027] Figure 1B This is a flowchart of a calibration method for an information acquisition system according to an embodiment of this application.

[0028] Figure 1C This is a schematic diagram of a three-dimensional Gray code image according to an embodiment of this application.

[0029] Figure 1D This is a schematic diagram of a horizontally arranged striped image according to an embodiment of this application.

[0030] Figure 1E This is a schematic diagram of a pair of positive and negative target images according to an embodiment of this application.

[0031] Figure 2A This is a schematic diagram of an information collection system according to an embodiment of this application.

[0032] Figure 2B This is a schematic diagram of a five-bit Gray code image according to an embodiment of this application.

[0033] Figure 2C This is a schematic diagram of a pair of two-bit forward and inverse Gray code images according to an embodiment of this application.

[0034] Figure 2DThis is an image captured by a camera according to one embodiment of this application, and the binarized Gray code image of that image after processing.

[0035] Figure 2E This is a schematic diagram of the original image captured by the camera according to an embodiment of this application, and the original image after correction.

[0036] Figure 3 This is a flowchart of a calibration method for an information acquisition system according to another embodiment of this application.

[0037] Figure 4 This is a schematic diagram of a calibration device for an information acquisition system according to an embodiment of this application.

[0038] Figure 5 This is a schematic diagram of a calibration device for an information acquisition system according to another embodiment of this application.

[0039] Figure 6 This is a schematic diagram of an information collection system according to an embodiment of this application. Detailed Implementation

[0040] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0041] In technical fields involving image measurement or machine vision, camera calibration is typically required. Cameras consist of optical components and imaging sensors. Distortions may occur in the optical components during manufacturing and assembly; camera calibration serves two purposes. Firstly, it corrects these distortions. Secondly, in information acquisition systems incorporating camera equipment, determining the 3D geometric position of a point on the surface of a spatial object and its corresponding point in the image generated by the imaging sensor requires establishing a geometric model of the camera's imaging capabilities. This model is then used to reconstruct a 3D scene from the captured image, enabling applications such as monocular or binocular ranging, 3D reconstruction, SLAM, and AR. The parameters of this geometric model (including intrinsic and extrinsic parameters) constitute the parameters of the camera equipment, and solving for these parameters is the process of camera calibration. Camera calibration is crucial; the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera during operation. Therefore, proper camera calibration is a prerequisite for successful subsequent work.

[0042] Taking Hardware-in-the-Loop (HIL) testing as an example, it is a development and testing technique for complex device controllers. Through HIL testing, the physical components of a machine or system are replaced by a simulation simulator, and it is widely used in the development of automotive controllers. In the development of ECUs (Electronic Control Units), system software and mechanical hardware structures are typically designed in parallel. Testing can only be carried out after integration. If serious security vulnerabilities are discovered after integration, it could lead to personal injury, equipment damage, and project delays. Hardware-in-the-Loop testing has become a crucial part of the development process, reducing the number of real-vehicle road tests, shortening development time, reducing costs, improving the quality of autonomous driving software, and reducing risks for automakers.

[0043] Existing vehicles are typically equipped with one or more cameras to observe the surrounding environment and provide the data to autonomous driving software for decision-making. Therefore, hardware-in-the-loop (HIL) testing scenarios often employ HIL test equipment to test autonomous driving software. This HIL test equipment includes a processor to run the autonomous driving software; to simulate the vehicle's driving process, it also includes cameras to mimic those installed in actual vehicles; and it further includes display recognition, where images are displayed to simulate the vehicle's environment. The cameras collect data from the images displayed on the screen, simulating data observed by the vehicle in a real-world scenario. Therefore, the images collected by the cameras can be used to test the autonomous driving software.

[0044] However, before testing autonomous driving software, it is necessary to calibrate the pose correspondence between the shooting device and the display device in order to process the raw images captured by the shooting device. Traditional calibration methods typically employ the aforementioned method of solving for the intrinsic and extrinsic parameters of the shooting device. Taking a binocular camera as an example, extrinsic parameter calibration calculates the displacement and rotation parameters between the two cameras, which are the external parameters. Using these intrinsic and extrinsic parameters, distortion correction can be further performed on any two different cameras, and the relative position and pose of the two cameras can be determined and horizontal alignment correction performed. After calibration, the principle of parallax geometry can be used to measure the distance to points in space using the intrinsic and extrinsic parameters. Uncalibrated cameras exhibit significant distortion and misalignment between frames, making it impossible to test autonomous driving software.

[0045] Traditional methods for calibrating imaging devices in hardware-in-the-loop testing scenarios typically rely on specialized external hardware preparation (such as checkerboard patterns) and require personnel familiar with calibration to manually move markers to complete the calibration process, making them highly manual. Furthermore, the varying calibration skills of different individuals lead to inconsistent calibration accuracy. Calibrating extrinsic parameters, on the other hand, requires multiple imaging devices to share a common field of view.

[0046] For example, in a checkerboard calibration scheme, a checkerboard in a known world coordinate system needs to be prepared beforehand as a calibration tool, and the three-dimensional coordinates of each feature point of the checkerboard in the world coordinate system need to be recorded. The checkerboard needs to be manually moved to cover the entire field of view and photographed (generally 30 to 120 images need to be collected). The feature points of the checkerboard need to be extracted and matched with the three-dimensional coordinates. Further optimization and iterative fitting of the optimal intrinsic parameters are required, and the extrinsic parameters are calibrated by using the landmark information in the shared field of view of a binocular camera.

[0047] Traditional calibration methods are inflexible, generally requiring pre-prepared calibration materials (such as a checkerboard pattern), and these materials must be manufactured with extremely high precision (above 10,000 dpi). Low-precision materials will result in inaccurate or unusable calibration parameters. Furthermore, calibration can only be performed by moving the calibration board in front of the camera, and the movement trajectory must cover the entire field of view, limiting its flexibility and making it impossible to conduct calibration in other locations.

[0048] Traditional binocular cameras are typically fixed to a rigid object (with a given baseline length) to allow both cameras to share a field of view. Without this shared field of view, extrinsic parameters cannot be calibrated. Furthermore, the relative pose of the camera and display device remains constant (any movement requires recalibration of extrinsic parameters), which severely hinders the flexibility of hardware-in-the-loop (HIL) test equipment design. Moreover, HIL test equipment simulates real-world scenarios to test autonomous driving algorithms, and camera calibration often requires engineers with relevant camera calibration expertise.

[0049] Based on this, this application provides a calibration method for an information acquisition system. Unlike the aforementioned methods that calibrate the intrinsic and extrinsic parameters of the shooting device, this embodiment can directly determine the positional correspondence between the shooting device and the display device, and the calibration process is very flexible and requires no significant restrictions. The following is a detailed description of this embodiment.

[0050] The solution in this application embodiment can be used to calibrate an information acquisition system; such as Figure 1A The diagram shown is a calibration scenario of an information collection system according to one embodiment of this application. Figure 1AThe system includes an information acquisition system 100, which includes a display device 110 and a shooting device 120. The shooting device 120 faces the display interface of the display device 110. The display interface includes at least three display areas, which are non-collinearly arranged and do not overlap with each other.

[0051] like Figure 1B The diagram shown is a flowchart illustrating a calibration method for an information acquisition system according to an exemplary embodiment of this application, comprising the following steps:

[0052] In step 102, the display device is controlled to sequentially display multiple target images on the display interface; wherein, during the sequential display of the target images, the sequence of pixel values ​​displayed sequentially in any one of the at least three display areas is different from the sequence of pixel values ​​displayed sequentially in any other display area of ​​the at least three display areas;

[0053] In step 104, during the process of sequentially displaying multiple target images, the capturing device is controlled to capture images of the display interface, and multiple images are sequentially recorded;

[0054] In step 106, pixel values ​​at the same pixel location in multiple images are obtained, and a sequence of pixel values ​​at the pixel location is generated;

[0055] In step 108, based on the sequence of pixel values ​​of the pixel positions of the recorded multiple images and the sequence of pixel values ​​of the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the pose of the display areas is determined.

[0056] This embodiment is used to determine the pose correspondence between a shooting device and a display device in an information acquisition system. In some examples, there can be one or more shooting devices, and the display devices correspond to the shooting devices, that is, one display device corresponds to one shooting device, and each shooting device faces the display interface of a display device. Therefore, there can also be one or more display devices. As an example, there can be at least two shooting devices, which can meet the calibration scenarios of most binocular or multi-view cameras. Figure 1A For ease of illustration, the diagram shows an example of a display device and a camera device. In practical applications, the number of shooting devices and display devices can be configured as needed. Based on this, the solution of this embodiment can be applied to a variety of business scenarios to meet user needs.

[0057] The information acquisition system in this embodiment can be implemented in different ways under different application scenarios. For example, in a hardware-in-the-loop testing scenario, the information acquisition system may include hardware-in-the-loop testing equipment. In practical applications, depending on different business needs, the information acquisition system may include other hardware; this embodiment does not limit this.

[0058] In this embodiment, the placement of the shooting device and the display device is not limited. For example, the shooting device and the display device can be placed horizontally, as long as the shooting device faces the display interface of the corresponding display device so that the shooting device can capture the complete display interface of the display device. Optionally, the shooting screen of the shooting device can be basically overlapped with the display interface of the display device so that the shooting device will not capture any screen other than the display interface of the display device, thereby reducing external interference and improving the calibration accuracy.

[0059] In some examples, the information acquisition system includes at least one mounting component, each of which is used to mount a shooting device; for example, the mounting component may be a bracket or a gimbal, etc., and this embodiment is not limited thereto. Based on this, any shooting device can be mounted on the mounting component as needed.

[0060] In some examples, the mounting component includes a movable mounting component, the movement of which adjusts the distance between the shooting device and the display device. The movable mounting component can be implemented in various ways. For example, it may include a guide rail and a bracket, the bracket for mounting the shooting device, and the bracket being movable along the guide rail; wherein there may be one or more guide rails, enabling the bracket to move in one or more directions. Alternatively, the mounting component may also be a bracket with a telescopic rod, the height of which is adjusted to change the distance between the shooting device and the display device.

[0061] In the case of two or more shooting devices, this embodiment does not limit the positional relationship of each shooting device. For example, taking a stereo camera as an example, the stereo cameras can be arranged horizontally, or they can be placed back to back, etc.

[0062] To determine the pose correspondence between the shooting device and the display device, in this embodiment, if at least three non-collinear and non-overlapping regions are identified on the display interface of the display device, and the positions of these at least three regions on the display interface are known, then by determining the imaging positions of these at least three regions on the shooting device, the pose correspondence between the shooting device and the display device can be determined by the positions of these at least three regions on the display interface of the display device and on the imaging of the shooting device, respectively. Based on this, this embodiment uses pixel value sequences to determine the imaging positions of at least three non-collinear and non-overlapping regions on the display interface on the shooting device.

[0063] The display interface can display multiple target images. During the sequential display of the target images, the sequence of pixel values ​​displayed sequentially in any one of the at least three display areas differs from the sequence of pixel values ​​displayed sequentially in any other display area of ​​the at least three display areas. Since the capturing device is directed towards the display device, it can capture images of the display interface during the sequential display of the multiple target images, thus recording multiple images sequentially. Because the sequence of pixel values ​​at each pixel position in the images can be obtained from these multiple images, the correspondence between the pixel positions in the images captured by the capturing device and the pose of the display areas can be determined.

[0064] In some examples, at least three non-collinear and non-overlapping regions can be flexibly determined in the display interface as needed. Optionally, the number of display regions can be three or more, and the specific number can be flexibly configured as needed. For example, there can be three display regions that are non-collinear and non-overlapping. For more than three display regions, this includes three non-collinear display regions, and each display region is non-overlapping. In some examples, the number of display regions can be set according to the actual application scenario. For example, in some calibration scenarios, the number of display regions can be three. By using the sequence of pixel values ​​displayed sequentially in the three display regions, and the sequence of pixel values ​​at each pixel position in the image, the correspondence between the shooting device and the display interface can be determined. In other examples, the number of display regions can be larger and evenly distributed across the display interface. Based on this, the correspondence between the shooting device and the display interface can more accurately solve the distortion problem of the shooting device. In some examples, the number of display regions is positively correlated with calibration accuracy; for example, the more display regions there are, the higher the calibration accuracy.

[0065] In some examples, the number of the aforementioned at least three display areas can be set by the user. For instance, the information collection system can provide user interaction functions, such as a user interface that allows the user to input the number of display areas. Alternatively, the information collection system can communicate with other devices. These other devices in this embodiment can include computers, smartphones, tablets, and other electronic devices capable of communicating with the information collection system. The user can set the number of display areas through other devices, which then send a message indicating the number of display areas to the information collection system. The information collection system determines the number of display areas by receiving the message from the other devices.

[0066] In some examples, users can set the number of display areas. In other examples, users can set the number of horizontal and vertical areas, and the product of the two is the number of display areas; for example, 8 for the horizontal area and 12 for the vertical area, the product of the two is 96, which represents the number of display areas.

[0067] In some examples, the positions of the various display areas on the display interface can be implemented in multiple ways as needed. For example, the positions of each display area on the display interface can be automatically determined based on the number of display areas. The determination strategy can be varied, such as a uniform distribution strategy on the display interface, or a strategy based on the center and edge positions of the display interface, for example, a higher and denser distribution at the edge positions. In other examples, the positions can also be set by the user; as mentioned above, user settings can be provided by the information collection system with user interaction functions for user operation, or the user can set the positions through other devices communicating with the information collection system.

[0068] In some cases, multiple target images can be prepared in advance as needed. These multiple target images are displayed sequentially by the display device, and the sequence of pixel values ​​displayed sequentially in each display area is different. The sequence corresponding to each display area refers to the sequence formed by the pixel values ​​displayed sequentially in that display area. For example, if the display device displays three target images sequentially, the pixel values ​​of the three target images in display area 1 are A, B, and C, respectively. That is, the sequence of pixel values ​​displayed sequentially in display area 1 is ABC.

[0069] This embodiment does not limit the pixel values ​​of pixels in the target image, and can be flexibly configured as needed in practical applications. For example, the color of a pixel can be represented using three values: red, green, and blue (RGB). In this embodiment, the pixel values ​​can include the values ​​of the RGB three channels. In other examples, pixel values ​​can also include grayscale values, or pixel values ​​can be represented using other color spaces; this embodiment does not limit these.

[0070] In this embodiment, there are at least two types of pixel values. In practical applications, the configuration can be adjusted as needed, as long as different pixel values ​​can be identified and distinguished from the images captured by the imaging device. For example, there can be two types, such as black and white; or there can be three types, such as red, green and blue, etc. Other colors are also optional; of course, more different types of pixel values ​​are also optional.

[0071] In other examples, to improve subsequent computational efficiency and image recognition accuracy, the pixel values ​​in this embodiment can be of two types, such as black and white. Since black and white pixel values ​​differ significantly, for example, if pixel values ​​are represented using grayscale values, black and white can be identified relatively accurately. Therefore, in this embodiment, when multiple target images are displayed sequentially on the display interface, the sequence of pixel values ​​displayed sequentially in the display area can be any combination of black and / or white, such as all black, all white, or any combination of black and white.

[0072] like Figure 1A As shown, the display device can sequentially display multiple target images on the display interface. In this embodiment, four images GC1 to GC4 are used as an example. During the sequential display of multiple target images, the shooting device is controlled to capture images of the display interface and sequentially record multiple images. Based on the sequence of pixel values ​​at the same pixel position in the multiple images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel position of the image captured by the shooting device and the pose of the display area can be determined.

[0073] In some cases, where there are two or more shooting and display devices, at least two of these display devices have the same display process, meaning they can display the same multiple target images in the same order. In other cases, it is optional for the display processes of these at least two display devices to be different; that is, they can display the same multiple target images in different orders, or they can display different target images, and so on.

[0074] In this embodiment, the sequence of pixel values ​​displayed sequentially in the display area and the sequence of pixel values ​​at each pixel position can also be obtained through multiple images. Based on this, if the sequence of pixel values ​​at the pixel position of the image from the shooting device is the same as the sequence of pixel values ​​in the display area, the pixel position of the image from the shooting device and the position of the display area on the display interface can be matched, and thus the pose correspondence between the two can be determined.

[0075] In some examples, each pixel value corresponds to a preset code, and the sequence of pixel values ​​is a coded sequence of pixel values. In this embodiment, a corresponding code can be configured for each pixel value as needed. The sequence of pixel values ​​displayed sequentially in each display area can be converted into a coded sequence, and the sequence of pixel values ​​at pixel positions in the image can also be converted into a coded sequence. Therefore, using the coded sequence allows for faster sequence comparison.

[0076] In some examples, the position information of each display area on the display interface can be pre-recorded. Thus, after determining the sequence that is the same as the sequence of pixel values ​​of the display area in the sequence of pixel values ​​of the pixel position of the image captured by the shooting device, the pixel position of the image captured by the shooting device can be matched with the position of the display area on the display interface according to the recorded position information of the display area on the display interface, thereby determining the pose correspondence between the two.

[0077] In other examples, to reduce storage space usage and improve processing efficiency, the display area sequentially displays a sequence of pixel values ​​to represent its position. This allows for rapid determination of the display area's location on the display interface based on the pixel value sequence during subsequent processing. For instance, assuming the display area's location on the display interface is at coordinates (3, 3), continuing the previous example, the sequentially displayed pixel value sequence ABC represents the coordinate information (3, 3). That is, sequence ABC not only distinguishes the various display areas of the display interface for subsequent processing to determine which pixel position on the imaging device corresponds to that display area, but also provides the coordinate information of the display area. In practical applications, the sequential display of pixel values ​​to represent the display area's position can be implemented in various ways as needed. For example, a predefined encoding method can be used to encode the display area's location on the display interface, ensuring that the location information corresponds to the pixel value sequence. The specific encoding method can be configured as needed in practical applications, and this embodiment does not limit this. Optionally, it could be the decoding result of the encoded sequence of pixel values ​​displayed sequentially in the display area, or the position information of the display area on the display interface.

[0078] In some examples, binary encoding can be used, and the pixel value encoding sequence can include a binary encoding sequence. Binary encoding is faster to compute, improving calibration efficiency. Furthermore, based on binary encoding, pixel values ​​in the target image can be either black or white, making pixel value recognition faster and more accurate. For example, black can be encoded as "0" and white as "1," or conversely, black can be encoded as "1" and white as "0." In practical applications, other encoding values ​​can also be used as needed; this embodiment does not limit this. Optionally, the binary encoding includes any of the following: Gray code, inverse Gray code, ordinary binary code, 8421 code, or 54221 code, etc.

[0079] Taking Gray code as an example, the multiple target images displayed by the display device are images encoded using Gray code, such as... Figure 1C As shown, this embodiment uses three-dimensional Gray code as an example, involving three Gray code images. For example, these three Gray code images use vertically oriented stripes. These three images use two shades of gray (white and black) to divide the display interface into eight vertically arranged striped areas, where white areas correspond to the code "1" and black areas correspond to the code "0". The display interface displays these three images sequentially. The encoding sequence of the pixel values ​​of any point on the display interface in the three images is the encoding sequence of the region where that point is located. For example, if a point P on the display interface of the display device is located in region number 3, and its encoding values ​​in the three images are "0", "1", and "1" respectively, then the region encoding value of that point is "011", which can be decoded into 3 according to Gray code, representing the location information of the region where that point is located.

[0080] In some examples, the display interface of the display device is a plane, and the position information of the display area includes: horizontal position information and vertical position information. Since the position information of the display area is two-dimensional, the position information of the display area can be encoded in the horizontal and vertical directions respectively. Based on this, the multiple target images include: a vertically arranged striped image for encoding the horizontal position information; and a horizontally arranged striped image for encoding the vertical position information.

[0081] like Figure 1C As shown, this is a vertically arranged stripe image. As described in the previous embodiment, the vertically arranged stripe image can divide the display interface into multiple vertically arranged stripe regions. Figure 1C The image shows eight vertically arranged strip-shaped areas, indicating that there are eight horizontally positioned information points on the display interface. For example... Figure 1D As shown, this is a horizontally arranged stripe image. A horizontally arranged stripe image can divide the display interface into multiple horizontally arranged stripe regions. Figure 1D The image shows eight horizontally arranged stripe regions, representing eight vertical positional information points on the display interface. Therefore, using these two different types of stripe images, the display interface can be divided into 64 display areas (8x8 grids), and the positional information of each display area can be encoded.

[0082] In some examples, the number of vertically arranged stripe images is determined based on the number of display areas to be encoded in the horizontal direction. For instance, the number of types of pixel values ​​in the target image and the number of display areas to be encoded in the horizontal direction can be combined to determine the number of vertically arranged stripe images. For example, taking N types of pixel values ​​as an example, using binary encoding, the following relationship can be used: N raised to the power of k is greater than or equal to the number of display areas to be encoded in the horizontal direction, where k is the number of vertically arranged stripe images. Taking black and white pixel values ​​as an example, using binary encoding, the following relationship can be used: 2 raised to the power of k is greater than or equal to the number of display areas to be encoded in the horizontal direction, where k is the number of vertically arranged stripe images.

[0083] In some examples, the number of horizontally arranged stripe images is determined based on the number of display areas to be encoded in the vertical direction. For example, the number of pixel values ​​in the target image and the number of display areas to be encoded in the vertical direction can be combined to determine the number of horizontally arranged stripe images. For instance, taking N pixel values ​​as an example, using binary encoding, the following relationship can be used: N raised to the power of j is greater than or equal to the number of display areas to be encoded in the vertical direction, where j is the number of horizontally arranged stripe images. Taking black and white pixel values ​​as an example, using binary encoding, the following relationship can be used: 2 raised to the power of j is greater than or equal to the number of display areas to be encoded in the vertical direction, where j is the number of horizontally arranged stripe images.

[0084] In this embodiment, during the sequential display of multiple target images, the capturing device captures images of the display interface, thereby sequentially recording multiple images. Further, it is necessary to obtain the pixel values ​​at the same pixel location in multiple images to generate a sequence of pixel values ​​at that location. However, due to the influence of ambient brightness and the exposure level of the capturing device, the acquisition of pixel values ​​in the images may be inaccurate. For example, a black pixel value displayed by the display device may appear whiter in the image due to factors such as high ambient brightness or excessive exposure, leading to lower accuracy in subsequent processing.

[0085] Based on this, the multiple target images include multiple pairs of positive target images and reverse target images, where the pixel values ​​at the same pixel position differ between each pair of positive and reverse target images. Therefore, the image obtained by the imaging device from the display interface includes both positive and reverse target image imaging. Since the pixel values ​​at the same position are different in each pair of positive and reverse target images, the pixel value of a pixel in the image can be determined by comparing the pixel values ​​at the same pixel position in the positive and reverse target image imaging.

[0086] Optionally, the pixel values ​​at the same pixel location in each pair of positive and negative target images can be different. This can be achieved in various ways as needed. For example, two significantly different pixel values ​​can be selected; alternatively, black and white can be chosen. Figure 1E As shown, a pair of positive and negative target images are illustrated. Based on this, in the imaging of the positive target image, if the pixel value at a pixel location is greater than the pixel value at the same pixel location in the imaging of the negative image, it can be determined that the pixel value at that location in the imaging of the positive target image represents white. Similarly, the pixel value at that location in the imaging of the corresponding negative target image represents black. When the target image has a pair of positive and negative images, the imaging includes both the positive and negative target images. The positive and negative images are sequences used to accurately obtain the pixel values ​​at pixel locations in the imaging. Therefore, the sequence of pixel values ​​at pixel locations generated in this embodiment can be generated based on the imaging of all positive target images; alternatively, it can be generated based on the imaging of all negative images.

[0087] This embodiment does not restrict the display order of each pair of positive and negative target images. By recording the display order of each pair of positive and negative target images, the imaging of the positive and negative target images can be determined from multiple images in subsequent imaging. Optionally, in some examples, each pair of positive and negative target images can be displayed sequentially. Based on this, the imaging device can capture each pair of positive and negative target images sequentially, thereby quickly acquiring the corresponding images of each pair of positive and negative target images from the sequentially recorded imaging.

[0088] In this embodiment, based on the sequence of pixel values ​​at the pixel positions of multiple recorded images and the sequence of pixel values ​​in the at least three display areas, and the correspondence between the sequence of pixel values ​​in each display area and the sequence of pixel values ​​at the pixel positions in the images, the pose correspondence between the pixel positions of the images captured by the imaging device and the display areas can be determined.

[0089] In some examples, a mapping table can be used to record the correspondence between the pixel positions of the image captured by the imaging device and the pose of the display device. For instance, the target image I_y displayed on the display device is captured by the imaging device as an unprocessed original image I_x. After I_y is captured by the imaging device and the lens itself, along with other optical and other characteristics, it becomes the original image I_x with distortion, rotation, and other properties. After the aforementioned processing steps, decoding each pixel position X of all the acquired I_x yields a decoded value Y. X is stored in position Y of the row direction Z_row and column direction Z_col of the mapping table Z. The Z table essentially stores the mapping relationship between the pixel position Y encoded by the target image and the position Y shifted to pixel X under the influence of the imaging device's characteristics after being captured.

[0090] The following example illustrates that the display interface in this embodiment includes multiple display areas, and multiple target images are displayed sequentially. The sequence of pixel values ​​displayed sequentially in each display area is different. Optionally, the sequence of pixel values ​​displayed sequentially in the display areas of this embodiment represents the position information of the display area. For example, the position information of the display area is (5, 6). Taking a binary encoded image as an example, the decimal "5" and "6" are converted into a set binary code. Taking 4 bits as an example, assuming that the binary codes corresponding to coordinates (5, 6) are (0111, 0101), that is, in the four-bit binary encoded target image, the sequence of pixel values ​​in the display area at position (5, 6) is (white black black black, white black black black).

[0091] The imaging device also captures images of the target images displayed sequentially, obtaining multiple images. By acquiring the pixel values ​​at the same pixel position in each image, a sequence of pixel values ​​at each pixel position can be generated. Due to the pose difference between the imaging device and the display device, as well as the distortion of the camera itself, the pixel value sequence at a certain pixel position can be determined to be a sequence of (white-black-black-black, white-black-black-black). However, this (white-black-black-black, white-black-black-black) sequence is not at position (5, 6), but at position (6, 7) of the image. Based on this, the position information (5, 6) of the display area is matched with the pixel position (6, 7) of the image captured by the imaging device through the (white-black-black-black, white-black-black-black) sequence. Therefore, the correspondence between the pixel position of the image captured by the imaging device and the pose of the display area can be determined through at least three display areas of the display device's display interface.

[0092] For example, a mapping table can actually be understood as a matrix with M rows and N columns, representing the encoding of M*N display areas. The mapping table is an M-row, N-column table. The matrix records the pose correspondences; the value of each element in the matrix represents the pixel position of the image captured by the camera, and the position of this element in the matrix corresponds to the pixel position of the image captured by the camera in the display interface. Conversely, the value of each element in the matrix can also correspond to the pixel position of the image captured by the camera in the display interface, and the position of this element in the matrix corresponds to the pixel position of the image captured by the camera. For example, the position (6, 7) in the image captured by the camera corresponds to the position (5, 6) in the display area. The pixel value sequence (0111, 0101) at position (6, 7) in the image is decoded to (5, 6), which is the position information of the display area. Therefore, (5, 6) can be recorded at position (6, 7) in the mapping table. That is, (5, 6) recorded at position (6, 7) in the matrix represents the pose correspondence between the camera and the display device. In other words, the position (6, 7) of the camera corresponds to the position (5, 6) of the display interface of the display device.

[0093] As can be seen from the above embodiments, the sequence of pixel values ​​displayed in the display area represents the position information of the display area, and can also achieve the effect of fast processing in determining the correspondence between the pixel position of the imaging device and the pose of the display device. The solution of this application will be described again through an embodiment.

[0094] like Figure 2A The diagram shown is a schematic representation of an information acquisition system 100 according to an exemplary embodiment of this application. Figure 2A The imaging device in the information acquisition system 100 is illustrated using a binocular camera as an example. It can be equipped with two monocular cameras, each individually fixed in its own space. The information acquisition system includes two displays (1101 and 1102), each positioned in front of the camera. The two cameras face their respective displays, allowing them to capture the entire simulated image from the simulator. Optionally, the information acquisition system may include movable mounting components such as slide rails, allowing the distance between the camera and the display to be flexibly adjusted when the camera is mounted on these components. In this embodiment, it is not limited that the cameras must share a field of view, nor is it limited that the two cameras must be fixed to the same rigid object.

[0095] Next, a series of coded images can be generated using appropriate encoding techniques. Taking Gray code as an example, the number of pixel positions to be encoded can be determined according to actual needs, such as how many pixels are needed in the width direction and how many pixels are needed in the height direction of the image. Optionally, the number of pixels in the width direction and the number of pixels in the height direction can be set by the user; as an example, the information acquisition system can provide user interaction functions to allow users to set the number of pixels in the width direction and the number of pixels in the height direction of the image.

[0096] Once determined, the coordinates of each row and column of the image can be encoded. For example, a 5-bit Gray code can encode 32 pixel positions; 1024 pixels would require a 10-bit Gray code. Each horizontal and vertical coordinate corresponds to a string of binary numbers in the Gray code, and the corresponding pixel position in the image is either black or white. This process is repeated to encode each pixel in the image. Figure 2B The image shown is a schematic diagram of a five-bit Gray code image according to an embodiment of this application. It shows a Gray code image in the vertical direction, which can encode the image width; and a Gray code in the horizontal direction, which can encode the image height.

[0097] This embodiment also involves Gray code inverse images, used to subsequently determine the images captured by the camera. Based on this, the required number of images to be generated is: (number of binary bits required for image height * 2 + number of binary bits required for image width * 2) forward and inverse Gray code images. For example, assuming the image height is 32 pixels and the image width is 64 pixels, the number of images is: 5 * 2 + 6 * 2. This includes 5-bit Gray code images horizontally (5 forward and 5 inverse images), and 6-bit Gray code images vertically (6 forward and 6 inverse images). Figure 2C As shown, a schematic diagram of a pair of two-bit forward and inverse Gray code images is presented.

[0098] Then, the control display sequentially shows the aforementioned Gray code forward and reverse images. For each image displayed on the display, the camera captures and stores the corresponding image. For ease of subsequent recognition, "sequentially" here can mean the Gray code images are displayed in the order of binary bits, from the first bit to the Nth bit, or from the Nth bit to the first bit. Of course, in practical applications, they can also be displayed in random order, as long as the display order of the Gray code images with different bit positions is known during subsequent recognition. Additionally, "sequentially" here can also include the sequential display of forward and reverse Gray code images for the same bit. For example, each pair of forward and reverse Gray code images can be displayed sequentially, either first the forward and reverse Gray code images, or first the reverse Gray code image followed by the forward Gray code image, so that two consecutive images captured by the camera correspond to the same forward and reverse Gray code image, facilitating subsequent processing.

[0099] In this embodiment, a pair of forward and inverse Gray code images with the same number of bits are displayed to accurately determine which Gray code image corresponds to the image captured by the camera. For example, for two consecutive images captured by the camera corresponding to the same forward and inverse Gray code bit, if the grayscale value of the forward image is larger than that of the inverse image, the pixel is white, and vice versa. N*2 forward and inverse Gray code images can be used to calculate N binarized Gray code images. Based on this, each image captured by the camera corresponding to each forward Gray code image can be determined. For example, based on the forward and inverse Gray code images, the images are binarized. Each pixel X of the captured horizontal and vertical Gray code images is binarized according to the above method, and the binarized Gray code encoded image is stored on the disk. After the operation is completed, a set of black and white binary Gray code encoded images (the number of binary bits required for the image height + the number of binary bits required for the image width) is obtained. For example... Figure 2D As shown, an image from the camera and a binarized Gray code image of that image are presented.

[0100] After the above processing, the binarized Gray code image (hereinafter referred to as the binarized image) can be decoded using Gray code. The true coordinate value of each pixel in the row (or column) direction will be composed of the binary values ​​(0 and 1 corresponding to black and white) of all the binarized images. N bits of Gray code represent N binarized images. Assume X is the coordinate value of a pixel in the original image, and Y is the decoded value of the pixel at position X in the N binarized images. Traverse each pixel X and store the X value at position Y in the mapping table Z, where the size of the Z table is the same as the size of the Gray code encoded value, because the range of the decoded Gray code value is the range of the encoded value. For example, if the Gray code has 1024 columns and 512 rows, then the size of the mapping table is 1024 * 512.

[0101] The mapping table records the relationship between the display interface and the camera's pose. For example, the Gray code image I_y displayed on the monitor is transformed into an unprocessed original image I_x captured by the camera. After the camera and lens's optical and other characteristics are applied, I_y becomes the original image I_x with distortion and rotation. After the aforementioned processing steps, decoding each pixel X of all acquired I_x yields a decoded value Y. X is stored at position Y in the row direction Z_row and column direction Z_col of the mapping table Z. Table Z essentially stores the mapping relationship between the pixel position Y encoded by the Gray code and the pixel X shifted by position Y after being captured by the camera.

[0102] Having the aforementioned mapping table completes the calibration of the information acquisition system. The mapping table, as the result of the calibration, can be used for subsequent processing. For example, when the imaging device subsequently captures images displayed on the display device, the images captured by the imaging device can be transformed based on the mapping table. This allows the distorted, misaligned original image to be mapped to a distortion-free, horizontally aligned target image, such as... Figure 2E As shown, a schematic diagram of the original image captured by the camera and the corrected image is presented. It can be seen that, due to the mapping table recorded by the above calibration method, regardless of the distortion of the camera or the degree of offset in its placement, the original image can be corrected back to a distortion-free, center-aligned target image after processing the image captured by the camera using this mapping table.

[0103] As can be seen from the above embodiments, the correspondence between the pixel position of the imaging device and the pose of the display area is recorded by a mapping table. This mapping table records the one-to-one pose relationship between the imaging plane of the imaging device and the display interface of the display device. Therefore, the image acquired by the imaging device is corrected based on this mapping table, and the corrected image is obtained.

[0104] For scenarios involving binocular or multi-view cameras, a shared field of view is not required during calibration. This is because the solution in this embodiment directly determines the correspondence between the pixel positions of the imaging device and the pose of the display area. Based on this pose correspondence, in scenarios such as hardware-in-the-loop testing, if it is necessary to simulate the parallax of a binocular camera, the images displayed by the two display devices can be controlled to overlap. Consequently, the images acquired by the two imaging devices will also overlap, thus simulating the parallax effect during testing. Furthermore, the overlap rate of the images displayed by the two display devices can be arbitrarily adjusted as needed. Therefore, the solution in this embodiment can also simulate arbitrary parallax, making the testing methods for autonomous driving software more flexible and meeting more testing requirements.

[0105] Therefore, the above calibration embodiments do not require technical personnel intervention, nor do they rely on pre-customized calibration environments, such as calibration objects or ground markings. In binocular or multi-camera scenarios, it is not required that each camera must have a shared field of view. It is convenient and flexible to build hardware-in-the-loop testing equipment for monocular, binocular, or multi-camera systems, and the calibration operation is convenient and can be completed automatically.

[0106] This invention primarily provides an automated method for calibrating binocular hardware-in-the-loop testing equipment that requires no technical personnel intervention, is not limited by calibration markers, and is not constrained by shared field of view. This method eliminates the need for calibration markers, eliminates the need for shared field of view between the binocular cameras, and does not require specialized camera calibration personnel. The method can automatically perform the binocular distortion correction and horizontal alignment correction required by the hardware device. Once completed, it can be used to test autonomous driving algorithms in a manner consistent with traditionally calibrated binocular cameras.

[0107] This embodiment also provides another calibration method for an information acquisition system, the system including a display device and a shooting device; the shooting device faces the display interface of the display device; the display interface includes at least M display areas, the at least M display areas are arranged non-collinearly and do not overlap each other;

[0108] like Figure 3 The present invention provides a flowchart of a calibration method for an information acquisition system according to this embodiment, the method comprising:

[0109] In step 302, the display device is controlled to display i target images on the display interface. During the display of the target images, the sequence of pixel values ​​displayed in any display area of ​​the M display areas is different from the sequence of pixel values ​​displayed in any other display area of ​​the at least M display areas. The number of types of pixel values ​​is N, where N raised to the power of i is greater than or equal to M, and i is greater than or equal to 1.

[0110] In step 304, during the process of displaying i target images, the capturing device is controlled to capture images of the display interface and record i images.

[0111] In step 306, based on the sequence of pixel values ​​at each pixel position in the i images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

[0112] In this embodiment, the target image can be one or multiple images; in practical applications, it can be configured according to the specific scenario. For example, if the imaging sensor of the capturing device can recognize a wide variety of pixel values ​​and accurately distinguish between different pixel values ​​in the image, a smaller number of target images can achieve sequences of different pixel values ​​in multiple display areas. For example, in this embodiment, there are three display areas. The pixel value sequences of the three display areas can be used to find the pixel positions in the image captured by the capturing device that have the same sequence as the pixel value sequences of these three display areas. Since the imaging sensor of the capturing device can recognize a wide variety of pixel values, for example, three types, by displaying these three different types of pixel values ​​in the three display areas respectively, the positions of these three different types of pixel values ​​in the image captured by the capturing device can be identified in one image captured by the capturing device using a single target image. Since the positions of these three display areas on the display interface of the display device are also determinable, the correspondence between the pixel positions in the image captured by the capturing device and the poses of the display areas can be determined.

[0113] In practical applications, the number of display areas, the number of target images, and the types of pixel values ​​can all be implemented in various ways as needed, as described in the foregoing embodiments.

[0114] In some cases, the number of display areas is set by the user.

[0115] In some examples, the positions of the various display areas on the display interface are set by the user.

[0116] In some cases, the pixel values ​​are of at least two types.

[0117] In some examples, the pixel values ​​include black and white.

[0118] In some examples, each pixel value corresponds to a preset code, and the sequence of pixel values ​​is a sequence of encoded pixel values.

[0119] In some examples, the sequence of pixel values ​​displayed sequentially in the display area represents the positional information of the display area.

[0120] In some examples, the decoding result of the encoded sequence of pixel values ​​displayed sequentially in the display area is the position information of the display area on the display interface.

[0121] In some examples, the encoded sequence of the pixel values ​​includes an encoded sequence using binary encoding.

[0122] In some examples, the binary encoding includes any of the following: Gray code, anti-Gray code, ordinary binary code, 8421 code, or 54221 code.

[0123] In some examples, the position information of the display area includes: horizontal position information and vertical position information;

[0124] The multiple target images include:

[0125] A vertically arranged striped image used to encode the horizontal position information; and,

[0126] A horizontally arranged striped image used to encode the vertical position information.

[0127] In some examples, the number of vertically arranged striped images is determined based on the number of display areas that need to be encoded in the horizontal direction.

[0128] In some examples, the number of horizontally arranged striped images is determined based on the number of display areas that need to be encoded in the vertical direction.

[0129] In some examples, the multiple target images include: multiple pairs of positive target images and inverse target images; between each pair of positive target images and inverse target images, the pixel values ​​at the same pixel position are different.

[0130] In some examples, each pair of positive and negative target images is displayed sequentially.

[0131] In some cases, the image captured by the camera device largely overlaps with the display screen.

[0132] In some examples, there are at least two shooting devices, each corresponding to a display device.

[0133] In some examples, the information collection system further includes:

[0134] At least one mounting component, each of which is used to mount a camera device.

[0135] In some examples, the mounting component includes a movable mounting component.

[0136] In some examples, the information acquisition system includes a hardware-in-the-loop test device.

[0137] The above method embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the image processing processor reading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of a calibration device 400 for implementing the information acquisition system of this embodiment. Besides... Figure 4 In addition to the processor 401 and memory 402 shown, the calibration device used to implement the calibration method of this information acquisition system in the embodiment may also include other hardware depending on the actual function of the calibration device, which will not be described in detail here.

[0138] In this embodiment, when the processor 401 executes the computer program, it performs the following steps:

[0139] The display device is controlled to sequentially display multiple target images on the display interface. During the sequential display of the target images, the sequence of pixel values ​​displayed sequentially in any one of the at least three display areas is different from the sequence of pixel values ​​displayed sequentially in any other display area of ​​the at least three display areas.

[0140] During the process of sequentially displaying multiple target images, the capturing device is controlled to capture images of the display interface, and multiple images are sequentially recorded.

[0141] Obtain pixel values ​​at the same pixel location from multiple images, and generate a sequence of pixel values ​​at that pixel location;

[0142] Based on the sequence of pixel values ​​at the pixel positions of multiple recorded images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

[0143] In some cases, the number of display areas is set by the user.

[0144] In some examples, the positions of the various display areas on the display interface are set by the user.

[0145] In some cases, the pixel values ​​are of at least two types.

[0146] In some examples, the pixel values ​​include black and white.

[0147] In some examples, each pixel value corresponds to a preset code, and the sequence of pixel values ​​is a sequence of encoded pixel values.

[0148] In some examples, the sequence of pixel values ​​displayed sequentially in the display area represents the positional information of the display area.

[0149] In some examples, the decoding result of the encoded sequence of pixel values ​​displayed sequentially in the display area is the position information of the display area on the display interface.

[0150] In some examples, the encoded sequence of the pixel values ​​includes an encoded sequence using binary encoding.

[0151] In some examples, the binary encoding includes any of the following: Gray code, anti-Gray code, ordinary binary code, 8421 code, or 54221 code.

[0152] In some examples, the position information of the display area includes: horizontal position information and vertical position information;

[0153] The multiple target images include:

[0154] A vertically arranged striped image used to encode the horizontal position information; and,

[0155] A horizontally arranged striped image used to encode the vertical position information.

[0156] In some examples, the number of vertically arranged striped images is determined based on the number of display areas that need to be encoded in the horizontal direction.

[0157] In some examples, the number of horizontally arranged striped images is determined based on the number of display areas that need to be encoded in the vertical direction.

[0158] In some examples, the multiple target images include: multiple pairs of positive target images and inverse target images; between each pair of positive target images and inverse target images, the pixel values ​​at the same pixel position are different.

[0159] In some examples, each pair of positive and negative target images is displayed sequentially.

[0160] In some cases, the image captured by the camera device largely overlaps with the display screen.

[0161] In some examples, there are at least two shooting devices, each corresponding to a display device.

[0162] In some examples, the information collection system further includes:

[0163] At least one mounting component, each of which is used to mount a camera device.

[0164] In some examples, the mounting component includes a movable mounting component.

[0165] In some examples, the information acquisition system includes a hardware-in-the-loop test device.

[0166] like Figure 5The diagram shown is a hardware structure diagram of a calibration device 500 for another information acquisition system provided in this embodiment. This calibration device includes a processor 501, a memory 502, and a computer program stored in the memory that can be executed by the processor. When the processor executes the computer program, it performs the following steps:

[0167] The display device is controlled to display i target images on the display interface. During the display of the target images, the sequence of pixel values ​​displayed in any display area of ​​the M display areas is different from the sequence of pixel values ​​displayed in any other display area of ​​the at least M display areas. The number of types of pixel values ​​is N, where N raised to the power of i is greater than or equal to M, and i is greater than or equal to 1.

[0168] During the display of i target images, the camera is controlled to capture images of the display interface, and i images are recorded.

[0169] Based on the sequence of pixel values ​​at each pixel position in the i images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

[0170] In some cases, the number of display areas is set by the user.

[0171] In some examples, the positions of the various display areas on the display interface are set by the user.

[0172] In some cases, the pixel values ​​are of at least two types.

[0173] In some examples, the pixel values ​​include black and white.

[0174] In some examples, each pixel value corresponds to a preset code, and the sequence of pixel values ​​is a sequence of encoded pixel values.

[0175] In some examples, the sequence of pixel values ​​displayed sequentially in the display area represents the positional information of the display area.

[0176] In some examples, the decoding result of the encoded sequence of pixel values ​​displayed sequentially in the display area is the position information of the display area on the display interface.

[0177] In some examples, the encoded sequence of the pixel values ​​includes an encoded sequence using binary encoding.

[0178] In some examples, the binary encoding includes any of the following: Gray code, anti-Gray code, ordinary binary code, 8421 code, or 54221 code.

[0179] In some examples, the position information of the display area includes: horizontal position information and vertical position information;

[0180] The multiple target images include:

[0181] A vertically arranged striped image used to encode the horizontal position information; and,

[0182] A horizontally arranged striped image used to encode the vertical position information.

[0183] In some examples, the number of vertically arranged striped images is determined based on the number of display areas that need to be encoded in the horizontal direction.

[0184] In some examples, the number of horizontally arranged striped images is determined based on the number of display areas that need to be encoded in the vertical direction.

[0185] In some examples, the multiple target images include: multiple pairs of positive target images and inverse target images; between each pair of positive target images and inverse target images, the pixel values ​​at the same pixel position are different.

[0186] In some examples, each pair of positive and negative target images is displayed sequentially.

[0187] In some cases, the image captured by the camera device largely overlaps with the display screen.

[0188] In some examples, there are at least two shooting devices, each corresponding to a display device.

[0189] In some examples, the information collection system further includes:

[0190] At least one mounting component, each of which is used to mount a camera device.

[0191] In some examples, the mounting component includes a movable mounting component.

[0192] In some examples, the information acquisition system includes a hardware-in-the-loop test device.

[0193] like Figure 6 As shown in the figure, this application embodiment also provides an information acquisition system 600, including: a shooting device 610; a display device 620; and a calibration device 630; as needed, there may be one or more shooting devices 610, and one or more display devices 620, with each shooting device 610 corresponding to one display device 620; the implementation of the calibration device 630 can be referred to as follows. Figure 4 The calibration device 400 in the illustrated embodiment or Figure 5 The calibration device 500 in the illustrated embodiment.

[0194] This application also provides a computer-readable storage medium storing a plurality of computer instructions, wherein the computer instructions, when executed, perform the steps of the calibration method of the information acquisition system described in any embodiment.

[0195] The embodiments of this specification may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0196] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0197] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0198] The methods and apparatus provided in the embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A calibration method for an information acquisition system, characterized in that, The information acquisition system includes a display device and a camera device; The shooting device faces the display interface of the display device; the display interface includes at least three display areas, which are non-collinearly arranged and do not overlap with each other; The method includes: The display device is controlled to sequentially display multiple target images on the display interface. During the sequential display of the target images, the sequence of pixel values ​​sequentially displayed in any one of the at least three display areas is different from the sequence of pixel values ​​sequentially displayed in any other display area of ​​the at least three display areas. Each pixel value corresponds to a preset code, and the sequence of pixel values ​​is a pixel value encoding sequence. The sequence of pixel values ​​sequentially displayed in the display areas represents the position information of the display areas. The decoding result of the encoding sequence of pixel values ​​sequentially displayed in the display areas is the position information of the display areas on the display interface. During the process of sequentially displaying multiple target images, the capturing device is controlled to capture images of the display interface, and multiple images are sequentially recorded. Obtain pixel values ​​at the same pixel location from multiple images, and generate a sequence of pixel values ​​at that pixel location; Based on the sequence of pixel values ​​at the pixel positions of multiple recorded images and the sequence of pixel values ​​in the at least three display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

2. The method according to claim 1, characterized in that, The number of display areas is set by the user.

3. The method according to claim 1, characterized in that, The position of each of the display areas on the display interface is set by the user.

4. The method according to claim 1, characterized in that, The pixel values ​​are of at least two types.

5. The method according to claim 1, characterized in that, The pixel values ​​include black and white.

6. The method according to claim 1, characterized in that, The encoded sequence of the pixel value includes: an encoded sequence using binary encoding.

7. The method according to claim 6, characterized in that, The binary encoding includes any of the following: Gray code, inverse Gray code, ordinary binary code, 8421 code, or 54221 code.

8. The method according to claim 1, characterized in that, The position information of the display area includes: horizontal position information and vertical position information; The multiple target images include: A vertically arranged striped image used to encode the horizontal position information; and, A horizontally arranged striped image used to encode the vertical position information.

9. The method according to claim 8, characterized in that, The number of vertically arranged stripe images is determined based on the number of display areas that need to be encoded in the horizontal direction.

10. The method according to claim 8, characterized in that, The number of horizontally arranged stripe images is determined based on the number of display areas that need to be encoded in the vertical direction.

11. The method according to claim 1, characterized in that, The multiple target images include: multiple pairs of positive target images and inverse target images; between each pair of positive target images and inverse target images, the pixel values ​​at the same pixel position are different.

12. The method according to claim 11, characterized in that, Each pair of positive and negative target images is displayed sequentially.

13. The method according to claim 1, characterized in that, The image captured by the shooting device and the display interface of the display device are basically overlapped.

14. The method according to claim 1, characterized in that, There are at least two shooting devices, and each shooting device corresponds to a display device.

15. The method according to claim 1, characterized in that, The information collection system also includes: At least one mounting component, each of which is used to mount a camera device.

16. The method according to claim 15, characterized in that, The mounting component includes: a movable mounting component.

17. The method according to claim 1, characterized in that, The information acquisition system includes: hardware-in-the-loop testing equipment.

18. A calibration method for an information acquisition system, characterized in that, The system includes a display device and a camera device; The shooting device faces the display interface of the display device; the display interface includes at least M display areas, which are non-collinearly arranged and do not overlap with each other; The method includes: The display device is controlled to display i target images on the display interface. During the display of the target images, the sequence of pixel values ​​displayed in any one of the M display areas is different from the sequence of pixel values ​​displayed in any other display area of ​​the at least M display areas. There are N types of pixel values, where N raised to the power of i is greater than or equal to M, and i is greater than or equal to 1. Each pixel value corresponds to a preset code, and the sequence of pixel values ​​is the encoded sequence of pixel values. The sequence of pixel values ​​displayed sequentially in the display areas represents the position information of the display areas. The decoding result of the encoded sequence of pixel values ​​displayed sequentially in the display areas is the position information of the display areas on the display interface. During the display of i target images, the camera is controlled to capture images of the display interface, and i images are recorded. Based on the sequence of pixel values ​​at each pixel position in i images and the sequence of pixel values ​​in the at least M display areas, the correspondence between the pixel positions of the images captured by the imaging device and the poses of the display areas is determined.

19. The method according to claim 18, characterized in that, The number of display areas is set by the user.

20. The method according to claim 18, characterized in that, The position of each of the display areas on the display interface is set by the user.

21. The method according to claim 18, characterized in that, The pixel values ​​are of at least two types.

22. The method according to claim 18, characterized in that, The pixel values ​​include black and white.

23. The method according to claim 18, characterized in that, The encoded sequence of the pixel value includes: an encoded sequence using binary encoding.

24. The method according to claim 23, characterized in that, The binary encoding includes any of the following: Gray code, inverse Gray code, ordinary binary code, 8421 code, or 54221 code.

25. The method according to claim 18, characterized in that, The position information of the display area includes: horizontal position information and vertical position information; The i target images include: A vertically arranged striped image used to encode the horizontal position information; and, A horizontally arranged striped image used to encode the vertical position information.

26. The method according to claim 25, characterized in that, The number of vertically arranged stripe images is determined based on the number of display areas that need to be encoded in the horizontal direction.

27. The method according to claim 25, characterized in that, The number of horizontally arranged stripe images is determined based on the number of display areas that need to be encoded in the vertical direction.

28. The method according to claim 18, characterized in that, The i target images include: multiple pairs of positive target images and inverse target images; between each pair of positive target images and inverse target images, the pixel values ​​at the same pixel position are different.

29. The method according to claim 28, characterized in that, Each pair of positive and negative target images is displayed sequentially.

30. The method according to claim 18, characterized in that, The image captured by the shooting device and the display interface of the display device are basically overlapped.

31. The method according to claim 18, characterized in that, There are at least two shooting devices, and each shooting device corresponds to a display device.

32. The method according to claim 18, characterized in that, The information collection system also includes: At least one mounting component, each of which is used to mount a camera device.

33. The method according to claim 32, characterized in that, The mounting component includes: a movable mounting component.

34. The method according to claim 18, characterized in that, The information acquisition system includes: hardware-in-the-loop testing equipment.

35. A calibration device for an information acquisition system, characterized in that, The information acquisition system includes a display device and a camera device; The calibration device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor. When the processor executes the computer program, it implements the steps of the calibration method according to any one of claims 1 to 17.

36. A calibration device for an information acquisition system, characterized in that, The information acquisition system includes a display device and a camera device; the camera device faces the display interface of the display device. The calibration device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor. When the processor executes the computer program, it implements the steps of the calibration method according to any one of claims 18 to 34.

37. An information acquisition system, characterized in that, The information acquisition system includes: a shooting device and a display device; and a calibration device as described in claim 35 and / or a calibration device as described in claim 36.

38. The information acquisition system according to claim 37, characterized in that, The information acquisition system includes: hardware-in-the-loop testing equipment.

39. A computer-readable storage medium, characterized in that, The readable storage medium stores a plurality of computer instructions, which, when executed, implement the steps of the calibration method according to any one of claims 1 to 17.

40. A computer-readable storage medium, characterized in that, The readable storage medium stores a plurality of computer instructions, which, when executed, implement the steps of the calibration method according to any one of claims 18 to 34.

Images