Viewpoint alignment method, device, storage medium and electronic device

Abstract

The present application discloses a viewpoint alignment method, device, storage medium and electronic device. The method comprises: determining the projection position of a target on a screen corresponding to a human eye viewpoint; displaying feature points according to the projection position; and adjusting the position of a camera device so that the target and the feature points in a target image obtained by the camera device shooting the target through the screen are aligned. The present application can improve the accuracy of the camera device in calibrating the positional relationship between the camera and the screen.

Description

Technical Field

[0001] The present application belongs to the field of electronic technology, and in particular, relates to a viewpoint alignment method, device, storage medium and electronic device. Background Art

[0002] For optical transmissive display devices, if AR development is required, a camera needs to be added, and the positional relationship between the camera and the screen of the optical transmissive display device needs to be calibrated, that is, the user needs to wear the optical transmissive display device and align some cursors on the screen with objects in the real world multiple times. Since the manual calibration method is inefficient, the relevant technology proposes to use a camera device to replace the human eye to calibrate the positional relationship between the camera and the screen, but this calibration method has low accuracy. Summary of the invention

[0003] The embodiments of the present application provide a viewpoint alignment method, device, storage medium and electronic device, which can improve the accuracy of calibration of the position relationship between the camera and the screen of a camera device.

[0004] In a first aspect, an embodiment of the present application provides a viewpoint alignment method, comprising:

[0005] Determine the projection position of the target on the screen corresponding to the human eye's viewpoint;

[0006] Display feature points according to projection positions;

[0007] The position of the camera device is adjusted so that the target and the feature point are aligned in the target image obtained by the camera device shooting the target through the screen.

[0008] In a second aspect, an embodiment of the present application provides a viewpoint alignment device, comprising:

[0009] A determination module, used to determine the projection position of the target on the screen corresponding to the viewpoint of the human eye;

[0010] A display module, used for displaying feature points according to projection positions;

[0011] The adjustment module is used to adjust the position of the camera device so that the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned.

[0012] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed on a computer, the computer executes the viewpoint alignment method provided by the embodiment of the present application.

[0013] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the processor is used to execute the viewpoint alignment method provided in the embodiment of the present application by calling a computer program stored in the memory.

[0014] In an embodiment of the present application, by determining the projection position of the target on the screen corresponding to the human eye viewpoint; displaying feature points according to the projection position; and adjusting the position of the camera device so that the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, the camera device and the human eye viewpoint can be aligned, thereby improving the accuracy of the camera device in calibrating the positional relationship between the camera and the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The technical solution and beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

[0016] Figure 1 This is a first flow chart of the viewpoint alignment method provided in an embodiment of the present application.

[0017] Figure 2 This is a layout scenario of multiple targets provided in an embodiment of the present application.

[0018] Figure 3 It is a schematic diagram of the target provided in the examples of the present application.

[0019] Figure 4 This is a second flow chart of the viewpoint alignment method provided in an embodiment of the present application.

[0020] Figure 5 It is a structural schematic diagram of a viewpoint alignment device provided in an embodiment of the present application.

[0021] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be noted that the terms "first", "second" and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products or devices.

[0023] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0024] The embodiment of the present application provides a viewpoint alignment method, a viewpoint alignment device, a storage medium and an electronic device, wherein the execution subject of the viewpoint alignment method can be the viewpoint alignment device provided in the embodiment of the present application, or an electronic device integrating the viewpoint alignment device, wherein the viewpoint alignment device can be implemented in hardware or software. The electronic device can be a transmissive display device.

[0025] Transmissive display devices may include transmissive head-mounted display devices, such as AR glasses and AR helmets. As the name suggests, the so-called transmissive head-mounted display device is a display device that can be worn on the head and can be used conveniently without the need for hands. The transmissive head-mounted display device has a semi-transmissive and semi-reflective optical system. On the one hand, it can transmit external ambient light like ordinary glasses, so that the user wearing the transmissive head-mounted display device can see the real world in front of them. On the other hand, it can reflect the image from the micro-display and superimpose it on the human field of vision. That is, the user wearing the transmissive head-mounted display device can not only see the virtual image, but also see the displayed world through the screen. In this way, the use of a perspective head-mounted display device can achieve the fusion of virtual information and the real environment.

[0026] See also Figure 1 , Figure 1 This is a schematic diagram of a first process flow of the viewpoint alignment method provided in an embodiment of the present application. The process may include:

[0027] 101. Determine the projection position of the target on the screen corresponding to the human eye's viewpoint.

[0028] In this embodiment, a target can be set in advance within a preset range of the field of view of the transmissive display device, such as within 5 meters. When it is necessary to align the viewpoints of the human eye and the camera device, the user can wear the transmissive display device, observe the preset target through the screen of the transmissive display device, and mark the position of the target on the screen, so that the transmissive display device can obtain the projection position of the target on the screen corresponding to the viewpoint of the human eye.

[0029] 102. Display feature points according to projection positions.

[0030] In this embodiment, after obtaining the projection position of the target on the screen corresponding to the human eye viewpoint, the transmissive display device can display the feature point at the projection position. For example, if the projection position is in the upper left corner of the screen, then the feature point can be displayed in the upper left corner of the screen. For another example, if the projection position is in the lower right corner of the screen, then the feature point can be displayed in the lower right corner of the screen.

[0031] It is understandable that in this embodiment, the feature point is a point displayed on the screen that can be seen by the human eye or captured by a camera. As for the type and size of the feature point, this embodiment of the application does not limit it, and it is subject to actual needs.

[0032] 103. Adjust the position of the camera device so that the target and the feature point are aligned in the target image obtained by the camera device shooting the target through the screen.

[0033] In this embodiment, the transmissive display device can control the adjustment device to adjust the position of the camera device so that the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned.

[0034] It should be noted that, in this embodiment, when the overlap percentage between the target and the feature point in the target image is greater than a preset percentage, it is determined that the target and the feature point in the target image are aligned. The preset percentage can be set according to actual conditions and is not specifically limited here. For example, when a higher alignment accuracy is required, the preset percentage can be set higher, such as 99.8%.

[0035] It can be understood that when the target and feature points in the target image obtained by the camera device shooting the target through the screen are aligned, the feature points on the screen observed by the human eye are also aligned with the target observed through the screen, thereby achieving alignment of the viewpoints of the human eye and the camera device.

[0036] In an embodiment of the present application, by determining the projection position of the target on the screen corresponding to the human eye viewpoint; displaying feature points according to the projection position; and adjusting the position of the camera device so that the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, the camera device and the human eye viewpoint can be aligned, thereby improving the accuracy of the camera device in calibrating the positional relationship between the camera and the screen.

[0037] In some embodiments, before adjusting the position of the camera device, the following steps may also be included:

[0038] Acquire a first target image obtained by photographing the target through a screen by a camera device;

[0039] Adjust the position of the camera equipment, which may include:

[0040] According to the first target image, the position of the camera device is adjusted.

[0041] It should be noted that, in this embodiment, the feature points are obtained by the user observing the arranged targets with his eyes, that is, when the feature points and the targets are observed by the human eye, the feature points and the targets are aligned. Then, when the target and the feature points in the target image obtained by the camera device shooting the target through the screen are not aligned, the viewpoints of the human eye and the camera device are also not aligned, and when the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, the viewpoints of the human eye and the camera device are also aligned. Therefore, the transmissive display device can adjust the position of the camera device according to the target image obtained by the camera device shooting the target through the screen.

[0042] For example, the transmissive display device can control the camera device to shoot the target through the screen of the transmissive display device to obtain a first target image. The revealing display device can obtain the first target image and adjust the position of the camera device according to the first target image.

[0043] It is understandable that when the target and the feature point in the first target image are aligned, the position of the camera device may not be adjusted and the process may be terminated directly. Alternatively, the position of the camera device may be adjusted according to the first target image, so that the overlap percentage of the target and the feature point in the target image obtained by the camera device after the position adjustment by shooting the target through the screen is greater than the overlap percentage of the target and the feature point in the first target image, thereby improving the alignment accuracy between the human eye and the camera device. When the target and the feature point in the first target image are not aligned, the position of the camera device may be adjusted directly according to the first target image.

[0044] In some embodiments, adjusting the position of the camera device according to the first target image may include:

[0045] Determining a first adjustment amount and a first adjustment direction of the camera device according to positions of the target and the feature point in the first target image;

[0046] The position of the imaging device is adjusted according to the first adjustment amount and the first adjustment direction.

[0047] For example, the position of the target and the feature point may include the coordinates of the target and the feature point. The transmissive display device determines the distance between the target and the feature point based on the coordinates of the target and the feature point in the first target image. The transmissive display device may also determine the relative direction between the target and the feature point in the first target image, and determine the first adjustment amount and the first adjustment direction of the camera device based on the distance and the relative direction. After determining the first adjustment amount and the first adjustment direction, the transmissive display device may control the adjustment device to adjust the position of the camera device based on the first adjustment amount and the first adjustment direction, so that the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned.

[0048] For example, a first mapping relationship between the distance and the adjustment amount, and a second mapping relationship between the relative direction and the adjustment direction may be preset and built into the transmissive display device. After obtaining the distance between the target and the feature point and the relative direction between the target and the feature point in the first target image, the transmissive display device may determine the first adjustment amount according to the distance and the first mapping relationship, and determine the first adjustment direction according to the relative direction and the second mapping relationship.

[0049] The first adjustment direction and the first adjustment amount are adjustment directions and adjustment amounts that can make the target and the feature point in the target image obtained by the camera device shooting the target through the screen after the position adjustment be aligned after the position adjustment of the camera device according to the first adjustment direction and the first adjustment amount. The first adjustment amount may include a translation amount and / or a rotation amount, and the first adjustment direction may include a translation direction of an x, y or z axis, or a rotation direction of pitch, yaw, roll.

[0050] In some embodiments, adjusting the position of the camera device may include:

[0051] receiving input of adjustment amount and adjustment direction;

[0052] Adjust the position of the camera device according to the input adjustment amount and adjustment direction.

[0053] In this embodiment, the transmissive display device can receive the adjustment amount and adjustment direction input by the user, and control the adjustment device to adjust the position of the camera device according to the adjustment amount and adjustment direction input by the user, so that the target and feature points in the target image obtained by the camera device shooting the target through the screen are aligned.

[0054] In some embodiments, after adjusting the position of the camera device according to the input adjustment amount and adjustment direction, the following steps may also be included:

[0055] When the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned, alignment prompt information is output.

[0056] It is understandable that after the adjustment device is controlled to adjust the position of the camera device according to the adjustment amount and adjustment direction input by the user, the transmissive display device can obtain the target image obtained by the camera device after the position is adjusted by shooting the target through the screen. The transmissive display device can detect whether the target in the target image is aligned with the feature point. If the target in the target image is aligned with the feature point, the transmissive display device can output alignment prompt information to prompt the user to align the human eye with the viewpoint of the camera device, without continuing to input the adjustment amount and adjustment direction. If the target in the target image is not aligned with the feature point, the transmissive display device can output misalignment prompt information to prompt the user that the human eye is not aligned with the viewpoint of the camera device, then the user can continue to input the adjustment amount and adjustment direction to adjust the position of the camera device until the target in the target image obtained by the camera device after the position is adjusted by shooting the target through the screen is aligned with the feature point.

[0057] It should be noted that, in this embodiment, there is no specific restriction on the output method of the alignment prompt information, which is subject to actual needs. For example, the alignment prompt information can be played and / or displayed, or other devices can be controlled to play and / or display the alignment prompt information. There is no specific restriction on the output method of the misalignment prompt information, which is subject to actual needs. For example, the misalignment prompt information can be played and / or displayed, or other devices can be controlled to play and / or display the misalignment prompt information.

[0058] In some embodiments, after adjusting the position of the camera device according to the input adjustment amount and adjustment direction, the following steps may also be included:

[0059] When the target and the feature point in the target image obtained by the camera device shooting the target through the screen are not aligned, obtaining a second target image obtained by the camera device shooting the target through the screen;

[0060] Determining a second adjustment amount and a second adjustment direction of the camera device according to the positions of the target and the feature point in the second target image;

[0061] Adjustment prompt information is output according to the second adjustment amount and the second adjustment direction.

[0062] After the adjustment device is controlled to adjust the position of the camera device according to the adjustment amount and adjustment direction input by the user, the transmissive display device can obtain a second target image obtained by the camera device after the position is adjusted by shooting the target through the screen. The transmissive display device can detect whether the target and the feature point in the second target image are aligned. If the target and the feature point in the second target image are not aligned, the transmissive display device can determine the second adjustment amount and the second adjustment direction of the camera device according to the second target image, and output adjustment prompt information according to the second adjustment amount and the second adjustment direction, so that the user can input the adjustment amount and the adjustment direction according to the prompt information, thereby more quickly realizing the alignment of the viewpoint of the human eye and the camera device.

[0063] For example, the position of the target and the feature point may include the coordinates of the target and the feature point. The transmissive display device determines the distance between the target and the feature point based on the coordinates of the target and the feature point in the second target image. The transmissive display device may also determine the relative direction between the target and the feature point in the second target image, and determine the second adjustment amount and the second adjustment direction of the camera device based on the distance and the relative direction. After determining the second adjustment amount and the second adjustment direction, the transmissive display device may control the adjustment device to adjust the position of the camera device based on the second adjustment amount and the second adjustment direction, so that the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned.

[0064] For example, a second mapping relationship between the distance and the adjustment amount, and a second mapping relationship between the relative direction and the adjustment direction may be preset and built into the transmissive display device. After obtaining the distance between the target and the feature point and the relative direction between the target and the feature point in the second target image, the transmissive display device may determine the second adjustment amount according to the distance and the second mapping relationship, and determine the second adjustment direction according to the relative direction and the second mapping relationship.

[0065] The second adjustment direction and the second adjustment amount are adjustment directions and adjustment amounts that can make the target and the feature point in the target image obtained by the camera device shooting the target through the screen after the position adjustment be aligned after the position adjustment of the camera device according to the second adjustment direction and the second adjustment amount. The second adjustment amount may include a translation amount and / or a rotation amount, and the second adjustment direction may include a translation direction of the x, y or z axis, or a rotation direction of pitch, yaw, roll.

[0066] It should also be noted that in this embodiment, there is no specific restriction on the output method of the adjustment prompt information, which is subject to actual needs. For example, the adjustment prompt information can be played and / or displayed, or other devices can be controlled to play and / or display the adjustment prompt information. For example, the first adjustment amount and the first adjustment direction can be directly announced by voice, or the first adjustment amount and the first adjustment direction can be directly displayed, or other devices can be controlled to display the first adjustment amount and the first adjustment direction.

[0067] In some embodiments, there are multiple targets, and the multiple targets are at different distances from the screen.

[0068] For example, multiple targets can be displayed within a preset range, such as 5 meters, in the field of view of the transmissive display device, where the distances between the multiple targets and the screen are different. Specifically, the multiple targets can be arranged at different depths and different heights within the preset range in the field of view of the transmissive display device. For example, the arrangement scenario of the multiple targets can be as follows: Figure 2 As shown. Wherein, the target can be as follows Figure 3The checkerboard target shown. It is understandable that the target can also be a target in other forms, which is not specifically limited here.

[0069] It can be understood that the human eye includes a left eye and a right eye, and a transmissive display device generally includes a first screen and a second screen, the first screen corresponds to the left eye, and the second screen corresponds to the right eye. Therefore, when aligning the viewpoints of the human eye and the camera device, it is necessary to align the viewpoints of the left eye and a camera device based on the first screen, and to align the viewpoints of the right eye and another camera device based on the second screen.

[0070] The following example uses the transmissive display device as AR glasses, the adjustment device as the calibration device, and the camera device as an industrial camera to align the viewpoint of the left eye with that of the industrial camera. The industrial camera can be a distortion-free industrial camera or an industrial camera that has undergone distortion correction to ensure that the image is distortion-free, thereby improving the alignment accuracy between the human eye and the industrial camera.

[0071] Among them, the calibration device may include a glasses fixing fixture and a six-axis turntable (three-axis rotation pitch, yaw, roll; three-axis translation x, y, z), wherein the fixing fixture is used to fix the AR glasses, and the six-axis turntable is used to set the industrial camera to ensure that the industrial camera is adjustable in six degrees of freedom. AR glasses use augmented reality (AR) technology. Among them, augmented reality is a technology that increases the user's perception of the real world through information provided by a computer system. It superimposes computer-generated virtual objects, scenes, or system prompt information and other content objects into real scenes to enhance or modify the perception of the real-world environment or data representing the real-world environment.

[0072] First, fix the AR glasses with a fixture, and then keep the calibration device stationary, such as Figure 2 As shown, within the 5-meter field of view of the AR glasses, checkerboard targets are arranged at different depths and heights.

[0073] The user puts his head into the calibration device and wears AR glasses normally. The left eye observes the target in the scene through the AR glasses, and uses the mouse to click on the screen corresponding to the left eye of the AR glasses to record the arranged target position, so that the AR glasses can obtain the projection position of the target on the screen corresponding to the left eye viewpoint. Subsequently, the AR glasses can display the feature points at the projection position. After recording the target position, the user can drill his head out of the calibration device.

[0074] Next, an industrial camera can be set at the position of the six-axis turntable corresponding to the left eye, that is, the position of the industrial camera is kept basically consistent with the position of the user's left eye. Then, the AR glasses can control the camera device to shoot the target through the screen to obtain a target image. The AR glasses can obtain the target image and detect whether the target and the feature points in the target image are aligned. Since the position of the industrial camera cannot be completely consistent with the position of the user's left eye, and the industrial camera itself is different from the human eye, the target and the feature points in the target image will not be aligned. Then, the six-axis machine can be controlled to adjust the position of the industrial camera so that the target and the feature points in the target image obtained by the industrial camera shooting the target through the screen are aligned. In some cases, if the target and the feature points in the target image obtained by the industrial camera are aligned before adjusting the position of the industrial camera, then the position of the industrial camera can be left unchanged, or the position of the industrial camera can be adjusted so that the percentage of overlap between the target and the feature points in the target image obtained is higher.

[0075] Based on the above process, the viewpoint alignment of the left eye and the industrial camera is achieved. The above process can also be referred to for the viewpoint alignment of the right eye and the industrial camera, which will not be repeated here. In addition, after the viewpoints of the left eye and the industrial camera are aligned, the industrial camera can be fixed so that the industrial camera does not move, thereby avoiding the problem of misalignment of the viewpoints of the left eye and the industrial camera due to the movement of the industrial camera. When aligning the viewpoints of the right eye and the industrial camera, another industrial camera can be used to align the viewpoints of the right eye and the industrial camera, and after alignment, the other industrial camera can be fixed.

[0076] It can be understood that the viewpoint alignment method provided in the embodiment of the present application can ensure that transmissive display devices, such as AR glasses, use camera equipment such as industrial cameras in the production to replace the human eye for large-scale rapid calibration while ensuring good accuracy. This solution can solve the time-consuming operation of manual calibration of each AR glass with the human eye, reduce the misjudgment caused by human operation, and save a lot of costs for the enterprise.

[0077] In some embodiments, considering that each person's pupil distance and eye size are different, such as an adult's pupil distance is larger than a child's, and an adult's eye size is also larger than a child's, different people's eyes can be classified according to pupil distance and / or eye size, and then the eyes of each class are aligned with the viewpoint of the camera device, so that AR eyes suitable for different groups of people can be calibrated with higher accuracy.

[0078] In some embodiments, the viewpoint alignment method can also be applied to the virtual-reality combined calibration of the vehicle-mounted AR-HUD, that is, the windshield display surface is a screen, and the user records the projection position of the human eye and the physical target in front of the windshield on the windshield in the cockpit. The feature points can be displayed at the projection position, and the physical target points and the feature points can be aligned using an industrial camera. At this time, the industrial camera position can be used instead of the human eye viewpoint position to complete large-scale rapid calibration of the virtual-reality fusion of the HUD, eliminating the time-consuming operation of manual calibration of the HUD of each car.

[0079] See also Figure 4 , Figure 4 This is a schematic diagram of a second process of the viewpoint alignment method provided in an embodiment of the present application. The process may include:

[0080] 201. Determine the projection position of the target on the screen corresponding to the viewpoint of the human eye.

[0081] In this embodiment, a target can be set in advance within a preset range of the field of view of the transmissive display device, such as within 5 meters. When it is necessary to align the viewpoints of the human eye and the camera device, the user can wear the transmissive display device, observe the preset target through the screen of the transmissive display device, and mark the position of the target on the screen, so that the transmissive display device can obtain the projection position of the target on the screen corresponding to the viewpoint of the human eye.

[0082] 202. Display feature points according to the projection positions.

[0083] In this embodiment, after obtaining the projection position of the target on the screen corresponding to the human eye viewpoint, the transmissive display device can display the feature point at the projection position. For example, if the projection position is in the upper left corner of the screen, then the feature point can be displayed in the upper left corner of the screen. For another example, if the projection position is in the lower right corner of the screen, then the feature point can be displayed in the lower right corner of the screen.

[0084] It is understandable that in this embodiment, the feature point is a point displayed on the screen that can be seen by the human eye or captured by a camera. As for the type and size of the feature point, this embodiment of the application does not limit it, and it is subject to actual needs.

[0085] 203. Acquire a first target image obtained by photographing the target through a screen by a camera device.

[0086] It should be noted that, in this embodiment, the feature points are obtained by the user observing the arranged targets with his eyes, that is, when the feature points and the targets are observed by the human eye, the feature points and the targets are aligned. Then, when the target and the feature points in the target image obtained by the camera device shooting the target through the screen are not aligned, the viewpoints of the human eye and the camera device are also not aligned, and when the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, the viewpoints of the human eye and the camera device are also aligned. Therefore, the transmissive display device can adjust the position of the camera device according to the target image obtained by the camera device shooting the target through the screen.

[0087] For example, the transmissive display device can control the camera to shoot the target through the screen of the transmissive display device to obtain a first target image. The revealing display device can obtain the first target image.

[0088] 204. Determine a first adjustment amount and a first adjustment direction of the camera device according to positions of the target and the feature point in the first target image.

[0089] For example, the position of the target and the feature point may include the coordinates of the target and the feature point. The transmissive display device determines the distance between the target and the feature point according to the coordinates of the target and the feature point in the first target image. The transmissive display device may also determine the relative direction of the target and the feature point in the first target image, and determine the first adjustment amount and the first adjustment direction of the camera device according to the distance and the relative direction.

[0090] For example, a first mapping relationship between the distance and the adjustment amount, and a second mapping relationship between the relative direction and the adjustment direction may be preset and built into the transmissive display device. After obtaining the distance between the target and the feature point and the relative direction between the target and the feature point in the first target image, the transmissive display device may determine the first adjustment amount according to the distance and the first mapping relationship, and determine the first adjustment direction according to the relative direction and the second mapping relationship.

[0091] The first adjustment direction and the first adjustment amount are adjustment directions and adjustment amounts that can make the target and the feature point in the target image obtained by the camera device shooting the target through the screen after the position adjustment be aligned after the position adjustment of the camera device according to the first adjustment direction and the first adjustment amount. The first adjustment amount may include a translation amount and / or a rotation amount, and the first adjustment direction may include a translation direction of an x, y or z axis, or a rotation direction of pitch, yaw, roll.

[0092] 205. Adjust the position of the camera device according to the first adjustment amount and the first adjustment direction, so that the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned.

[0093] For example, after determining the first adjustment amount and the first adjustment direction, the transmissive display device can control the adjustment device to adjust the position of the camera device according to the first adjustment amount and the first adjustment direction, so that the target in the target image obtained by the camera device shooting the target through the screen is aligned with the feature point.

[0094] For example, assuming that the first adjustment amount is 5 mm and the first adjustment direction is the x-axis direction, the transmissive display device can control the adjustment device to move the camera device 5 mm in the x-axis direction, so that the target in the target image obtained by the camera device shooting the target through the screen is aligned with the feature point.

[0095] In an embodiment of the present application, by determining the projection position of the target on the screen corresponding to the human eye viewpoint; displaying feature points according to the projection position; and adjusting the position of the camera device so that the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, the camera device and the human eye viewpoint can be aligned, thereby improving the accuracy of the camera device in calibrating the positional relationship between the camera and the screen.

[0096] See also Figure 5 , Figure 5 The structure diagram of the viewpoint alignment device provided in the embodiment of the present application is as follows: The viewpoint alignment device 300 comprises: a determination module 301 , a display module 302 and an adjustment module 303 .

[0097] A determination module 301 is used to determine the projection position of the target on the screen corresponding to the viewpoint of the human eye;

[0098] Display module 302, used to display feature points according to projection positions;

[0099] The adjustment module 303 is used to adjust the position of the camera device so that the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned.

[0100] In some embodiments, the adjustment module 303 may be used to: obtain a first target image obtained by the camera device shooting the target through the screen; and adjust the position of the camera device according to the first target image.

[0101] In some embodiments, the adjustment module 303 can be used to: determine a first adjustment amount and a first adjustment direction of the camera device according to the positions of the target and the feature point in the first target image; and adjust the position of the camera device according to the first adjustment amount and the first adjustment direction.

[0102] In some embodiments, the adjustment module 303 may be configured to: receive an input adjustment amount and an adjustment direction; and adjust the position of the camera device according to the input adjustment amount and the adjustment direction.

[0103] In some embodiments, the viewpoint alignment device 300 may further include an output module, which may be used to: after adjusting the position of the camera device according to the input adjustment amount and adjustment direction, output alignment prompt information when the target in the target image obtained by the camera device shooting the target through the screen is aligned with the feature point.

[0104] In some embodiments, the output module can be used to: obtain a second target image obtained by the camera device shooting the target through the screen when the target and the feature point in the target image obtained by the camera device shooting the target through the screen are not aligned; determine a second adjustment amount and a second adjustment direction of the camera device according to the positions of the target and the feature point in the second target image; and output adjustment prompt information according to the second adjustment amount and the second adjustment direction.

[0105] In some embodiments, there are multiple targets, and the multiple targets are at different distances from the screen.

[0106] It should be noted that the viewpoint alignment device provided in the embodiment of the present application and the viewpoint alignment method in the above embodiment belong to the same concept, and its specific implementation process is detailed in the above embodiment and will not be repeated here.

[0107] The viewpoint alignment device 200 provided in the embodiment of the present application determines the projection position of the target on the screen corresponding to the human eye viewpoint through the determination module 301; displays the feature points according to the projection position through the display module 302; and adjusts the position of the camera device through the adjustment module 303, so that the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, which can achieve the alignment of the camera device and the human eye viewpoint, thereby improving the accuracy of the camera device in calibrating the positional relationship between the camera and the screen.

[0108] The present application embodiment provides a storage medium on which a computer program is stored. When the computer program stored therein is executed on a processor of an electronic device provided in the present application embodiment, the processor of the electronic device executes any of the steps in the above viewpoint alignment method suitable for an electronic device. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0109] This application also provides an electronic device, please refer to Figure 6 , the electronic device 400 includes components such as a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 6The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the electronic device 400 may also include a screen for displaying feature points according to the projection position, etc.

[0110] The processor 401 in the embodiment of the present application may be a general-purpose processor, such as an ARM architecture processor.

[0111] The memory 402 stores a computer program, which may be a high-speed random access memory or a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402. The processor 401 executes the computer program in the memory 402 to perform:

[0112] Determine the projection position of the target on the screen corresponding to the human eye's viewpoint;

[0113] Display feature points according to projection positions;

[0114] The position of the camera device is adjusted so that the target and the feature point are aligned in the target image obtained by the camera device shooting the target through the screen.

[0115] In some embodiments, before the processor 401 executes the adjustment of the position of the camera device, it can also execute: obtaining a first target image obtained by the camera device shooting the target through the screen; when the processor 401 executes the adjustment of the position of the camera device, it can execute: adjusting the position of the camera device according to the first target image.

[0116] In some embodiments, when the processor 401 executes the adjustment of the position of the camera device according to the first target image, it can execute: determining a first adjustment amount and a first adjustment direction of the camera device according to the positions of the target and feature points in the first target image; and adjusting the position of the camera device according to the first adjustment amount and the first adjustment direction.

[0117] In some implementations, when the processor 401 executes the step of adjusting the position of the imaging device, the steps may include: receiving an input adjustment amount and an adjustment direction; and adjusting the position of the imaging device according to the input adjustment amount and the adjustment direction.

[0118] In some embodiments, after the processor 401 executes the adjustment of the position of the camera device according to the input adjustment amount and adjustment direction, it can also execute: when the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned, output alignment prompt information.

[0119] In some embodiments, after the processor 401 executes the adjustment of the position of the camera device according to the input adjustment amount and adjustment direction, it can also execute: when the target and the feature point in the target image obtained by the camera device shooting the target through the screen are not aligned, obtaining a second target image obtained by the camera device shooting the target through the screen; determining a second adjustment amount and a second adjustment direction of the camera device according to the position of the target and the feature point in the second target image; and outputting adjustment prompt information according to the second adjustment amount and the second adjustment direction.

[0120] In some embodiments, there are multiple targets, and the multiple targets are at different distances from the screen.

[0121] The electronic device provided in the embodiment of the present application can align the camera device with the human eye's viewpoint by determining the projection position of the target on the screen corresponding to the human eye's viewpoint; displaying feature points according to the projection position; and adjusting the position of the camera device so that the target and the feature points in the target image obtained by the camera device shooting the target through the screen are aligned, thereby achieving alignment of the camera device with the human eye's viewpoint, thereby improving the accuracy of the camera device in calibrating the positional relationship between the camera and the screen.

[0122] The above is a detailed introduction to a viewpoint alignment method, device, storage medium and electronic device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A viewpoint alignment method, characterized in that: include: Determine the projection position of the target on the screen corresponding to the viewpoint of the human eye obtained by observing the target through the screen; Displaying feature points at the projection position, wherein the feature points are points that can be photographed by a camera device; The position of the camera device is adjusted so that the target in the target image obtained by the camera device shooting the target through the screen is aligned with the feature points in the target image.

2. The viewpoint alignment method according to claim 1, characterized in that: Before adjusting the position of the camera device, the method further comprises: Acquire a first target image obtained by photographing the target through the screen by the camera device; The step of adjusting the position of the camera device comprises: The position of the imaging device is adjusted according to the first target image.

3. The viewpoint alignment method according to claim 2, characterized in that: The step of adjusting the position of the camera device according to the first target image includes: Determining a first adjustment amount and a first adjustment direction of the camera device according to positions of the target and the feature point in the first target image; The position of the imaging device is adjusted according to the first adjustment amount and the first adjustment direction.

4. The viewpoint alignment method according to claim 1, characterized in that: The step of adjusting the position of the camera device comprises: receiving input of adjustment amount and adjustment direction; The position of the camera device is adjusted according to the input adjustment amount and adjustment direction.

5. The viewpoint alignment method according to claim 4, characterized in that: After adjusting the position of the camera device according to the input adjustment amount and adjustment direction, the method further includes: When the target and the feature point in the target image obtained by the camera device shooting the target through the screen are aligned, alignment prompt information is output.

6. The viewpoint alignment method according to claim 4, characterized in that: After adjusting the position of the camera device according to the input adjustment amount and adjustment direction, the method further includes: When the target and the feature point in the target image obtained by the camera device photographing the target through the screen are not aligned, obtaining a second target image obtained by the camera device photographing the target through the screen; Determining a second adjustment amount and a second adjustment direction of the camera device according to the positions of the target and the feature point in the second target image; Adjustment prompt information is output according to the second adjustment amount and the second adjustment direction.

7. The viewpoint alignment method according to any one of claims 1 to 6, characterized in that: There are multiple targets, and the distances between the multiple targets and the screen are different.

8. A viewpoint alignment device, characterized in that: include: A determination module, used to determine a projection position of the target on the screen corresponding to a viewpoint of a human eye obtained by observing the target through the screen; A display module, used for displaying feature points at the projection position, wherein the feature points are points that can be photographed by a camera device; The adjustment module is used to adjust the position of the camera device so that the target in the target image obtained by the camera device shooting the target through the screen is aligned with the feature points in the target image.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is run on a computer, the computer is enabled to execute the viewpoint alignment method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the viewpoint alignment method according to any one of claims 1 to 7 by calling the computer program stored in the memory.

Images