Color cast correction method, device, equipment, system and readable storage medium

Abstract

The application is suitable for the technical field of display screens, and provides a color deviation correction method, device, equipment and readable storage medium. The color deviation correction method specifically comprises: obtaining a display image obtained by a first acquisition device shooting a first display screen, the display image comprising at least one image region, each image region corresponding to a display sub-region of the first display screen, and each display sub-region forming a deflection angle with the first acquisition device; determining current optical information of each image region; determining a gain coefficient of the corresponding deflection angle according to target optical information and the current optical information of each image region, the gain coefficient being used for color deviation correction of an image collected by a second acquisition device or an image displayed by a second display screen. The embodiment of the application can improve the efficiency of color deviation correction.

Description

Technical Field

[0001] This application belongs to the field of display screen technology, and particularly relates to a color distortion correction method, apparatus, device, system and readable storage medium. Background Technology

[0002] Each pixel of a display screen is typically formed by an arrangement of red, green, and blue (RGB) LEDs. Due to limitations and influences such as the display screen's manufacturing process, the physical characteristics of the LEDs, and the arrangement order of different types of LEDs, viewing the display screen from different viewing angles will usually result in varying degrees of brightness attenuation and color shift. This color distortion phenomenon is also known as color cast in the display screen.

[0003] In fields such as virtual photography and the metaverse, the content displayed on the screen in real time often serves as the background for foreground objects, captured by the acquisition device along with the foreground objects, and then blended with the virtual image. The shooting angle of the acquisition device towards the foreground objects often changes dynamically, leading to different color casts on the screen at different shooting angles. Therefore, how to achieve color cast correction has become a critical pain point that the industry urgently needs to address.

[0004] In related technologies, color cast correction typically requires capturing images of the display screen from different angles using a data acquisition device before performing correction processing. To meet the requirements of color cast correction, this method is usually inefficient and demands high system processing power. Summary of the Invention

[0005] This application provides a color cast correction method, apparatus, device, system, and readable storage medium, which can solve the problem of low efficiency in current color cast correction methods.

[0006] The first aspect of this application provides a color cast correction method, comprising: acquiring a display image obtained by a first acquisition device capturing an image of a first display screen, the display image including at least one image region, each image region corresponding to a display sub-region of the first display screen, and each display sub-region forming a deflection angle with the first acquisition device; determining the current optical information of each image region; and determining a gain coefficient of the corresponding deflection angle based on the target optical information and the current optical information of each image region, the gain coefficient being used to perform color cast correction on an image acquired by a second acquisition device or an image displayed on a second display screen.

[0007] A color cast correction device provided in the second aspect of this application includes: an image acquisition unit for acquiring a display image obtained by a first acquisition device capturing an image of a first display screen, the display image including at least one image region, each image region corresponding to a display sub-region of the first display screen, and each display sub-region forming a deflection angle with the first acquisition device; an optical information determination unit for determining the current optical information of each image region; and a gain coefficient determination unit for determining a gain coefficient corresponding to the deflection angle based on the target optical information and the current optical information of each image region, the gain coefficient being used to perform color cast correction on the image acquired by the second acquisition device or the image displayed on the second display screen.

[0008] A third aspect of this application provides a display control device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the color cast correction method described above.

[0009] A fourth aspect of this application provides a color cast correction system, including an acquisition device and a display control device. The acquisition device is used to capture images of the display screen to obtain a display image; the display control device is used to acquire the display image and process the display image according to the color cast correction method described in the first aspect to obtain a gain coefficient.

[0010] A fifth aspect of this application provides a virtual shooting system, including a data acquisition device, a positioning device, a reference display screen, a display screen to be calibrated, and a display control device. The reference display screen and the display screen to be calibrated may be the same or different. The data acquisition device is used to capture images of the reference display screen to obtain a display image. The positioning device is used to acquire position information of the data acquisition device and the reference display screen, the position information including virtual position information in a virtual model formed by the data acquisition device and the reference display screen, or physical position information in a physical environment formed by the data acquisition device and the reference display screen. The display control device is used to acquire the display image of the reference display screen, process the display image of the reference display screen according to the color cast correction method described in the first aspect, obtain a gain coefficient table, and send the gain coefficient table to the data acquisition device to perform color cast correction on the image acquired by the data acquisition device, or send the gain coefficient table to the display screen to be calibrated to perform color cast correction on the image displayed on the display screen to be calibrated.

[0011] A sixth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the color cast correction method described above.

[0012] A seventh aspect of this application provides a computer program product that, when run on a display control device, causes the display control device to execute the color correction method described in the first aspect above.

[0013] In embodiments of this application, the display image obtained by the first acquisition device from the first display screen may include at least one image region, wherein each image region corresponds to a display sub-region of the first display screen, and each display sub-region forms a deflection angle with the first acquisition device. The display control device determines the gain coefficient of the corresponding deflection angle based on the current optical information and target optical information of each image region, and uses the gain coefficient to perform color cast correction on the image acquired by the second acquisition device or the image displayed on the second display screen. This allows the display image acquired in a single instance to obtain one or more gain coefficients for deflection angles through partitioning, thereby reducing the number of times the display image is acquired, improving the efficiency of gain coefficient generation, and thus improving the efficiency of color cast correction. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the color cast correction system provided in the embodiments of this application;

[0016] Figure 2 This is a schematic diagram illustrating the implementation process of a color cast correction method provided in an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of adjusting the pose of the first acquisition device provided in an embodiment of this application;

[0018] Figure 4 This is a schematic diagram of the display interface provided in an embodiment of this application;

[0019] Figure 5 This is a schematic diagram of the specific implementation process for determining the reference shape information provided in the embodiments of this application. Figure 1 ;

[0020] Figure 6 This is a schematic diagram of the specific implementation process for determining the reference shape information provided in the embodiments of this application. Figure 2 ;

[0021] Figure 7This is a schematic diagram illustrating the specific implementation process for determining the target gain coefficient provided in an embodiment of this application;

[0022] Figure 8 This is a schematic diagram of the structure of the virtual imaging system provided in the embodiments of this application;

[0023] Figure 9 This is a schematic diagram of the structure of a color cast correction device provided in an embodiment of this application;

[0024] Figure 10 This is a schematic diagram of the structure of the display control device provided in the embodiments of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0026] In related technologies, based on the display image obtained by the acquisition device from a certain shooting angle, the display control device can calculate the gain coefficient of a single deflection angle corresponding to that shooting angle. To meet the requirements of color cast correction, this method usually requires image acquisition at a large number of shooting angles to obtain a sufficient number of deflection angle gain coefficients, resulting in low efficiency in color cast correction.

[0027] In the embodiments of this application, by partitioning the display screen and partitioning the display images obtained by the acquisition device from the display screen, the gain coefficient of one or more deflection angles can be calculated from the display images acquired by the acquisition device in a single acquisition. This reduces the number of times the display images are acquired and improves the efficiency of gain coefficient generation and color cast correction.

[0028] To illustrate the technical solution of this application, specific embodiments are described below.

[0029] Figure 1 A schematic diagram of a color cast correction system provided in an embodiment of this application is shown. The color cast correction system 10 may include a data acquisition device 11 and a display control device 12.

[0030] The acquisition device 11 can be used to capture images of the display screen 13 (hereinafter also referred to as the first display screen) to obtain the displayed images.

[0031] Specifically, the acquisition device 11 can capture images of the entire display screen 13 or a portion of the display screen 13. The display screen 13 may or may not be part of the color correction system 10.

[0032] The display control device 12 can be used to acquire the display image captured by the acquisition device 11, and process the display image according to the color cast correction method provided in this application to obtain a gain coefficient. The gain coefficient refers to the coefficient used for color cast correction.

[0033] The aforementioned color cast correction system 10 can be applied to fields such as virtual photography or the metaverse. The acquisition device can capture images of the display screen to obtain an image to be processed against the background of the content displayed on the screen in real time. The color cast correction system 10 may also include an image processing device for image processing, which can fuse the image to be processed and the virtual background image to output a fused image. As can be seen from this example, virtual photography, the metaverse, and other fields use images captured against the background of the content displayed on the screen in real time. Therefore, the color cast phenomenon of the display screen actually manifests as color distortion in the image captured by the acquisition device.

[0034] Based on this, the gain coefficient mentioned in the embodiments of this application can be used to correct color cast of images displayed on a display screen (hereinafter referred to as the second display screen), or it can be used to correct color cast of images acquired by an acquisition device (hereinafter referred to as the second acquisition device). Regardless of the method, the result can solve the color distortion problem of the display screen in the images captured by the acquisition device.

[0035] It should be understood that Figure 1 This is merely an example of the color correction system 10 and does not constitute a limitation on the display control device. The color correction system 10 described above may also include more or fewer components. For example, the color correction system 10 may also include a positioning device for positioning the display screen 13 and the acquisition device 11, a video source device for providing a video source to the display screen 13, and so on.

[0036] It should be noted that the acquisition device mentioned in the embodiments of this application can be an industrial camera, video camera, or other image acquisition device, or a smart device with camera function such as a smartphone or tablet computer. The display screen mentioned in the embodiments of this application can be an LED display screen, an LCD display screen, a micro LED display screen, or other types of display screens. The display control device mentioned in the embodiments of this application can be a computer, mobile phone, or other smart device. For example, the aforementioned computer can refer to a server, which can control the display screen, acquisition device, and other devices, and perform color correction on the display screen. The display control device can also be a dedicated device for display screen calibration; for example, the display control device can be a controller configured with a sending card or a control device configured with a receiving card. In some embodiments, the display control device may specifically refer to a sending card or a receiving card. This application does not impose any limitations in this regard.

[0037] For details, please refer to Figure 2 , Figure 2 The illustration shows a schematic diagram of the implementation process of a color cast correction method provided in an embodiment of this application. This method can be applied to the above-mentioned display control device and is applicable to situations where it is necessary to improve the efficiency of color cast correction.

[0038] Specifically, the above color correction method may include the following steps S201 to S203.

[0039] Step S201: Obtain the display image captured by the first acquisition device on the first display screen.

[0040] Here, the first display screen refers to the display screen used to determine the gain coefficient. The first acquisition device refers to the acquisition device used to acquire data from the first display screen to determine the gain coefficient. The first acquisition device can obtain a display image used to determine the gain coefficient by taking a picture of the first display screen.

[0041] In embodiments of this application, the displayed image may include at least one image region, each image region corresponding to a display sub-region of the first display screen, and each display sub-region forming a deflection angle with the acquisition device. In other words, the first display screen can be divided into one or more display sub-regions. Since these display sub-regions are located in different spaces, the deflection angles formed by different display sub-regions and the acquisition device may be the same or different, and each display sub-region occupies a different image region in the displayed image. Therefore, there is a one-to-one correspondence between the image region, the display sub-region, and the deflection angle. In the displayed image, the image content of an image region is a display sub-region on the display screen corresponding to that image region, and the pixel value of the pixel in that image region can characterize the luminous intensity of the display sub-region corresponding to that image region.

[0042] The deflection angle refers to the angle between the line connecting the first acquisition device and the display sub-area and the normal to the first display screen. Depending on the specific deflection direction, the deflection angle can be divided into horizontal deflection angle and vertical deflection angle. The deflection angle can be determined based on the location of the first acquisition device, the location of the display sub-area, and the location of the first display screen.

[0043] It should be noted that the division method of the display sub-region can be adjusted according to the actual situation. For example, the required division method can be selected based on the difficulty of determining the image region, the accuracy of the gain coefficient, the efficiency of color correction, the luminous characteristics of the first display screen, etc.

[0044] As one example, each cabinet in the first display screen can be considered a sub-region. Using this division, the display control device can identify the corresponding image region from the displayed image based on the cabinet's outline, simplifying image region determination. As another example, each LED in the first display screen can be considered a sub-region. This division allows the display control device to obtain gain coefficients for more deflection angles at once, improving color correction efficiency. Other examples include using each LED panel in the first display screen as a sub-region, or using the entire first display screen as a sub-region.

[0045] In order to acquire a display image, before the first acquisition device captures the display image, the display control device can adjust the parameters of the first acquisition device to a preset reference threshold, so that the imaging of the first acquisition device meets the imaging requirements. The adjusted parameters include, but are not limited to, the aperture, focal length, and macro focus of the first acquisition device. Among them, the aperture can be used to adjust the brightness of the image, the focal length can be used to adjust the image size, and the macro focus can be used to adjust the blur level of the image.

[0046] After the parameters of the first acquisition device are adjusted, the display control device can control the first display screen to display a solid color pattern of the target color in order to obtain the display image when the first display screen displays a solid color pattern.

[0047] The target color can be red, blue, green, white, or other colors. Preferably, the target color can be white, in which case each LED on the first display screen will emit its brightest light, reducing the impact of external factors such as ambient light on color correction.

[0048] Step S202: Determine the current optical information for each image region.

[0049] The current optical information refers to the optical information of each image region in the displayed image before color correction. Optical information, also known as luminous and color information, can include one or more of the following: color information, luminous flux information, luminous intensity information, luminance information, and chromaticity information. Color information characterizes the color presented by the corresponding display sub-region of the image region. Luminous flux information characterizes the luminous flux per unit area within the corresponding display sub-region of the image region. Luminous intensity information characterizes the luminous intensity within the corresponding display sub-region of the image region. Luminance information characterizes the brightness of the corresponding display sub-region of the image region. Chromaticity information characterizes the hue and / or saturation of the color displayed in the corresponding display sub-region of the image region. The optical information of each image region can also be specifically divided into the optical information of the red channel, blue channel, and green channel.

[0050] Specifically, according to the division method of the display sub-regions in the first display screen, the display control device can use algorithms such as contour recognition, feature recognition, and threshold segmentation to divide the display image into image regions corresponding to the display sub-regions. Based on the pixel values ​​of the pixels in each image region, the display control device can determine the optical information of the corresponding image region.

[0051] As an example, a display control device can calculate the average channel value of all pixels in a given image region on the R channel. Average channel value on the G channel and the average value of the channel on channel B. Then, and This serves as the current optical information for that image region.

[0052] Step S203: Determine the gain coefficient of the corresponding deflection angle based on the target optical information and the current optical information of each image region.

[0053] Here, target optical information refers to the target value that the optical information of the image area needs to reach after color cast correction. Specifically, target optical information can refer to the optical data of the image area where the light point is located in the image when the first acquisition device takes a picture of the light point in the normal direction of the light point, that is, the optical information corresponding to the image area with a deflection angle of 0°.

[0054] In some embodiments, the display control device can acquire a reference image and determine the target optical information using the reference image. The reference image may refer to the front view of the first display screen, and the display control device can use the optical information of the image area located in the center of the front view as the target optical information. In other embodiments, the display control device may also acquire target optical information set by the user.

[0055] Based on the target optical information and the current optical information of a certain image region, the display control device can determine the gain coefficient of the deflection angle formed by the display sub-region corresponding to that image region and the first acquisition device. Based on the target optical information and the current optical information of different image regions, the gain coefficient of the deflection angle formed by the display sub-region and the first acquisition device corresponding to each image region can be calculated.

[0056] The obtained gain coefficient can be used to correct color cast of images acquired by the second acquisition device or displayed on the second display screen. Here, the second display screen refers to the display screen to be corrected for color cast, and the second acquisition device refers to the acquisition device used to correct color cast on the second display screen.

[0057] To ensure effective color correction, the display attributes of the second display screen and the first display screen must meet preset consistency conditions. Specifically, the second display screen and the first display screen can be the same display screen; or, the second display screen and the first display screen can be display screens from the same production batch; or, the brightness variation characteristics or luminance / color variation characteristics of the second display screen and the first display screen are consistent or the error is less than a preset threshold. The error threshold can be adjusted according to actual conditions. As an example, when the peak brightness of the second display screen and the first display screen is consistent or close (e.g., within ±50 nits), the difference in light decay characteristics between the first and second display screens within the target deflection angle range does not exceed ±5 nits, or ±10 nits, or other error thresholds. Light decay characteristics refer to the degree of brightness reduction of the display screen within the target deflection angle range compared to when there is no light decay (i.e., in the normal direction of the display screen). The difference in light decay characteristics between the first and second display screens is also the difference between their respective light decay characteristics. As another example, the difference in light decay characteristics between the first and second display screens within the target deflection angle range does not exceed ±5%, or ±10%, or other error thresholds.

[0058] It should be understood that the consistency conditions for display screen attributes are not limited to these. The consistency conditions for display screen attributes may also include: the same size, the same resolution, the same process (or manufacturing process), the same material, etc. This application does not limit this.

[0059] Similarly, the acquisition device attributes of the first acquisition device and the second acquisition device can also meet the preset consistency conditions. Specifically, the first acquisition device and the second acquisition device can be the same acquisition device, acquisition devices from the same production batch, or acquisition devices with the same structure and imaging principle. This application does not impose any restrictions in this regard.

[0060] In embodiments of this application, the display image obtained by the first acquisition device from the first display screen may include at least one image region, wherein each image region corresponds to a display sub-region of the first display screen, and each display sub-region forms a deflection angle with the first acquisition device. The display control device determines the gain coefficient of the corresponding deflection angle based on the current optical information and target optical information of each image region, and uses the gain coefficient to perform color cast correction on the image acquired by the second acquisition device or the image displayed on the second display screen. This allows the display image acquired in a single instance to obtain one or more gain coefficients for deflection angles through partitioning, thereby reducing the number of times the display image is acquired, improving the efficiency of gain coefficient generation, and thus improving the efficiency of color cast correction.

[0061] For example, suppose multiple boxes need to be calibrated, and each box forms a different deflection angle with the first acquisition device. Using related technologies, it is necessary to take pictures of the boxes from multiple shooting angles to perform color cast correction for multiple boxes. However, using the color cast correction method provided in this application, the display image obtained from a single shot can be used to divide the image area corresponding to the display sub-regions of multiple deflection angles, and then the gain coefficients of multiple deflection angles can be calculated. Color cast correction for multiple boxes can be achieved with fewer image shots (e.g., only one shot is needed).

[0062] The process of determining the gain coefficient and the process of applying the gain coefficient are explained below.

[0063] First, the display control device can acquire some parameters involved in determining the gain coefficient. These parameters may include, but are not limited to, the resolution, size, and shape of the first display screen, the distance between the first display screen and the first acquisition device, the target coverage angle range, and the shooting deflection angle step.

[0064] Specifically, the resolution and size of the first display screen can be obtained from the screen configuration information of the control device for the first display screen, or they can be set by the user. The unit of resolution is usually a physical unit, while the unit of size can be either a physical unit or a virtual unit used to represent size in 3D modeling.

[0065] The distance between the first display screen and the first acquisition device can be obtained through a positioning system, positioning software, positioning device, or can be set by the user. Similarly, the distance can be determined by physical dimensions or by virtual units used to represent dimensions in 3D modeling.

[0066] The target coverage angle range refers to the coverage range of the deflection angle, such as +50° to -50°, +90° to -90°, or other ranges. The target coverage angle range can be set by the user or obtained through limit value calibration in a virtual shooting scene using the first acquisition device. For example, the target coverage angle range can be determined based on site limitations or shooting requirements.

[0067] Deflection angle step refers to the angular interval of the deflection angle. For example, the gain coefficient can be determined every 1 degree of deflection angle, or every 5 degrees of deflection angle. The deflection angle step can be set by the user according to the needs of color cast correction accuracy and efficiency.

[0068] In embodiments of this application, the first acquisition device can acquire and display images in different ways.

[0069] As an example, the display control device can control the first display screen to display a solid color pattern across the entire screen, and control the first acquisition device to acquire images of the entire first display screen. If the display is divided into sub-regions according to the cabinets, the deflection angles between each cabinet of the first display screen and the first acquisition device are different, so a single image capture can obtain optical information of image regions corresponding to multiple deflection angles.

[0070] As another example, the display control device can control the target cabinet in the first display screen to display a solid color pattern, and control the first acquisition device to acquire images of the target cabinet. If the display is divided into sub-regions according to the cabinet, then based on the deflection angle between the target cabinet and the first acquisition device, a single shot can obtain the optical information of the image area corresponding to that deflection angle.

[0071] In some implementations, the display control device can acquire a preset target coverage angle range and a preset deflection angle step, and calculate the deflection angle of the gain coefficient to be determined based on the target coverage angle range and the deflection angle step.

[0072] For example, assuming the target coverage angle range is +90° to -90° and the deflection angle step is 1°, the deflection angle for the gain coefficient to be determined can be +90°, +89°, +88°, ..., -89°, -90°.

[0073] Accordingly, the display control device can acquire the display image captured by the first acquisition device in a certain pose, and determine the deflection angle corresponding to each image region in the display image. If there is a deflection angle other than the first deflection angle among the deflection angles for which the gain coefficient to be determined, a new display image is acquired after the pose of the first acquisition device is adjusted, until the first deflection angle includes the deflection angle for each gain coefficient to be determined. Here, the first deflection angle is the deflection angle corresponding to the image region in the acquired display image.

[0074] In other words, if the display image acquired by the first acquisition device in a certain pose cannot cover all the deflection angles of the gain coefficients to be determined, the pose of the first acquisition device can be adjusted, and a new deflection angle can be obtained using the new display image, until all the acquired display images can cover all the deflection angles of the gain coefficients to be determined, and then the gain coefficient of each deflection angle of the gain coefficients to be determined can be obtained.

[0075] The calculation process for the deflection angle is as follows: Assuming the three-dimensional coordinates of the sub-area displayed on the first screen are (x1, y1, z1), and the coordinates of the first acquisition device are (x2, y2, z2), then the horizontal deflection angle θ formed between the first acquisition device and the sub-area is... The vertical deflection angle φ formed between the first acquisition device and the display sub-area is: Here, arctan is the inverse trigonometric sine function.

[0076] For example, the display control device can control the first acquisition device to move left and right respectively, starting from the shooting angle corresponding to the main view, to supplement the optical information of the image area corresponding to the deflection angle of the uncaptured gain coefficient. The display control device can also control the first acquisition device to move starting from the maximum positive angle or the minimum negative angle. Please refer to [reference needed]. Figure 3 Taking the movement starting from the minimum negative angle as an example, the first acquisition device can be moved until the deflection angle θ1 between the first acquisition device and the right side of the first display screen is less than or equal to the minimum deflection angle of the target coverage angle. The deflection angle θ2 between the first acquisition device and the left side of the first display screen at this time is calculated. If the deflection angle θ2 on the left side is greater than or equal to the maximum deflection angle of the target coverage angle, the shooting can be stopped. Otherwise, the first acquisition device is moved until the deflection angle θ1 between the first acquisition device and the right side of the first display screen at the current moment is equal to the deflection angle θ2 between the camera and the left side of the first display screen at the previous moment.

[0077] To facilitate image acquisition, the display control device can also acquire reference shape information of the target shooting angle. This reference shape information can be used to indicate the shape of the image captured by the third acquisition device at the target shooting angle.

[0078] Next, the pose of the first acquisition device is adjusted until the real-time shape information of the display image acquired by the first display screen on the first acquisition device meets the preset overlap condition with the reference shape information. Then, the first acquisition device is controlled to take a picture of the first display screen to obtain the display image.

[0079] Specifically, the target shooting angle refers to the shooting angle required for color cast correction. In the embodiments of this application, the target shooting angle can be adjusted according to the actual situation.

[0080] In some embodiments of this application, the display control device can adjust the pose of the first acquisition device so that the first acquisition device can acquire images of the display screen at a target shooting angle to obtain a display image at the target shooting angle. The first acquisition device and the third acquisition device can be the same or different acquisition devices. That is, the pose adjustment process of the first acquisition device can be instructed using the reference shape information obtained through the third acquisition device. In the case where the first and third acquisition devices are different acquisition devices, they can be acquisition devices of the same model, batch, or structure. In some embodiments, the third acquisition device can also be a virtual acquisition device; for example, the reference shape information is obtained by virtually drawing a third acquisition device in 3D modeling software, and the reference shape information is applied to the physical first acquisition device.

[0081] In embodiments of this application, real-time shape information can be used to indicate the shape of the display screen image obtained by the third acquisition device in real-time acquisition of the first display screen. It should be understood that the first display screen with the same shape will appear to have different shapes when viewed from different angles. Therefore, when the third acquisition device acquires data from the first display screen at different shooting angles, the shape of the acquired display screen image will have some differences.

[0082] During the pose adjustment of the first acquisition device, the display control device can detect in real time whether the real-time shape information and the reference shape information meet the preset overlap conditions.

[0083] When the real-time shape information and the reference shape information meet the overlap condition, it indicates that the shape of the image of the display screen obtained by the first acquisition device in real-time acquisition of the first display screen is exactly the same as or nearly the same as the shape of the image of the first display screen obtained by the third acquisition device at the target shooting angle. This means that the observation angle of the first acquisition device on the first display screen and the observation angle corresponding to the target shooting angle are the same or similar. Therefore, the shooting angle at which the first acquisition device acquires the image of the first display screen in this pose is the target shooting angle, or a shooting angle close to the target shooting angle. In other words, the relative angle between the first acquisition device and the first display screen after pose adjustment is the same as the target shooting angle, or the error between the relative angle between the first acquisition device and the first display screen after pose adjustment and the target shooting angle is less than a preset threshold. Here, the relative angle can refer to the shooting angle at which the first acquisition device captures the image of the first display screen in the corresponding pose. The preset threshold for the error between the relative angle and the target shooting angle can be set according to the relative distance or step accuracy between the first display screen and the first acquisition device, or it can be user-defined. For example, the preset threshold can be set to ±1°, ±3°, or ±5° of the target shooting angle, etc.

[0084] Specifically, in some embodiments, the aforementioned display control device can display a reference legend and a real-time legend on the display interface. The reference legend may include the aforementioned reference shape information at the target shooting angle, and the real-time legend may include the aforementioned real-time shape information captured in real time. The display format of the reference legend and the real-time legend can be selected according to the actual situation. For example, the display control device can display the positions of each vertex of the shape of the image on the display screen in the form of indicator points within the display interface. Another example is that the display control device can display the shape of the image on the display screen in the form of an indicator frame within the display interface; this indicator frame can be a closed indicator frame or an open indicator frame. It should be understood that this application is not limited to these, and other display formats are also applicable to this application.

[0085] Please refer to Figure 4 During the adjustment of the pose of the first acquisition device, the aforementioned display control device can update the real-time legend in the display interface based on changes in real-time shape information, and detect in real-time whether the reference legend and the real-time legend meet preset overlap conditions to confirm whether the real-time shape information and the reference shape information meet preset overlap conditions. When the real-time shape information and the reference shape information meet the preset overlap conditions, the display control device can also generate an instruction message to instruct the user to stop the pose adjustment.

[0086] When the real-time shape information and the reference shape information do not meet the overlap condition, it indicates that there is a significant deviation between the shape of the display screen image obtained by the first acquisition device from the real-time acquisition of the first display screen and the shape of the display screen image obtained by the third acquisition device at the target shooting angle. This means that the observation angle of the first acquisition device on the first display screen and the observation angle corresponding to the target shooting angle are different. Therefore, there is a significant difference between the shooting angle of the first acquisition device acquiring the first display screen in this pose and the target shooting angle. In other words, the error between the relative angle between the first acquisition device and the first display screen after pose adjustment and the target shooting angle is greater than or equal to a preset threshold.

[0087] It should be noted that this application does not limit the specific implementation method of pose adjustment. In some embodiments, the display control device can adjust the fixing device of the first acquisition device, so that the first acquisition device changes its pose when the fixing device is adjusted. In other embodiments, the display control device can display pose adjustment indication information to prompt the user to adjust the position of the first acquisition device.

[0088] After the pose adjustment of the first acquisition device is completed, the display control device can acquire images from the first display screen through the first acquisition device to obtain the display image at the target shooting angle.

[0089] When there are multiple target shooting angles, the aforementioned display control device can determine the reference shape information for each target shooting angle and sequentially acquire images from each target shooting angle. Based on the images from different target shooting angles, the display control device can perform multi-view reconstruction and model the luminous characteristics to help correct the color shift problem of the display screen caused by different shooting angles.

[0090] In the embodiments of this application, by acquiring the reference shape information of the target shooting angle, since the reference shape information is used to indicate the shape of the display image obtained by the third acquisition device from the first display screen at the target shooting angle, the pose of the first acquisition device is adjusted until the real-time shape information of the display image obtained by the first acquisition device from the first acquisition device and the reference shape information meet the preset overlap conditions. Then, the shooting angle of the first acquisition device in this pose is basically consistent with the target shooting angle, which improves the accuracy of the image shooting angle and enables the optical information of the first display screen at the viewpoint corresponding to the target shooting angle to be acquired more accurately.

[0091] To facilitate the acquisition of reference shape information, in some embodiments of this application, the display control device can obtain reference shape information by transforming the acquired initial shape information.

[0092] Specifically, such as Figure 5 As shown, in some embodiments, the process of obtaining reference shape information includes the following steps S501 to S503.

[0093] Step S501: Obtain the initial shape information of the initial shooting angle.

[0094] The initial shape information is used to transform and obtain the reference shape information. It can be used to indicate the shape of the image of the display screen obtained by the third acquisition device from the first display screen at the initial shooting angle. In some embodiments, the initial shape information can be obtained by the third acquisition device from the first display screen at the initial shooting angle, or calculated based on the physical properties of the first display screen itself.

[0095] The initial shooting angle can be selected according to the actual situation. For ease of operation, the preferred initial shooting angle is the shooting angle corresponding to the main view. The shooting angle corresponding to the main view refers to the shooting angle when the optical axis of the third acquisition device passes through the center of the display screen and is perpendicular to the display screen.

[0096] In some embodiments of this application, the display control device can acquire a reference image obtained by the third acquisition device from the first display screen at an initial shooting angle, determine the area information of the first display screen in the reference image, and determine the initial shape information based on the area information.

[0097] The region information can be used to characterize the outline of the display image in the reference image, and may include the coordinate range, area, and coordinate position of a specific point (such as a vertex) in the image region where the first display is located in the reference image.

[0098] Specifically, the display control device can control the third acquisition device to capture an image of the first display screen at an initial shooting angle to obtain a reference image. Then, through contour recognition, feature point recognition, threshold segmentation, or other algorithms, the device extracts the contour of the display screen from the reference image to obtain region information. Correspondingly, since the region information can be used to characterize the contour of the display screen, that is, to characterize the shape of the image formed by the display screen, the display control device can use the region information as initial shape information.

[0099] In some other embodiments of this application, the display control device may also acquire the physical properties of the first display screen and determine the initial shape information of the first display screen in the front view based on the physical properties.

[0100] The aforementioned physical attributes may include resolution information and contour information of the first display screen. The contour information may characterize the shape and aspect ratio of the first display screen, while the resolution information may include the resolution (or size) of the first display screen and its resolution percentage in the reference image. Based on the contour information and resolution information, the display control device can generate initial shape information. For example, based on the aspect ratio and resolution of the first display screen, the display control device can generate the shape of the image of the first display screen in the main view.

[0101] In other embodiments, the initial shape information can be generated in conjunction with a reference image. For example, the center of the reference image can be used as the center of the display image, and the region information of the image area where the display image is located can be determined based on the aspect ratio and resolution of the display screen, so as to use the region information as the initial shape information. Considering that the initial shape information is used to transform and obtain the reference shape information, in order to avoid the display image corresponding to the reference shape information after transformation from exceeding the image range, the length of the display image in the short side direction of the reference image can be less than or equal to half of the short side of the reference image.

[0102] Step S502: Obtain the angular deflection relationship between the initial shooting angle and at least one shooting angle.

[0103] Step S503: Determine at least one reference shape information based on the angle deflection relationship, the distance information between the display screen and the third acquisition device, and the initial shape information.

[0104] Here, the shooting angle refers to the shooting angle required by the first acquisition device to acquire data from the display screen. Each reference shape information corresponds to a shooting angle, and the target shooting angle is any one of the shooting angles.

[0105] Specifically, the aforementioned angle deflection relationship can include a deflection axis and a deflection angle. It should be understood that the target shooting angle can be considered as the shooting angle obtained after the third acquisition device deflects along the deflection axis until the deflection angle is reached. The deflection axis can refer to the horizontal or vertical centerline of the display screen. When the initial shooting angle corresponds to the shooting angle of the main view, the deflection angle can refer to the angle between the line connecting the third acquisition device and the center of the display screen and the normal direction of the display screen. Depending on the specific deflection direction, the deflection angle can be divided into a horizontal deflection angle and a vertical deflection angle. The display control device can calculate the deflection axis and deflection angle based on the relative relationship between the display screen and the third acquisition device, the target shooting angle, and the initial shooting angle; alternatively, it can obtain the deflection axis and deflection angle set by the user.

[0106] Specifically, the baseline shape information can be represented as M multiplied by the initial shape information. Here, M represents the transformation matrix, which is related to the adjustment operations performed on the initial shape information based on dimensions such as the distance, deflection axis, and deflection angle between the third acquisition device and the display screen in the camera coordinate system. These adjustment operations include, but are not limited to, one or more of translation, rotation, and scaling.

[0107] It should be understood that the distance between the display screen and the third acquisition device can be in the world coordinate system, and the unit can be meters. However, in the camera coordinate system of the third acquisition device, the unit of the distance between the display screen and the third acquisition device is pixels, and it is proportional to the distance in the world coordinate system.

[0108] In some implementations, the display control device can determine the reference shape information using the following formula:

[0109]

[0110]

[0111] Where (x,y) is the position of the vertex of the display screen in the camera coordinate system in the initial shape information, (x2,y2) is the position of the vertex of the display screen in the camera coordinate system in the reference shape information, and r is the distance between the display screen and the third acquisition device in the camera coordinate system.

[0112] The rotation transformation of the display screen is processed in the camera coordinate system. Assuming that the points (x, y) in the initial shape information are all displayed on the plane z = r, and assuming that the distance between the third acquisition device and the first display screen before rotation is r, that is, the coordinates of the corresponding points in the camera coordinate system are (x, y, r), then the corresponding points after rotation transformation are (x1, y1, z1). Since the image is two-dimensional, the transformed points (x1, y1, z1) still need to be projected back to the plane z = r for display. Therefore, x1 and y1 need to be scaled and projected according to the ratio of r and z1. At this time, the corresponding points (x2, y2) in the reference shape information can be obtained.

[0113] As an example, taking the vertical centerline of the display screen as the deflection axis, and θ as the horizontal deflection angle, then the transformation matrix...

[0114]

[0115] As another example, using the horizontal centerline of the display screen as the deflection axis. Let the vertical deflection angle be represented by the transformation matrix.

[0116]

[0117] In some embodiments, the aforementioned reference shape information may include a set of target vertex coordinates for the shape of the display image obtained by the third acquisition device from the target shooting angle. The initial shape information may include a set of initial vertex coordinates for the shape of the display image obtained by the third acquisition device from the initial shooting angle. That is, in step S503, at least one set of target vertex coordinates can be determined based on the angle deflection relationship, the distance information between the display and the third acquisition device, and the initial vertex coordinate set.

[0118] The vertex coordinate set can contain one or more preset points. These preset points can be any point on the display screen, such as the vertex, center point, or other points on the screen. For example, when the reference shape information is displayed as a non-closed indicator box, the preset points can also be the endpoints of the edges of the non-closed indicator box. Based on the preset points in the initial vertex coordinate set, another preset point in the target vertex coordinate set can be obtained by considering the angle deflection relationship and the distance information between the display screen and the third acquisition device. For example, based on the coordinates of the top-left vertex of the display screen in the initial vertex coordinate set, the coordinates of the top-left vertex of the display screen in the target vertex coordinate set can be obtained.

[0119] In other implementations, such as Figure 6 As shown, the process of obtaining the reference shape information includes the following steps S601 to S603.

[0120] Step S601: Obtain the initial shape information of the initial shooting angle.

[0121] Step S602: Obtain the angular deflection relationship between the initial shooting angle and at least one shooting angle.

[0122] The specific implementation methods of steps S601 and S602 can be found in [reference]. Figure 5 The explanations provided are not repeated here.

[0123] Step S603: Determine at least one reference shape information based on the angle deflection relationship, the angle information formed by the vertex of the display screen and the third acquisition device under the shooting angle corresponding to the main view, and the initial shape information.

[0124] Specifically, the angle information may include the angle formed by the line connecting the vertex of the display screen and the third acquisition device and the plane on which the display screen is located, or the deflection angle formed by the line connecting the vertex of the display screen and the third acquisition device and the normal of the display screen. Based on the angle information of at least two vertices, mathematical modeling can be performed to obtain the distance information between the display screen and the third acquisition device. At this point, according to the method in step S503, at least one reference shape information can be determined using the angle deflection relationship, the distance information, and the initial shape information.

[0125] In the embodiments of this application, the initial shape information of the initial shooting angle, which is relatively easy to obtain, can be used to convert the reference shape information of the target shooting angle, which is more difficult to obtain, thereby improving the ease of obtaining the reference shape information. When it is necessary to acquire images from multiple shooting angles, the reference shape information of each shooting angle can be converted using the same initial shape information, resulting in higher acquisition efficiency.

[0126] After acquiring a display image that covers the deflection angle to be determined, the gain coefficient can be determined using the target optical information and the current optical information of the image area.

[0127] Specifically, the gain coefficient can be linearly calculated based on the current optical information of the image region, the optical information of the image region when the first display screen is off, and the target optical information. Assuming the current optical information of a certain image region in the R, G, and B channels is (105, 95, 103), the optical information of the image region in the R, G, and B channels when the first display screen is off is (0, 0, 0), and the target optical information in the R, G, and B channels is (100, 100, 100), then the gain coefficient of the deflection angle corresponding to this image region in the R, G, and B channels is: That is (1.05, 0.95, 1.03).

[0128] Based on the determined gain coefficient, the display control device can also generate a gain coefficient table to characterize the correspondence between the deflection angle and the gain coefficient, so that other devices can easily query and use the gain coefficient.

[0129] To cover the deflection angles for which the gain coefficients to be determined, in some other embodiments, the display control device may, after determining the gain coefficients and when there are deflection angles among the deflection angles for which the gain coefficients to be determined are not yet determined, acquire a new display image after the pose of the first acquisition device is adjusted, in order to determine the gain coefficient corresponding to the new deflection angle, until the gain coefficient for each deflection angle for which the gain coefficients to be determined is obtained. The specific process for determining the deflection angle and gain coefficient can be referred to the preceding description, and will not be repeated here.

[0130] Since different display images may correspond to the same deflection angle, different display images can be used to generate a gain coefficient at the same deflection angle. At this time, the display control device can also perform average calculation on multiple gain coefficients at the same deflection angle and use the average value as the gain coefficient of that deflection angle.

[0131] After obtaining the gain coefficient, the display control device can use the gain coefficient to perform color correction on the image acquired by the second acquisition device or the image displayed on the second display screen.

[0132] For details, please refer to Figure 7 The above color correction application process may include steps S701 to S702.

[0133] Step S701: Determine the target deflection angle formed between the target area to be corrected in the second display screen and the second acquisition device.

[0134] The second display screen (i.e., the display screen to be color-corrected) can also be divided into multiple display sub-regions. The display sub-region requiring color correction can be called the correction area. There can be one or more correction areas; for example, all or part of the display sub-regions exhibiting color distortion can be used as correction areas. The target correction area can refer to any one of the correction areas. The method for determining the target deflection angle can refer to the aforementioned explanation of deflection angle calculation. That is, the display control device can obtain the first position of the target correction area and the second position of the second acquisition device, and determine the target deflection angle based on the first and second positions.

[0135] Step S702: Determine the target gain coefficient of the target area to be corrected based on the target deflection angle and the gain coefficient of each deflection angle.

[0136] The target gain coefficient is the gain coefficient used for the target area to be corrected. Based on the target deflection angle formed by the target area to be corrected and the second acquisition device, the target gain coefficient of the target area to be corrected can be determined according to the gain coefficient corresponding to each deflection angle. It should be understood that the target gain coefficient may be the same or different for different target areas to be corrected, depending on the specific degree of the target deflection angle.

[0137] Specifically, if among the deflection angles corresponding to the gain coefficients determined in step S203, there exists a deflection angle with the same number as the target deflection angle, then the gain coefficient corresponding to that deflection angle can be used as the target gain coefficient for the target region to be corrected.

[0138] If there is no deflection angle with the same deflection angle as the target deflection angle among the deflection angles corresponding to the gain coefficients determined in step S203, the target gain coefficient of the target deflection angle can be determined by the known gain coefficient of the known deflection angle.

[0139] In some implementations, the gain coefficient of the deflection angle closest to the target deflection angle can be used as the target gain coefficient. When there are multiple deflection angles closest to the target deflection angle, for example, when the target deflection angle is the median of deflection angles A and B, the gain coefficient of any one of deflection angles A and B can be used as the target gain coefficient.

[0140] In other implementations, the target gain coefficient of the target region to be corrected can be calculated through mathematical modeling. The mathematical modeling method can be an interpolation algorithm (such as linear interpolation or nonlinear interpolation), a fitting algorithm (such as curve fitting), or other methods that can estimate the gain coefficient of the target deflection angle using a known gain coefficient with a known deflection angle.

[0141] Specifically, the display control device can select a first gain coefficient and a second gain coefficient from the gain coefficients corresponding to each deflection angle, and perform mathematical modeling on the first and second gain coefficients to obtain the target gain coefficient. The deflection angle corresponding to the first gain coefficient is greater than the target deflection angle, and the deflection angle corresponding to the second gain coefficient is less than the target deflection angle.

[0142] Taking bilinear interpolation as an example, based on the target deflection angle, the deflection angle corresponding to the first gain coefficient, and the deflection angle corresponding to the second gain coefficient, the weights can be calculated. Multiplying the weights by their corresponding gain coefficients and summing them yields the target gain coefficient. Assuming the target region to be corrected has a horizontal deflection angle of +43° and a vertical deflection angle of +31°, and given that the gain coefficients for a horizontal deflection angle of +42° and a vertical deflection angle of +30° are 1.15, 1.2, 1.19, and 1.25 respectively, the target gain coefficient can be...

[0143] Taking curve fitting as an example, based on the target deflection angle, the deflection angle corresponding to the first gain coefficient, and the deflection angle corresponding to the second gain coefficient, the function curve expression with the deflection angle as the independent variable and the gain coefficient as the dependent variable can be determined. Substituting the target deflection angle into the function curve expression yields the target gain coefficient.

[0144] Accordingly, the display control device can perform partitioned calibration on the second display screen according to the target area to be calibrated. That is, the gain coefficient of each target area to be calibrated is applied to the corresponding position of the image displayed on the second display screen or the corresponding position of the image captured by the second acquisition device.

[0145] It should be understood that the gain coefficient determined above is a gain coefficient calculated based on the optical information of the image region. Optical information is a linear domain data, so the gain coefficient is a coefficient that acts on linear domain data. If the gain coefficient needs to be applied to nonlinear domain data, the display control device also needs to convert it into a gain coefficient that is actually applied to nonlinear domain data.

[0146] Specifically, let's take the gain coefficient of the image acquired by the second acquisition device as an example. The process of the second acquisition device acquiring an image involves sensing light signals through a photosensitive element and converting these signals into digital image data (such as raw data). After processing by the Image Signal Processing (ISP) unit, the raw image data yields electrical signal data. The raw image data may contain information necessary to create a visual image, such as the RGB values ​​of each pixel; it is uncompressed image data, i.e., linear domain data. The electrical signal data, on the other hand, is the final visual image data output by the second acquisition device, such as data in BMP or JPG formats; it is compressed image data, i.e., non-linear domain data.

[0147] Since the original image data acquired by the second acquisition device is linear domain data, in some implementations, if a gain coefficient is applied to the original image data, the display control device can send the gain coefficient to the second acquisition device to perform color correction on the original image data acquired by the second acquisition device.

[0148] Assume C R C G C B These are the gain coefficients of the RGB channels calculated in step S203, where r1, g1, and b1 are the RGB values ​​(original image data) of a pixel in the image acquired by the second acquisition device, and the corresponding gain coefficients are C. R C G C B Then, after color cast correction, r1', g1', and b1' are respectively r1' = C R ×r1、g1'=C G ×g1、b1'=C B ×b1.

[0149] Since the electrical signal data of the image acquired by the second acquisition device is nonlinear domain data, in some embodiments, if correction data is applied to this electrical signal data, the display control device can compress the gain coefficient according to the nonlinear curve of the second acquisition device and send the compressed gain coefficient to the second acquisition device to perform color correction on the electrical signal data of the image acquired by the second acquisition device. The compressed gain coefficient is also the gain coefficient actually applied to the electrical signal data of the image acquired by the second acquisition device.

[0150] Compression of the gain coefficient based on a nonlinear curve can be called nonlinear compression. Nonlinear compression can be the nonlinear electro-optic conversion that occurs when electrical signal data is applied to a display screen, or the nonlinear electro-optic conversion that occurs when linear light emission brightness / intensity captured by a second acquisition device is converted into an image. When the gain coefficient is applied to electrical signal data, photoelectric conversion compression is required. This photoelectric conversion is the inverse of the aforementioned electro-optic conversion. That is, assuming photoelectric conversion is f(x) and electro-optic conversion is g(x), then f(g(1)) = g(f(1)) = 1. Taking a common nonlinear compression curve based on gamma as an example, if the gain coefficients of the LED in the RGB channels are (1.05, 0.95, 1.03) and the gamma value is 2.2, then the compressed gain coefficient is Rounded to two decimal places, the values ​​are approximately (1.02, 0.98, 1.01). If the gamma value is 0, the nonlinear compression curve is a linear straight line, and the gain coefficient applied to the data stream is (1.05, 0.95, 1.03).

[0151] Taking the gain coefficient applied to the image displayed on the second display screen as an example, since the image displayed on the second display screen is essentially electrical signal data, if the gain coefficient is applied to the electrical signal data of the image displayed on the second display screen, the display control device can compress the gain coefficient according to the non-linear curve of the second display screen and send the compressed gain coefficient to the second display screen to perform color correction on the electrical signal data of the image displayed on the second display screen. The compressed gain coefficient is also the gain coefficient actually applied to the electrical signal data of the image displayed on the second display screen.

[0152] Specifically, the gain coefficient of the image displayed on the second screen should first be processed by the camera calibration matrix and then subjected to nonlinear compression.

[0153] The specific processing steps can be represented as follows:

[0154] in, For the camera calibration matrix, C R C G C B The gain coefficients c are calculated in step S203. r c g c b These are the gain coefficients of the RGB channels after nonlinear compression.

[0155] Assume C R C G C B The gain coefficients of the RGB channels calculated in step S203 are c, respectively. r '、c g '、c b 'respectively from C R C G C B The gain coefficients of the RGB channels obtained after camera calibration matrix processing, c r c g c b They are respectively composed of c r '、c g '、c b The gain coefficients of the RGB channels obtained after nonlinear compression. Assume r2, g2, and b2 are the RGB values ​​(electrical signal data) of a pixel on the image displayed on the second screen, and the corresponding gain coefficients are c. r c g c b Then, after color cast correction, r2', g2', and b2' are respectively r2' = c r ×r2、g2'=c g ×g2、b2'=cb ×b2.

[0156] In the embodiments of this application, the display control device can apply the gain coefficient to the image acquired by the second acquisition device or to the image displayed on the second display screen. When it is inconvenient for the user to adjust the second acquisition device, the user can choose to adjust the second display screen. Conversely, when it is inconvenient for the user to adjust the second display screen, the user can choose to adjust the second acquisition device, thereby improving the universality of color cast correction.

[0157] By using the aforementioned color cast correction method, the color distortion problem of the second display screen in the image output by the second acquisition device can be resolved. Accordingly, virtual photography can be performed using the color-corrected second acquisition device or the color-corrected second display screen.

[0158] Specifically, such as Figure 8 As shown, this application also provides a virtual imaging system 80, including an acquisition device, a positioning device, a reference display screen, a display screen to be calibrated, and a display control device.

[0159] The reference display screen may refer to the aforementioned first display screen, and the display screen to be calibrated may refer to the aforementioned second display screen. The reference display screen and the display screen to be calibrated may be the same display screen or different display screens, for example... Figure 8 The display screen in the system can serve as both a reference display screen and a display screen to be calibrated. The positioning device can be used to locate the acquisition device, the reference display screen, and the display screen to be calibrated; specifically, it can refer to a rangefinder, a camera tracking device, or a device capable of reading position data from a virtual 3D model.

[0160] Specifically, the acquisition device can be used to capture images of a reference display screen to obtain the displayed image.

[0161] The positioning device can be used to acquire the location information of the acquisition device and the reference display screen. The location information may include virtual location information in a virtual model composed of the acquisition device and the reference display screen, or physical location information in the physical environment composed of the acquisition device and the reference display screen.

[0162] The display control device can be used to acquire the display image of the reference display screen. Figure 2 The color cast correction method shown involves processing the image displayed on a reference display screen to obtain a gain coefficient table, and then sending the gain coefficient table to an acquisition device to correct the color cast of the image acquired by the acquisition device. Alternatively, the gain coefficient table can be sent to the display screen to be corrected to correct the color cast of the image displayed on that display screen.

[0163] For a detailed explanation of the color cast correction process, please refer to the previous text. Figures 1 to 7 The description of the subject matter will not be repeated here.

[0164] In addition, the virtual shooting system may also include an image processing device. This device acquires the image to be processed obtained by the acquisition device after color correction of the display screen, and processes the image to be processed to fuse it with the virtual background image, resulting in a merged image. It should be noted that the image processing device and the display control device can be the same device or different devices.

[0165] In the aforementioned virtual imaging method, the image to be processed used in virtual imaging is an image obtained after color cast correction, which can reduce the color distortion problem of the display screen in the displayed image and help improve the realism of virtual imaging.

[0166] It should be understood that Figure 8 This is merely an example of the virtual imaging system 80 and does not constitute a limitation on the display control device. The virtual imaging system 80 may also include more or fewer components. For example, the virtual imaging system may also include a light source to assist the display device in image acquisition. This application does not impose any limitations in this regard. Accordingly, the display control device and image processing device may also be used to control the light source, positioning system, and acquisition device. Furthermore, the image processing device may also be used to transmit the fused image to the display screen to be calibrated for display, or to other devices.

[0167] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders.

[0168] like Figure 9 The diagram shown is a structural schematic of a color correction device 900 provided in an embodiment of this application. The color correction device 900 is disposed on a display control device.

[0169] Specifically, the color correction device 900 may include:

[0170] The image acquisition unit 901 is used to acquire a display image obtained by the first acquisition device from the first display screen. The display image includes at least one image area, each image area corresponds to a display sub-area of ​​the first display screen, and each display sub-area forms a deflection angle with the first acquisition device.

[0171] An optical information determination unit 902 is used to determine the current optical information of each of the image regions;

[0172] The gain coefficient determination unit 903 is used to determine the gain coefficient of the corresponding deflection angle based on the target optical information and the current optical information of each of the image regions. The gain coefficient is used to perform color correction on the image acquired by the second acquisition device or the image displayed on the second display screen.

[0173] In some embodiments of this application, the color correction device 900 may further include a color correction unit, specifically used for: determining the target deflection angle formed between the target area to be corrected in the second display screen and the second acquisition device; and determining the target gain coefficient of the target area to be corrected based on the target deflection angle and the gain coefficient of each deflection angle.

[0174] In some embodiments of this application, the color correction unit described above can also be used to: select a first gain coefficient and a second gain coefficient from the gain coefficients corresponding to each deflection angle, wherein the deflection angle corresponding to the first gain coefficient is greater than the target deflection angle, and the deflection angle corresponding to the second gain coefficient is less than the target deflection angle; and perform mathematical modeling on the first gain coefficient and the second gain coefficient to obtain the target gain coefficient.

[0175] In some embodiments of this application, the color correction unit described above can also be used to: obtain a first position of the target area to be corrected and a second position of the second acquisition device; and determine the target deflection angle based on the first position and the second position.

[0176] In some embodiments of this application, the display attributes of the second display screen and the display attributes of the first display screen meet a preset consistency condition.

[0177] In some embodiments of this application, the second display screen and the first display screen are the same display screen; or, the second display screen and the first display screen are from the same production batch; or, the brightness variation characteristics or luminance and color variation characteristics of the second display screen and the first display screen are consistent or the error is less than a preset threshold.

[0178] In some embodiments of this application, the color cast correction unit may also be used to: send the gain coefficient to the second acquisition device to perform color cast correction on the original image data of the image acquired by the second acquisition device; or, perform nonlinear compression on the gain coefficient according to the nonlinear curve of the second acquisition device, and send the nonlinearly compressed gain coefficient to the second acquisition device to perform color cast correction on the electrical signal data of the image acquired by the second acquisition device; or, perform nonlinear compression on the gain coefficient according to the nonlinear curve of the second display screen, and send the nonlinearly compressed gain coefficient to the second display screen to perform color cast correction on the electrical signal data of the image displayed on the second display screen.

[0179] In some embodiments of this application, the color correction device 900 may further include a deflection angle determination unit, specifically used for: acquiring a preset target coverage angle range and a preset deflection angle step; calculating the deflection angle of the gain coefficient to be determined based on the target coverage angle range and the deflection angle step; the image acquisition unit 901 may be specifically used for: if there is a deflection angle other than the first deflection angle among the deflection angles of the gain coefficient to be determined, then after the pose of the first acquisition device is adjusted, acquiring a new display image until the first deflection angle includes the deflection angle of each of the gain coefficients to be determined, wherein the first deflection angle is the deflection angle corresponding to the image area in the acquired display image.

[0180] In some embodiments of this application, the above-mentioned correction unit can be specifically used to: if there is a deflection angle with an undetermined gain coefficient among the deflection angles of the gain coefficient to be determined, then after the pose of the first acquisition device is adjusted, a new display image is acquired to determine the gain coefficient corresponding to the new deflection angle, until the gain coefficient of each deflection angle of the gain coefficient to be determined is obtained.

[0181] In some embodiments of this application, the image acquisition unit 901 described above may be specifically used to: acquire reference shape information of the target shooting angle, wherein the reference shape information is used to indicate the shape of the display screen image obtained by the third acquisition device at the target shooting angle; adjust the pose of the first acquisition device until the real-time shape information of the display screen image obtained by the first acquisition device and the reference shape information meet a preset overlap condition; and control the first acquisition device to take a picture of the first display screen to obtain the display image.

[0182] In some embodiments of this application, the image acquisition unit 901 described above may be specifically used to: acquire initial shape information of an initial shooting angle, wherein the initial shape information is acquired by the third acquisition device at the initial shooting angle from the first display screen, or calculated based on the physical properties of the first display screen itself; acquire the angle deflection relationship between the initial shooting angle and at least one shooting angle; determine at least one reference shape information based on the angle deflection relationship, the distance information between the first display screen and the third acquisition device, and the initial shape information; or, determine at least one reference shape information based on the angle deflection relationship, the angle information formed by the vertex of the first display screen and the third acquisition device at the shooting angle corresponding to the main view, and the initial shape information.

[0183] It should be noted that, for the sake of convenience and brevity, the specific working process of the color correction device 900 described above can be found in the following reference: Figures 1 to 8 The corresponding process of the method will not be described in detail here.

[0184] like Figure 10 The diagram shown is a schematic of a display control device provided in an embodiment of this application. The display control device 10 may include: a processor 1000, a memory 1001, and a computer program 1002 stored in the memory 1001 and executable on the processor 1000, such as a color correction program. When the processor 1000 executes the computer program 1002, it implements the steps in the various color correction method embodiments described above, for example... Figure 2 The steps S201 to S203 are shown. Alternatively, when the processor 1000 executes the computer program 1002, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 9 The image acquisition unit 901, optical information determination unit 902, and gain coefficient determination unit 903 are shown.

[0185] The computer program can be divided into one or more modules / units, which are stored in the memory 1001 and executed by the processor 1000 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the display control device.

[0186] For example, the computer program can be divided into: an image acquisition unit, an optical information determination unit, and a gain coefficient determination unit. The specific functions of each unit are as follows: the image acquisition unit is used to acquire a display image obtained by the first acquisition device from the first display screen. The display image includes at least one image region, each image region corresponding to a display sub-region of the first display screen, and each display sub-region forming a deflection angle with the first acquisition device; the optical information determination unit is used to determine the current optical information of each image region; the gain coefficient determination unit is used to determine the gain coefficient of the corresponding deflection angle based on the target optical information and the current optical information of each image region. The gain coefficient is used to perform color cast correction on the image acquired by the second acquisition device or the image displayed on the second display screen.

[0187] The display control device may include, but is not limited to, a processor 1000 and a memory 1001. Those skilled in the art will understand that... Figure 10This is merely an example of a display control device and does not constitute a limitation on the display control device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the display control device may also include input / output devices, network access devices, buses, etc.

[0188] The processor 1000 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0189] The memory 1001 can be an internal storage unit of the display control device, such as a hard disk or RAM. The memory 1001 can also be an external storage device of the display control device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 1001 can include both internal and external storage units of the display control device. The memory 1001 is used to store the computer program and other programs and data required by the display control device. The memory 1001 can also be used to temporarily store data that has been output or will be output.

[0190] It should be noted that, for the sake of convenience and brevity, the structure of the above-mentioned display control device can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0192] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0193] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0194] In the embodiments provided in this application, it should be understood that the disclosed apparatus / display control device and method can be implemented in other ways. For example, the apparatus / display control device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0197] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0198] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A color cast correction method, characterized by, The method comprises the following steps: obtaining a display image of the first display screen captured by the first acquisition device through a single acquisition, wherein the display image comprises at least two image regions, each of the image regions corresponds to a display sub-region of the first display screen, and each of the display sub-regions forms a deflection angle with the first acquisition device; determining current optical information of each of the image regions; determining a gain coefficient of the corresponding deflection angle according to target optical information and the current optical information of each of the image regions, wherein the gain coefficient is used for color cast correction of an image captured by a second acquisition device or an image displayed by a second display screen, wherein the acquisition device attributes of the first acquisition device and the second acquisition device satisfy a preset consistency condition, and the first acquisition device and the second acquisition device are different acquisition devices, and the display screen attributes of the first display screen and the second display screen satisfy a preset consistency condition and are different display screens; determining a target deflection angle formed by a target to-be-corrected region in the second display screen and the second acquisition device according to a first position of the target to-be-corrected region in the second display screen and a second position of the second acquisition device, and determining a target gain coefficient of the target to-be-corrected region according to the target deflection angle and the gain coefficient of each of the deflection angles.

2. The color cast correction method of claim 1, wherein, The method comprises the following steps: selecting a first gain coefficient and a second gain coefficient from the gain coefficients corresponding to each of the deflection angles, wherein the first gain coefficient corresponds to a deflection angle greater than the target deflection angle, and the second gain coefficient corresponds to a deflection angle less than the target deflection angle; mathematically modeling the first gain coefficient and the second gain coefficient to obtain the target gain coefficient.

3. The color cast correction method of claim 1, wherein, The second display screen and the first display screen are display screens of the same production batch. Alternatively, the luminance variation characteristics or the light chrominance variation characteristics of the second display screen and the first display screen are consistent or the error is less than a preset threshold.

4. The color cast correction method according to any one of claims 1 to 3, wherein After the gain coefficient is determined according to the target optical information and the current optical information of each of the image regions, the color cast correction method further comprises the following steps: sending the gain coefficient to the second acquisition device to perform color cast correction on raw image data of an image captured by the second acquisition device; alternatively, performing nonlinear compression on the gain coefficient according to a nonlinear curve of the second acquisition device, and sending the nonlinearly compressed gain coefficient to the second acquisition device to perform color cast correction on electrical signal data of an image captured by the second acquisition device; alternatively, performing nonlinear compression on the gain coefficient according to a nonlinear curve of the second display screen, and sending the nonlinearly compressed gain coefficient to the second display screen to perform color cast correction on electrical signal data of an image displayed by the second display screen.

5. The color cast correction method according to any one of claims 1 to 3, wherein The method comprises the following steps: Obtaining reference shape information of a target shooting angle, the reference shape information being used to indicate a shape of display screen imaging obtained by the third acquisition device collecting the first display screen at the target shooting angle; Adjusting the pose of the first acquisition device until real-time shape information of display screen imaging obtained by the first acquisition device collecting the first display screen satisfies a preset coincidence condition with the reference shape information, and then controlling the first acquisition device to shoot the first display screen to obtain the display image.

6. The color cast correction method of claim 5, wherein, The obtaining of the reference shape information of the target shooting angle comprises: Obtaining initial shape information of an initial shooting angle, the initial shape information being obtained by the third acquisition device collecting the first display screen at the initial shooting angle or being calculated according to physical attributes of the first display screen itself; Obtaining an angle deflection relationship between the initial shooting angle and at least one to-be-shot angle; According to the angle deflection relationship, distance information between the first display screen and the third acquisition device, and the initial shape information, determining at least one reference shape information; Or, According to the angle deflection relationship, angle information formed by a vertex of the first display screen and the third acquisition device at a shooting angle corresponding to a main view, and the initial shape information, determining at least one reference shape information.

7. A color cast correction device, characterized by, The method comprises: An image acquisition unit is configured to acquire, through single collection, a display image obtained by a first acquisition device shooting a first display screen, the display image comprising at least two image regions, each of the image regions corresponding to a display sub-region of the first display screen, and each of the display sub-regions forming a deflection angle with the first acquisition device; An optical information determination unit is configured to determine current optical information of each of the image regions; A gain coefficient determination unit is configured to determine, according to target optical information and the current optical information of each of the image regions, a gain coefficient corresponding to each of the deflection angles, the gain coefficient being used for color cast correction of an image collected by a second acquisition device or an image displayed by a second display screen; wherein acquisition device attributes of the first acquisition device and the second acquisition device satisfy a preset consistency condition, and the first acquisition device and the second acquisition device are different acquisition devices, and display screen attributes of the first display screen and the second display screen satisfy a preset consistency condition, and the first display screen and the second display screen are different display screens; A color cast correction unit is configured to determine a target deflection angle formed by a target to-be-corrected region in the second display screen and the second acquisition device according to a first position of the target to-be-corrected region in the second display screen and a second position of the second acquisition device, and determine a target gain coefficient of the target to-be-corrected region according to the target deflection angle and the gain coefficient of each of the deflection angles.

8. A display control device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the color cast correction method according to any one of claims 1 to 6.

9. A color cast correction system characterized by comprising: The color cast correction system comprises an acquisition device and a display control device; The acquisition device is configured to shoot a display screen to obtain a display image; The display control device is configured to acquire the display image and process the display image according to the color cast correction method of any one of claims 1 to 6 to obtain the gain coefficient.

10. A virtual shooting system characterized by comprising: The system comprises a collection device, a positioning device, a reference display screen, a display screen to be corrected, and a display control device, wherein the reference display screen is different from the display screen to be corrected. The collection device is configured to capture the reference display screen to obtain a display image. The positioning device is configured to acquire position information of the collection device and the reference display screen, wherein the position information comprises virtual position information in a virtual model formed by the collection device and the reference display screen, or physical position information in a physical environment formed by the collection device and the reference display screen. The display control device is configured to acquire a display image of the reference display screen, process the display image of the reference display screen according to the color cast correction method of any one of claims 1 to 6 to obtain a gain coefficient table, and send the gain coefficient table to the collection device to correct the color cast of the image collected by the collection device, or send the gain coefficient to the display screen to be corrected to correct the color cast of the image displayed by the display screen to be corrected.

11. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is configured to implement the steps of the color cast correction method of any one of claims 1 to 6 when executed by a processor.

Images