Image processing method, device and system

Abstract

The present application discloses an image processing method, device and system. The method includes: obtaining the position information of the target image in the display screen when the target image is displayed on the display screen; determining the working position information of the image acquisition device when performing image acquisition at at least one position, and obtaining the compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire the display screen of the display screen; determining the compensation weight of the target image under different shooting directions based on the position information of the target image in the display screen, and determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set; compensating the target image according to the target compensation coefficient. The present application solves the technical problem in the related art that the process of modifying the configuration of the monitoring terminal is complicated and inefficient, which affects the flexibility of the entire monitoring system.

Description

Technical Field

[0001] The present application relates to the field of virtual photography, and more specifically, to an image processing method, device and system. Background Art

[0002] LED screens are used as display backgrounds in film and television related shooting. In scenarios where multiple screens are set up using the screens, color inconsistencies introduced by viewing angle deviations at different shooting positions can easily lead to shooting problems.

[0003] In the solution of the related art, the current position of the camera is obtained from the server, and the color-biased picture is obtained according to the LED at the current position. The server renders the picture to solve the color cast problem. Since the solution of the related art has the problem of large image delay in the server-side processing, especially when the camera moves quickly, the increased rendering speed of the server does not match the dynamic adjustment of the display screen properly, resulting in constrained shooting, or color difference after fast movement, or low accuracy of image rendering on the server side, resulting in improper image compensation.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device and system to at least solve the technical problem in the related art that the process of modifying the configuration of the monitoring terminal is complicated and inefficient, which affects the flexibility of the entire monitoring system.

[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, comprising: obtaining position information of a target image in a display screen when the target image is displayed on the display screen; determining working position information of an image acquisition device when performing image acquisition at at least one position, and obtaining a compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire a display screen of the display screen; determining a compensation weight of the target image in different shooting directions according to the position information of the target image in the display screen, and determining a target compensation coefficient based on the compensation weight and the first compensation coefficient set; and compensating the target image according to the target compensation coefficient.

[0007] Optionally, obtain position information of multiple calibration points in the image acquisition device, and determine the size information of the display screen, wherein the calibration point positions corresponding to the multiple calibration points correspond one-to-one to different shooting directions, and the size information is determined based on the number of pixels in the display screen; determine the target viewing angle corresponding to the position information of the multiple calibration points, wherein the target viewing angle is the angle formed by the shooting direction of the image acquisition device and the normal of the plane where the display screen is located when the image acquisition device is in different working positions; based on the difference information between the coordinate information and the size information corresponding to the position information of the target image in the display screen; determine the compensation weight based on the difference information and the target viewing angle.

[0008] Optionally, a difference operation is performed on the size information and the coordinate information corresponding to the position information of the target image in the display screen, and a compensation weight is determined according to the operation result and the target viewing angle.

[0009] Optionally, the first compensation coefficient set is a coefficient matrix, wherein each element in the coefficient matrix is ​​a compensation coefficient corresponding to each calibration point position information in the multiple calibration point position information; determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set, including: determining the second coefficient set based on all coefficients in the above-mentioned coefficient matrix and the compensation weight; determining the target compensation coefficient based on the average value of all coefficients in the second coefficient set.

[0010] Optionally, the multiple calibration points corresponding to the multiple calibration point position information at least include calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other.

[0011] Optionally, a compensation coefficient corresponding to the working position information is received from an external device; or, a compensation coefficient corresponding to the working position information is determined from a set of compensation coefficients pre-stored in a receiving device.

[0012] Optionally, a sample image acquired at a sampling point in a candidate sampling point set when the image acquisition device moves within the effective moving space is acquired; a quality evaluation index of the sample image is acquired; a target sampling point is selected from the candidate sampling point set based on the quality evaluation index, and the position of the target sampling point is used as working position information.

[0013] According to another aspect of an embodiment of the present application, an image processing device is also provided, including: an acquisition module, used to acquire position information of a target image in a display screen when the target image is displayed on the display screen; a first determination module, used to determine the working position information of the image acquisition device when performing image acquisition at different working positions, and acquire the compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set; a second determination module, used to determine the compensation weights under different shooting directions according to the position information of the target image in the display screen, and determine the target compensation coefficient based on the compensation weights and the first compensation coefficient set; a compensation module, used to compensate the target image according to the target compensation coefficient.

[0014] According to another aspect of an embodiment of the present application, an image processing system is also provided, including: an image source device, a sending device, and a receiving device, wherein: the image source device is used to provide an original image and send the original image to the sending device; the sending device is used to receive the original image and process the target image to obtain the target image; the receiving device is used to execute any one of the above-mentioned image processing methods.

[0015] In the embodiment of the present application, firstly, the position information of the target image in the display screen is obtained when the target image is displayed on the display screen; the working position information of the image acquisition device when performing image acquisition at at least one position is determined, and the compensation coefficient corresponding to the working position information is obtained to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire the image corresponding to the display screen of the display screen; the compensation weight of the target image under different shooting directions is determined according to the position information of the target image in the display screen, and the target compensation coefficient is determined based on the compensation weight and the first compensation coefficient set; the target image is compensated according to the target compensation coefficient. Thus, the technical problem that the process of modifying the configuration of the monitoring terminal is complicated and inefficient in the related art, which affects the flexibility of the entire monitoring system is solved.

[0016] In an embodiment of the present application, an image processing method is provided, including: obtaining the position information of the target image in the display screen when the target image is displayed on the display screen; determining the working position information of the image acquisition device when performing image acquisition at at least one position, and obtaining the compensation coefficient corresponding to the working position information, and obtaining a first compensation coefficient set, wherein the image acquisition device is at least used to acquire the image corresponding to the display screen of the display screen; determining the compensation weight of the target image under different shooting directions according to the position information of the target image in the display screen, and determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set; and compensating the target image according to the target compensation coefficient. In the process of fast movement during shooting, the image rendering speed is matched with the dynamic adjustment of the display screen, and the image is rendered with high precision at the same time, so that the image can be appropriately compensated under the premise of reducing the shooting cost, thereby solving the technical problem that the process of modifying the configuration of the monitoring terminal in the related technology is complicated and inefficient, which affects the flexibility of the entire monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] FIG. 1( a ) is a schematic diagram of the structure of an image processing method and device provided according to an embodiment of the present application;

[0019] FIG. 1( b ) is a schematic diagram of the structure of another image processing method and device provided according to an embodiment of the present application

[0020] FIG. 1( c ) is a schematic diagram of the structure of another image processing method and device provided according to an embodiment of the present application

[0021] Figure 2 is a flowchart of an image processing method provided according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a characteristic position of a camera provided according to an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of compensation information description according to a scheme provided in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of another compensation information description according to an embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of the working process of a position information conversion device in an image processing method device provided in an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of the relationship between the grayscale values ​​of an input and output image after a Gamma transformation according to an embodiment of the present application;

[0027] Figure 8 It is a structural schematic diagram of an image processing device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] In order to better understand the embodiments of the present application, the terms involved in the embodiments of the present application are explained as follows:

[0031] Server: Perform image rendering processing according to user needs, generate target images, and output them to the LED sending device.

[0032] Sending device: receives the images generated by the server, and sends the image data to the LED display screen according to the software configuration.

[0033] LED display screen: LED display screen is a large screen made of LED cabinets, and has an LED control system drive circuit inside, which receives the image data sent by the sending device and drives the LED for display.

[0034] Position information generator: a device mainly used to obtain the current position of the camera.

[0035] Camera: A combination of equipment consisting of one or more cameras to shoot LED display screens.

[0036] Example 1

[0037] In order to solve the technical problems in the related art that the process of modifying the configuration of the monitoring terminal is complicated and inefficient, which affects the flexibility of the entire monitoring system, the present application proposes an embodiment of a new image processing system, which can be applied to the image processing system shown in Figure 1(a), but is not limited to this.

[0038] Specifically, the image processing system provided by the embodiment of the present application is shown in FIG1( a ), and the system at least includes: an image source device 11, a sending device 12, and a receiving device 13, wherein:

[0039] The image source device 11 is used to provide an original image and send the original image to a sending device;

[0040] During the image processing process, the received original image is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, the brightness can be changed by dragging the brightness slider to ensure that the target image received by the receiving device has a better image display effect. Similarly, we can continue to locally modify the original image using similar adjustment methods as mentioned above until the desired visual effect is achieved.

[0041] The sending device 12 is used to receive the original image and process the original image to obtain the target image;

[0042] During the image processing process, the received original image is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, the brightness can be changed by dragging the brightness slider to ensure that the image received by the receiving device has a better display effect. We can use the same adjustment method to make local modifications to the original image until the desired visual effect is achieved.

[0043] The receiving device 13 is used to execute the image processing method provided in the embodiment of the present application. For example, Figure 2 The image processing method shown.

[0044] In the process of performing image processing, since it is also necessary to determine the position information of the image acquisition device and the camera shooting direction information (such as the lens direction) and convert the position information into coordinate information, the image processing system, as shown in Figure 1(b), also needs to include: an information generator 14, which is used to obtain the coordinate information corresponding to the working position information of the image acquisition device 15 when performing image acquisition at different working positions, and obtain the shooting direction of the image acquisition device. The information generator 14 can be located in the sending device (such as a sending card), or it can be a device relatively independent of the sending device.

[0045] In addition, in order to complete the data distribution, as shown in Figure 1(c), an information converter 16 may also be included. The information converter may be located in a sending device (such as a sending card) or may be a device relatively independent of the camera information of the sending device. The information converter may include the following processing units: an acquisition unit: acquiring the existing position information of one or more cameras from the outside; a camera information conversion compensation coefficient: this module converts the camera's position information into compensation data information of the LED display screen; a compensation coefficient sending module: sending the compensation coefficient to a receiving card device, which is a control unit of the LED control system that drives the LED light board.

[0046] Based on the above-mentioned image processing system, the embodiment of the present application further provides an image processing method for specifically describing the process executed by the position information generator in the image processing system. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0047] Figure 2 is a flow chart of an optional image processing method according to an embodiment of the present application, such as Figure 2 As shown, the method at least includes steps S202-S208, wherein:

[0048] Step S202, obtaining position information of the target image on the display screen when the target image is displayed on the display screen.

[0049] Specifically, after the image device source acquires the original image, the target image is determined from the original image and the target image is sent to a display screen. While the display screen displays the target image, the receiving device acquires the position information of the target image on the display screen, wherein the display screen may be a common LED display screen, the smallest display unit of the LED display screen is an LED light point, and each smallest display unit may correspond to different position coordinate information on the display screen. Therefore, the position coordinate information corresponding to the target image may be acquired on the LED display screen, and the receiving card receives the position coordinate information corresponding to the target image on the LED display screen.

[0050] When the target image is displayed on the display screen, the position information of the target image on the display screen can be expressed in the following implementation manner, but is not limited thereto: after receiving the original image, the original image is locally adjusted to obtain the target image, and the information of the target pixel is obtained according to the image data of the target image, and the information of the target pixel is the position information of the target pixel. The following will give an example to illustrate in detail how to determine the position information of the target pixel, which will not be repeated here.

[0051] For example: the commonly used LED display screen includes an LED display panel and LED lights. The LED lights on the LED display panel are distributed in a matrix. The pixel points distributed on the LED display panel are the same as the LED lights. They are also distributed in a matrix and have the characteristics of being horizontal and vertical. In a 300*300 multi-pixel matrix, the position coordinate information corresponding to the LED light is determined. The horizontal coordinate range can be any point in 1-300, and the vertical coordinate range can be any point in 1-300. The different position coordinate information of each LED light corresponds to the pixel point on the LED display panel. In addition, the position information of the target pixel is the two-dimensional coordinate information of the target pixel, that is, the horizontal coordinate information of the target pixel and the vertical coordinate information of the target pixel. Therefore, the position information coordinate of the target pixel can be obtained based on the position information coordinate of the pixel point on the LED display panel.

[0052] Step S204, determining the working position information of the image acquisition device when performing image acquisition at at least one position, and obtaining the compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire an image corresponding to a display screen of the display screen.

[0053] Specifically, the working position information of the image acquisition device when performing image acquisition at at least one position is first determined, and the working position information of the image acquisition device when performing image acquisition at at least one position is obtained through a position information generator, wherein the image acquisition device can be at least one of a camera, a camcorder, a scanner, and other devices with a camera function (mobile phone, tablet computer, etc.).

[0054] The working position information of the image acquisition device when performing image acquisition at at least one position can be determined by obtaining a sample image acquired at a sampling point in a candidate sampling point set when the image acquisition device moves within an effective moving space range; obtaining a quality evaluation index of the sample image; selecting a target sampling point from the candidate sampling point set according to the quality evaluation index, and using the position of the target sampling point as the working position information, wherein the effective moving space range is the maximum distance that the image acquisition device can move in each direction (in each direction in two or three dimensions). For example, the sampling point corresponding to the sample image with a quality evaluation index greater than a preset threshold is used as the above-mentioned target sampling point, and conversely, the sampling point corresponding to the sample image with a quality evaluation index less than the preset threshold is used as an invalid sampling point, that is, the position of the invalid sampling point is not used as the above-mentioned working position information, that is, in the process of processing the above-mentioned target image, image sampling is not performed at the invalid sampling point.

[0055] For example, the above-mentioned quality assessment indicators include but are not limited to at least one of the following: image integrity, image clarity, image distortion, etc., wherein image integrity has two meanings: when there is only one display screen (i.e., the background image is provided by one display screen), only the integrity of the image displayed on the display screen is detected; when there are multiple display screens, i.e., the images displayed by multiple display screens jointly provide the background image, it is necessary to detect the integrity of all display screen images, i.e., when the integrity of all display screens is greater than the integrity threshold, the sample image is considered to have passed the integrity detection. For example, in the images collected on each display screen, if the borders of all display screens are detected in the image, it is considered to have passed the integrity detection. Among them, before determining the image integrity, an image segmentation algorithm can be used to segment the border and non-border images, and then the segmented border image is detected. If the border image is not closed, it fails the integrity detection, otherwise it is considered to have passed the integrity detection. When determining the completeness of an image, the following methods can also be used, but are not limited to these: the image can be divided into multiple parts and multiple or all of the multiple parts can be selected as sub-parts to be processed; the color feature value of each sub-part is calculated according to the value of each sub-part in the color space; and the image completeness is calculated according to whether the color of each sub-part is similar to that of its adjacent sub-part, wherein the color of the sub-part and its adjacent sub-part is determined to be similar when the difference between the color feature values ​​of the sub-part and its adjacent sub-part is less than a first threshold. For example, the more sub-parts with similar colors, the more single the color, and the image can be considered incomplete. Here, the color of the sub-part and its adjacent sub-part can be determined to be similar when the difference between the color feature values ​​of the sub-part and its adjacent sub-part is less than a first threshold.

[0056] Optionally, a compensation coefficient corresponding to the working position information is received from an external device, wherein the external device includes a sending device or a server; or, a compensation coefficient corresponding to the working position information is determined from a compensation coefficient set pre-stored in a receiving device, wherein the compensation coefficient is based on the position information of multiple calibration points of the image acquisition device, and according to the compensation coefficient algorithm, the compensation coefficient required for each calibration point of the image acquisition device is calculated by using the sending device of the position information converter, and the compensation coefficient is a coefficient matrix, which has a large amount of data and can compensate for multiple calibration points at the same time. Alternatively, a mapping relationship between the calibration point position and the compensation coefficient is maintained on the external device side, and after the calibration point position is determined, the compensation coefficient corresponding to the calibration point position is determined using the mapping relationship.

[0057] In order to more accurately determine the specific compensation coefficient, according to the compensation coefficient algorithm preset in the receiving device, the mapping relationship between multiple calibration point positions and compensation coefficients is used to obtain the compensation coefficient in the mapping relationship corresponding to the calibration point, wherein the compensation coefficient algorithm can be a De-Mura compensation algorithm, which analyzes the pixel distribution of the display screen observed at the calibration point position, and then calculates the compensation coefficient through the De-Mura compensation algorithm based on the pixel data, but is not limited to this.

[0058] Among them, the receiving device may include: a compensation coefficient set storage device or a reservation device, the compensation coefficient set storage device is used to store the compensation coefficient set corresponding to the calibration point working position of the image acquisition device; and, a reservation device is used to reserve multiple calibration point positions of the image acquisition device and the corresponding compensation coefficient sets.

[0059] For example, taking the image acquisition device as a camera, the camera has 9 characteristic positions in space, and the position information generator calculates the real-time position of the camera based on the relationship between the camera and the characteristic positions, wherein the position information generator can be a module in the sending device or the receiving device, or can be a device relatively independent of the sending device or the receiving device. Figure 3 The calculation process of the real-time position of the above camera is described as follows:

[0060] When the user has completed the selection of the characteristic position, the position of 9 points is selected (this solution is not limited to 9 points, it can be any point); and the matrix compensation coefficient of the LED display screen corresponding to the characteristic point is obtained through algorithm or manual adjustment;

[0061] After the camera position is moved, the compensation coefficient corresponding to the coordinate position is determined and sent to the LED display screen according to the different coordinate positions of the camera;

[0062] When the camera is at the coordinate position (100, 100), after the coordinate (100, 100) is given to the sending device, the sending device will send the compensation coefficient corresponding to (100, 100) for the entire LED display screen to the LED display screen in real time;

[0063] When the camera moves to (2000, 100), after the coordinate (2000, 100) is given to the sending device, the sending device will send the compensation coefficient corresponding to (2000, 100) for the entire LED display screen to the LED display screen in real time.

[0064] Specifically, when shooting people and scenes indoors, using the display screen as the display background can reduce a lot of post-production costs. Use the camera to shoot the display screen to collect the display image on the screen. Since the camera shoots at different angles and directions, there will be a color cast problem. The common display screen is an LED display screen. The smallest display unit of the LED display screen is an LED light point. Each LED light point contains three independent light-emitting units, RGB. These light-emitting units have a certain spatial arrangement relationship, including: vertical one-word arrangement, horizontal one-word arrangement, triangular arrangement, etc. For example, when the light-emitting units are arranged as RGB from left to right horizontally, when observed from the left side of the LED display screen, the light intensity relationship of the display screen is R>G>B, introducing a phenomenon of being reddish; when observed from the right side, the light intensity seen is: R<G<B, then a bluish phenomenon is seen. It is necessary to determine the specific compensation coefficient according to the color cast situation of the image observed during specific shooting. That is, when the image acquisition device collects images of the display screen at different positions, the target images collected at different positions will have a color cast problem. Determine the compensation coefficient for the target image based on the working position information of the image acquisition device at different positions. The compensation coefficient refers to the multiple of the color difference of the target image to be processed. The compensation coefficient is a non-zero positive number. If the compensation coefficient is greater than 1, it means that the target image to be processed is backlit and too dark, and the brightness needs to be increased; if the compensation coefficient is less than 1, it means that the target image to be processed is frontlit and too bright and overexposed, and the brightness needs to be reduced.

[0065] Step S206, determine the compensation weight of the target image in different shooting directions according to the position information of the target image in the display screen, and determine the target compensation coefficient based on the compensation weight and the first compensation coefficient set. The weight is determined by the distance between the calibration point position of the image acquisition device and the target image. The distance is inversely proportional to the weight, that is, the greater the distance, the smaller the weight. For example, there are 4 points. The closer the pixel is to the calibration point corresponding to the shooting direction, the greater the compensation weight corresponding to that calibration point.

[0066] Optionally, obtain multiple calibration point position information in the image acquisition device, and determine the size information of the display screen, wherein the calibration point positions corresponding to the multiple calibration point position information correspond one-to-one to different shooting directions, wherein the size information can be determined based on the number of pixels in the display screen, and can also be determined by other measurement methods; determine the target viewing angle corresponding to the multiple calibration point position information, wherein the target viewing angle is the angle formed by the shooting direction of the image acquisition device and the normal of the plane where the display screen is located when the image acquisition device is in different working positions; based on the difference information between the coordinate information and the size information corresponding to the position information of the target image in the display screen; determine the compensation weight based on the difference information and the target viewing angle.

[0067] Specifically, when the image acquisition device acquires the picture displayed on the display screen, the image acquisition device can acquire pictures at different angles in different working positions. By matching the calibration points with the corresponding shooting directions one by one, the corresponding different shooting directions can be acquired by acquiring multiple calibration points. At the same time, when the acquisition devices corresponding to the multiple calibration points are shooting at different working positions, the angle between the image acquisition device and the normal of the display screen is determined and recorded as the target angle.

[0068] While obtaining the calibration point position information and the target viewing angle corresponding to the calibration point position information, determine the size information of the display screen. The size of a common LED display screen is affected by many factors, including area, display ratio, model, module size, cabinet size, etc. There are various models. The size of the cabinet can be determined according to the actual situation. The size of the display screen is determined based on the cabinet size. In the specific application process, it is not limited to this implementation method.

[0069] In order to save computing resources, the multiple calibration points corresponding to the multiple calibration point position information include at least calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other. The two straight lines can be lines connecting calibration points located in different diagonal directions, but are not limited thereto. For example, Figure 4 The straight line formed by (128, 0) and (128, 256) and the straight line formed by (0, 128) and (256, 128) are perpendicular to each other, and there are 4 compensation coefficients for the above 4 coordinate positions. The number of the above multiple marking points can be an even number or an odd number, which can be flexibly set according to the actual situation.

[0070] After determining the above display screen size information, the difference information between the coordinate information corresponding to the position information of the target image in the display screen and the display screen size information is also required; and the compensation weight is determined based on the difference information and the target viewing angle. In other words, when determining the above compensation weight, it can be determined based on the above difference information. However, it should be noted that it is not limited to this implementation method in the specific application process. For example, the above compensation weight can be pre-set, for example, the mapping relationship between the mark point and the weight is maintained in the database, and the corresponding compensation weight is determined based on the mapping relationship.

[0071] When determining the compensation weight based on the difference information and the target viewing angle, it can be implemented as follows, but is not limited thereto: performing a difference operation on the coordinate information corresponding to the size information and the position information of the target image in the display screen, and determining the compensation weight based on the operation result and the target viewing angle, wherein the compensation coefficients in the first compensation coefficient set are obtained based on the position information of multiple calibration points of the image acquisition device, and the target compensation coefficients are obtained based on the position information of the current target image, the size of the box, and the first compensation coefficient set, and the target compensation coefficient is a compensation coefficient that matches the target image determined by the position information of the current target image. The following will give a detailed example, which will not be repeated here.

[0072] In some optional embodiments of the present application, the first compensation coefficient set is a coefficient matrix, wherein each element in the coefficient matrix is ​​a compensation coefficient corresponding to each marking point in a plurality of marking points; the target compensation coefficient is determined based on the compensation weight and the first compensation coefficient set, for example, the second coefficient set is determined based on all coefficients in the coefficient matrix and the compensation weight; then the target compensation coefficient is determined based on the average value of all coefficients in the second coefficient set, specifically, each coefficient in the coefficient matrix is ​​multiplied by its corresponding weight, and then the average value of the operation results after the multiplication is taken, and the average value is used as the target compensation coefficient.

[0073] The following combination Figure 4 , Figure 5 , taking the LED cabinet with a 256*256 rectangular resolution as an example, two compensation schemes are explained. Figure 4 As shown, compensation scheme B is described. The four circles represent four coordinate positions, where each coordinate position has four compensation coefficients, and the adjustment coefficients have the following characteristics:

[0074] The data that needs to be able to represent the gradient is not necessarily 4, it may also be 2 or 1 data to represent the compensation gradient of the LED display from left to right or from top to bottom; the meaning of the gradient here is explained using 4 coefficients as an example, each 2 coefficients are a pair, and the 2 coefficients only represent the left and right ends, or the upper and lower ends, but the pixel by pixel will be gradually processed according to the coefficients corresponding to the two calibration points, that is, the corresponding gradient processing.

[0075] The adjustment parameters can be 3*3 matrix data or 1*3 brightness compensation;

[0076] For example: The adjustment parameter M11Coef of screen 1 obtained by the feature point (128,0)

[0077]

[0078] Adjustment parameter M12Coef of screen 1 from feature point (0,128)

[0079]

[0080] …

[0081] Adjustment parameter M22Coef of screen 1 from feature point (128,256)

[0082]

[0083] That is, the compensation coefficients are M11Coef, M12Coef, M21Coef, and M22Coef.

[0084] Depend on Figure 6 It can be seen that in the process of displaying the 256*256 box image, each point needs to obtain the position information of the current image, obtain the current image data based on the position information, and then perform Gamma conversion. In the next operation process, it is necessary to determine the coordinates (X, Y) of the current image based on the position information of the current image. According to XY, the data weight value corresponding to the current image data at the four coordinates is calculated, and then the matrix coefficient of the current image is obtained based on the weight value.

[0085] If the coordinates of the current image are X=128, Y=128; Width represents the width of the LED cabinet pixel, and Height represents the height of the LED cabinet pixel. Then the corresponding current adjustment coefficient calculation method is:

[0086]

[0087]

[0088]

[0089]

[0090] M=M11tmp1+M12tmp+M21tmp+M22tmp

[0091] According to the relationship between the current position information and the feature position information, the final compensation weight values ​​of A and B are calculated according to the angle values ​​to obtain the final realized data.

[0092] Then the M value corresponding to (128,128) is

[0093]

[0094] The image information is compensated according to the above gradient operation to obtain compensated first image data, which is output and displayed on the display screen to ensure that the display screen image captured by the camera at various angles and directions has no color deviation.

[0095] Step S208, compensating the target image according to the target compensation coefficient.

[0096] The compensation coefficients in the first compensation coefficient set are obtained based on the position information of multiple calibration points of the image acquisition device, the size information of the display screen, the first compensation coefficient set and the position information of the target image, and the target compensation coefficient is a unique compensation coefficient corresponding to the target image determined based on the relationship between the position information of the current target image and the position information of multiple calibration points and the current target angle. In order to ensure the compensation effect of the image, before compensating the target image according to the target compensation coefficient, the target image can also be processed as follows: the original grayscale image of the target image is obtained, and the original grayscale image is converted to obtain the target grayscale image, wherein the target grayscale image is exponentially related to the original grayscale image.

[0097] Specifically, the sensitivity of the human eye to the external light source is not linearly related to the input light intensity, but exponentially related. Under low illumination, the human eye can more easily distinguish the change in brightness. As the illumination increases, the human eye cannot easily distinguish the change in brightness. However, the camera sensitivity is linearly related to the input light intensity. Gamma conversion is used to enhance the first image data to ensure the accuracy of the first image data. Among them, Gamma conversion is a nonlinear operation on the grayscale value of the input image, so that the grayscale value of the output image is exponentially related to the grayscale value of the input image. Among them, the grayscale value of the input image ranges from 0 to 1, so it needs to be normalized first and then exponentialized.

[0098] Gamma transformation is used for image enhancement, which improves the details in dark areas. Simply put, through nonlinear transformation, the linear response of the image exposure intensity is changed to a response closer to the human eye's perception, that is, to correct bleached (camera exposure) or too dark (underexposed) images.

[0099] The relationship between the grayscale values ​​of the input and output images after Gamma transformation is as follows: Figure 7As shown in the figure: the horizontal axis is the input grayscale value, the vertical axis is the output grayscale value, the open downward curve is the input-output relationship when the gamma value is less than 1, and the open upward curve is the input-output relationship when the gamma value is greater than 1. It can be observed that when the gamma value is less than 1, the overall brightness of the image is improved, and the contrast of low grayscale is increased, which is more conducive to distinguishing image details at low grayscale values.

[0100] That can be summarized as follows:

[0101] When gamma>1, the grayscale of brighter areas is stretched, and the grayscale of darker areas is compressed to be darker, and the image becomes darker overall.

[0102] gamma<1, the grayscale of the brighter area is compressed, the grayscale of the darker area is stretched to be brighter, and the overall image becomes brighter;

[0103] Among them, the grayscale value is 0 for black and 255 for white.

[0104] In the embodiment of the present application, firstly, the position information of the target image in the display screen is obtained when the target image is displayed on the display screen; the working position information of the image acquisition device when performing image acquisition at at least one position is determined, and the compensation coefficient corresponding to the working position information is obtained to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire the image corresponding to the display screen of the display screen; the compensation weight of the target image under different shooting directions is determined according to the position information of the target image in the display screen, and the target compensation coefficient is determined based on the compensation weight and the first compensation coefficient set; the target image is compensated according to the target compensation coefficient. Thus, the technical problem that the related technology over-compensates or improperly compensates the image on the server side in the virtual shooting scene, or the server-side processing causes a large image delay is solved.

[0105] Example 2

[0106] According to an embodiment of the present application, an image processing device for implementing the above image processing method is also provided. Figure 8 As shown, it includes: an acquisition module 81, a first determination module 82, a second determination module 83, and a compensation module 84.

[0107] in:

[0108] An acquisition module 81 is used to acquire position information of a target image on the display screen when the target image is displayed on the display screen;

[0109] A first determination module 82 determines working position information of an image acquisition device when performing image acquisition at at least one position, and obtains a compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire an image corresponding to a display screen of the display screen;

[0110] Specifically, the working position information of the image acquisition device when performing image acquisition at at least one position is first determined, and the working position information of the image acquisition device when performing image acquisition at at least one position is obtained through a position information generator, wherein the image acquisition device can be at least one of a camera, a camcorder, a scanner, and other devices with a camera function (mobile phone, tablet computer, etc.).

[0111] Optionally, a compensation coefficient corresponding to the working position information is received from an external device, wherein the external device includes a sending device or a server; or, a compensation coefficient corresponding to the working position information is determined from a set of compensation coefficients pre-stored in the receiving device, wherein the compensation coefficient is based on the position information of multiple calibration points of the image acquisition device and the compensation coefficient algorithm is calculated by using the sending device of the position information converter to calculate the compensation coefficient required for each calibration point of the image acquisition device. The compensation coefficient is a coefficient matrix with a relatively large data volume, and multiple calibration points can be compensated simultaneously.

[0112] For example, a camera has 9 characteristic positions in space. The position information generator calculates the real-time position of the camera based on the relationship between the camera and the characteristic positions. The position information generator can be a module in the sending device or the receiving device, or a device relatively independent of the sending device or the receiving device. Figure 3 The calculation process of the real-time position of the above camera is described as follows:

[0113] When the user has completed the selection of the characteristic position, the position of 9 points is selected (this solution is not limited to 9 points, it can be any point); and the matrix compensation coefficient of the LED display screen corresponding to the characteristic point is obtained through algorithm or manual adjustment;

[0114] After the camera position is moved, the compensation coefficient corresponding to the coordinate position is determined and sent to the LED display screen according to the different coordinate positions of the camera;

[0115] When the camera is at the coordinate position (100,100), after the coordinate (100,100) is given to the sending device, the sending device will send the compensation coefficient of the entire LED display corresponding to (100,100) to the LED display in real time;

[0116] When the camera moves to (2000,100), the coordinates of (2000,100) are given to the sending device, and the sending device sends the compensation coefficient of the entire LED display corresponding to (2000,100) to the LED display in real time.

[0117] Specifically, when shooting people and scenes indoors and using the display screen as the display background, a lot of post-production costs can be reduced. The camera is used to shoot and collect the display image on the display screen. Since the camera shoots at different angles and directions, color cast problems will occur. The commonly used display screen is an LED display screen. The smallest display unit of the LED display screen is an LED light point. Each single LED light point contains three independent light-emitting units, namely RGB. These light-emitting units have a certain arrangement relationship in space, including: vertical one-word arrangement, horizontal one-word arrangement, triangular arrangement, etc. For example, when the light-emitting units are arranged as RGB from left to right horizontally, when observed from the left side of the LED display screen, the light intensity relationship of the display screen is R>G>B, introducing a phenomenon of red bias; when observed from the right side, the light intensity seen is: R<G<B, then a blue bias phenomenon is seen. The specific compensation coefficient needs to be determined according to the color cast situation of the image observed during specific shooting. That is, when the image acquisition device acquires images of the display screen at different positions, the target images acquired at different positions will have color cast problems. The compensation coefficient for the target image is determined based on the working position information of the image acquisition device at different positions. The compensation coefficient refers to the multiple of the color difference of the target image to be processed. The compensation coefficient is a non-zero positive number. If the compensation coefficient is greater than 1, it means that the target image to be processed is backlit and too dark, and the brightness needs to be increased; if the compensation coefficient is less than 1, it means that the target image to be processed is frontlit and too bright and overexposed, and the brightness needs to be reduced.

[0118] The second determination module 83 determines the compensation weight of the target image in different shooting directions based on the position information of the target image on the display screen, and determines the target compensation coefficient based on the compensation weight and the first compensation coefficient set. The compensation weight is determined by the distance between the calibration point position of the image acquisition device and the target image. The distance is inversely proportional to the compensation weight, that is, the greater the distance, the smaller the weight. For example, there are 4 points. The closer the pixel is to the calibration point corresponding to the shooting direction, the greater the compensation weight corresponding to that calibration point.

[0119] Optionally, obtain the position information of multiple calibration points in the image acquisition device, and determine the size information of the display screen. Among them, the calibration point positions corresponding to the multiple calibration point position information are in one-to-one correspondence with different shooting directions. The size information is determined based on the number of pixel points in the display screen, and other measurement methods can also be used; determine the target viewing angle corresponding to the position information of multiple calibration points. The target viewing angle is the angle formed by the shooting direction of the image acquisition device and the normal line of the plane where the display screen is located when the image acquisition device is in different working positions; according to the difference information between the coordinate information and the size information corresponding to the position information of the target image on the display screen; determine the compensation weight according to the difference information and the target viewing angle.

[0120] Specifically, when the image acquisition device acquires the picture displayed on the display screen, the image acquisition device can acquire pictures at different angles in different working positions. By matching the calibration points with the corresponding shooting directions one by one, the corresponding different shooting directions can be acquired by acquiring multiple calibration points. At the same time, when the acquisition devices corresponding to the multiple calibration points are shooting at different working positions, the angle between the image acquisition device and the normal of the display screen is recorded as the target angle, wherein the target angle is determined based on the angle between the position information of the multiple calibration points of the image acquisition device and the direction vector of the normal of the display screen.

[0121] While obtaining the calibration point position information and the target viewing angle corresponding to the calibration point position information, determine the size information of the display screen. The size of a common LED display screen is affected by many factors, including area, display ratio, model, module size, cabinet size, etc. There are various models. The size of the cabinet can be determined according to the actual situation. The size of the display screen is determined based on the cabinet size. In the specific application process, it is not limited to this implementation method.

[0122] In order to save computing resources, the plurality of calibration points corresponding to the plurality of calibration point position information at least include calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other. Figure 4 The straight line formed by (128, 0) and (128, 256) is perpendicular to the straight line formed by (0, 128) and (256, 128). There are 4 compensation coefficients for the above 4 coordinate positions. The number of the above multiple marking points can be an even number or an odd number, which can be flexibly set according to the actual situation.

[0123] After determining the above display screen size information, the difference information between the coordinate information corresponding to the position information of the target image in the display screen and the display screen size information is also required; and the compensation weight is determined based on the difference information and the target viewing angle. In other words, when determining the above compensation weight, it can be determined based on the above difference information. However, it should be noted that it is not limited to this implementation method in the specific application process. For example, the above compensation weight can be pre-set, for example, the mapping relationship between the mark point and the weight is maintained in the database, and the corresponding compensation weight is determined based on the mapping relationship.

[0124] When determining the compensation weight based on the difference information and the target viewing angle, it can be implemented as follows, but is not limited to this: performing a difference operation on the coordinate information corresponding to the size information and the position information of the target image in the display screen, and determining the compensation weight based on the operation result and the target viewing angle.

[0125] The compensation module 84 is used to compensate the target image according to the target compensation coefficient.

[0126] Specifically, the target compensation coefficient is obtained according to the position information of the current target image, the size of the box and the first compensation coefficient set, and the target compensation coefficient is a compensation coefficient that matches the target image and is determined by the position information of the current target image.

[0127] In order to ensure the compensation effect of the image, before compensating the target image according to the target compensation coefficient, the target image can also be processed as follows: an original grayscale image of the target image is obtained, and the original grayscale image is converted to obtain a target grayscale image, wherein the target grayscale image is exponentially related to the original grayscale image.

[0128] Specifically, the sensitivity of the human eye to external light sources is not linearly related to the input light intensity, but exponentially related. Under low illumination, the human eye can more easily distinguish changes in brightness. As the illumination increases, it is difficult for the human eye to distinguish changes in brightness. However, the camera's sensitivity is linearly related to the input light intensity. Gamma conversion is used to enhance the first image data to ensure that the accuracy of the first image data is improved. According to an embodiment of the present application, an image processing system for implementing the above image processing method is also provided, as shown in 1(a), the system includes at least: an image source device 11, a sending device 12, and a receiving device 13, wherein the image source device 11 is used to provide an original image and send the original image to the sending device;

[0129] In the process of the image processing method, the received original image is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, the brightness can be changed by dragging the brightness slider to ensure that the image display effect of the target image received by the receiving device is better. Similarly, we can continue to locally modify the original image using similar adjustment methods as mentioned above until the desired visual effect is achieved.

[0130] The sending device 12 is used to receive the original image and process the original image to obtain the target image;

[0131] In the process of the image processing method, the received original image is processed. In order to obtain a higher quality target image, the parameters can be adjusted manually. For example, when the brightness of the received original image is too dark or too bright, the brightness can be changed by dragging the brightness slider to ensure that the image received by the receiving device has a better display effect. We can use the same adjustment method to make local modifications to the original image until the desired visual effect is achieved.

[0132] The receiving device 13 is used to execute the image processing method provided in the embodiment of the present application. For example, Figure 2 The image processing method shown;

[0133] In the process of executing the image processing method, since it is also necessary to determine the position information of the image acquisition device and the camera shooting direction information (such as the lens direction) and convert the position information into coordinate information, the image processing system, as shown in Figure 1 (b), also needs to include: an information generator 14, which is used to obtain the coordinate information corresponding to the working position information of the image acquisition device 15 when performing image acquisition at different working positions, and obtain the shooting direction of the image acquisition device. The information generator 14 can be located in the sending device (such as a sending card), or it can be a device relatively independent of the camera information of the sending device.

[0134] In addition, in order to complete the data distribution, as shown in Figure 1(c), an information converter 16 may also be included. The information converter may be located in a sending device (such as a sending card) or may be a device relatively independent of the camera information of the sending device. The information converter may include the following processing units: an acquisition unit: acquiring the existing position information of one or more cameras from the outside; a camera information conversion compensation coefficient: this module converts the camera's position information into compensation data information of the LED display screen; a compensation coefficient sending module: sending the compensation coefficient to a receiving card device, which is a control unit of the LED control system that drives the LED light board.

[0135] Example 3

[0136] According to an embodiment of the present application, a non-volatile storage medium is also provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the image processing method in Example 1.

[0137] Optionally, the following steps are executed when the program is running: obtaining position information of a target image in a display screen when the target image is displayed on the display screen; determining working position information of an image acquisition device when the image acquisition device acquires images at different working positions, and obtaining a compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire an image corresponding to a display screen of the display screen; determining compensation weights in different shooting directions based on the position information of the target image in the display screen, and determining a target compensation coefficient based on the compensation weight and the first compensation coefficient set; and compensating the target image based on the target compensation coefficient.

[0138] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0139] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0144] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: Acquiring position information of the target image on the display screen when the target image is displayed on the display screen; Determine working position information of an image acquisition device when performing image acquisition at at least one position, and obtain a compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set, wherein the image acquisition device is at least used to acquire a display screen of the display screen; Determining a compensation weight of the target image under different shooting directions according to position information of the target image in the display screen, and determining a target compensation coefficient based on the compensation weight and the first compensation coefficient set; Compensating the target image according to the target compensation coefficient; Wherein, determining the compensation weight of the target image under different shooting directions according to the position information of the target image in the display screen includes: determining the difference information between the coordinate information corresponding to the position information of the target image in the display screen and the size information of the display screen; determining the compensation weight according to the difference information and the target viewing angle corresponding to the position information of multiple calibration points of the image acquisition device, wherein the target viewing angle is the angle formed by the shooting direction of the image acquisition device and the normal of the plane where the display screen is located when the image acquisition device is in different working positions.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the plurality of calibration point position information and determine the size information, wherein the calibration point positions corresponding to the plurality of calibration point position information correspond one-to-one to the different shooting directions; The target viewing angle is determined.

3. The method according to claim 1, characterized in that Determining the compensation weight according to the difference information and the target viewing angle corresponding to the position information of the plurality of calibration points of the image acquisition device includes: A difference operation is performed on the size information and the coordinate information, and the compensation weight is determined according to the operation result and the target viewing angle.

4. The method according to claim 1, characterized in that: The first compensation coefficient set is a coefficient matrix, wherein each element in the coefficient matrix is ​​a compensation coefficient corresponding to each calibration point position information in the plurality of calibration point position information; determining the target compensation coefficient based on the compensation weight and the first compensation coefficient set includes: A second coefficient set is determined according to all coefficients in the coefficient matrix and the compensation weight; and a target compensation coefficient is determined based on an average value of all coefficients in the second coefficient set.

5. According to the method of claim 1, the multiple calibration points corresponding to the multiple calibration point position information at least include calibration points distributed on two straight lines, and the two straight lines are perpendicular to each other.

6. The method according to claim 1, characterized in that Obtaining the compensation coefficient corresponding to the working position information includes: receiving a compensation coefficient corresponding to the working position information from an external device; or, A compensation coefficient corresponding to the working position information is determined from a set of compensation coefficients pre-stored in a receiving device.

7. The method according to any one of claims 1 to 6, characterized in that: It is characterized in that Determining working position information of an image acquisition device when performing image acquisition at at least one position includes: Acquire a sample image acquired at a sampling point in a set of candidate sampling points when the image acquisition device moves within an effective moving space; Obtaining a quality assessment index of the sample image; A target sampling point is selected from the candidate sampling point set according to the quality evaluation index, and the position of the target sampling point is used as the working position information.

8. An image processing device, characterized in that: include: An acquisition module, used for acquiring position information of a target image on the display screen when the target image is displayed on the display screen; A first determination module is used to determine the working position information of the image acquisition device when performing image acquisition at different working positions, and obtain the compensation coefficient corresponding to the working position information to obtain a first compensation coefficient set; a second determination module, configured to determine compensation weights under different shooting directions according to position information of the target image in the display screen, and determine a target compensation coefficient based on the compensation weights and the first compensation coefficient set; A compensation module, used for compensating the target image according to the target compensation coefficient; Wherein, the second determination module is used to determine the compensation weight of the target image under different shooting directions according to the position information of the target image in the display screen by executing the following steps: determining the difference information between the coordinate information corresponding to the position information of the target image in the display screen and the size information of the display screen; determining the compensation weight according to the difference information and the target viewing angle corresponding to the position information of multiple calibration points of the image acquisition device, wherein the target viewing angle is the angle formed by the shooting direction of the image acquisition device and the normal of the plane where the display screen is located when the image acquisition device is in different working positions.

9. An image processing system, characterized in that: include: Image source device, sending device, receiving device, including: The image source device is used to provide an original image and send the original image to a sending device; The sending device is used to receive the original image and process the original image to obtain a target image; The receiving device is used to execute the image processing method described in any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 7.

Images