An image overlay method, device, medium and product

Abstract

The present invention discloses an image superposition method, device, medium and product, relating to the field of image processing, including: an image processor receives a first current frame image and a second current frame image respectively transmitted by two data sources, and when receiving the upper half image of any current frame image, predicts the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when receiving the upper half image of any current frame image, thereby obtaining a complete current image to be superposed; then superposes the current images to be superposed respectively corresponding to the two data sources to obtain a current superposed image. The present invention is based on dual-channel image sampling and image prediction, improves the acquisition speed of the image to be superposed, superposes the image to be superposed within a short time delay, thereby improving the speed of image superposition and the real-time performance of image superposition display.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular, to an image overlay method, device, medium, and product. Background Art

[0002] The VGA (Video Graphic Array) display controller integrated inside the BMC (Baseboard Management Controller) can process the local display screen of the host and output it to the display for display. At this time, the BMC is used as the graphics card of the host, and this graphics card is only controlled by the host. The SoC (System on Chip) display controller integrated inside the BMC can process some custom screens and output them to the display for display. For example, the custom screen can be a screen generated based on the real-time status monitoring information of the host, or a screen generated based on the real-time status monitoring information of the BMC, etc. At this time, the BMC is no longer used as the graphics card of the host.

[0003] Since two display controllers are integrated inside the BMC, there are three ways to display the screen of the BMC. The first is to output Screen 1 to the display through the VGA display controller for display. The second is to output Screen 2 to the display through the SoC display controller for display. The third is to overlay Screen 1 and Screen 2 and output the overlaid screen to the display for display. For the third method, currently, generally, two DDR (Double DataRate) chips are externally connected to the two display controllers to write the two images before overlaying into the DDR chips, and then the two images in the DDR chips are read again through the internal control logic for overlay processing, and then the overlaid image is output to the display for display. However, this way of writing - then reading - then overlaying - then displaying is very likely to cause the problem of slow image overlay speed, thus affecting the real-time performance of image overlay display. It can be seen that how to improve the speed of image overlay is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide an image overlay method, device, medium, and product, which can improve the acquisition speed of the images to be overlaid based on dual-channel image sampling and image prediction, so as to overlay the images to be overlaid within a short time delay, and further improve the speed of image overlay and the real-time performance of image overlay display. The specific solutions are as follows:

[0005] In a first aspect, the present invention provides an image overlay method, which is applied to an image processor and includes:

[0006] Receive the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively. When receiving the upper half image of any current frame image, predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when receiving the upper half image of any current frame image;

[0007] Based on the received upper half image of any current frame image and the predicted lower half image of any current frame image, determine the complete current image to be superimposed;

[0008] Superimpose the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain the current superimposed image;

[0009] Wherein, the previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image bisect any current frame image.

[0010] Optionally, predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image includes:

[0011] Predict the lower half image of any current frame image by using the target association relationship and based on the upper half image of any current frame image and the previous frame image;

[0012] Wherein, the target association relationship includes the first association relationship between any current frame image and the previous frame image and the second association relationship between each pixel point in the same frame image.

[0013] Optionally, predicting the lower half image of any current frame image by using the target association relationship and based on the upper half image of any current frame image and the previous frame image includes:

[0014] Use the second association relationship and based on any pixel point in the upper half image of any current frame image to determine the pixel point to be predicted in the lower half image of any current frame image;

[0015] Use the first association relationship and based on the pixel point to be predicted to determine the corresponding first target pixel point from the previous frame image;

[0016] Predict the pixel value of the pixel point to be predicted based on the pixel value of any pixel point and the pixel value of the first target pixel point.

[0017] Optionally, any pixel point and the pixel point to be predicted are symmetric about the image center point of any current frame image.

[0018] Optionally, using the second association relationship and based on any pixel point in the upper half of any current frame image, determining the to-be-predicted pixel point in the lower half of any current frame image includes:

[0019] Determining the first row number of the pixel row where any pixel point is located and the first column number of the pixel column where it is located in any current frame image;

[0020] Based on the total number of pixel rows of any current frame image and the first row number, determining the second row number of the pixel row where the to-be-predicted pixel point is located in any current frame image; the sum of the first row number and the second row number is equal to the total number of pixel rows plus one;

[0021] Based on the total number of pixel columns of any current frame image and the first column number, determining the second column number of the pixel column where the to-be-predicted pixel point is located in any current frame image; the sum of the first column number and the second column number is equal to the total number of pixel columns plus one;

[0022] Determining the to-be-predicted pixel point in the lower half of any current frame image according to the second row number and the second column number.

[0023] Optionally, using the first association relationship and based on the to-be-predicted pixel point, determining the corresponding first target pixel point from the previous frame image includes:

[0024] Determining the first pixel position of the to-be-predicted pixel point in any current frame image;

[0025] Based on the first pixel position, determining the pixel point at the corresponding position from the previous frame image to obtain the first target pixel point.

[0026] Optionally, predicting the pixel value of the to-be-predicted pixel point based on the pixel value of any pixel point and the pixel value of the first target pixel point includes:

[0027] Determining the feature value of any pixel point based on the pixel value of any pixel point; wherein, the feature value of any pixel point includes a horizontal feature value and a vertical feature value;

[0028] Predicting the pixel value of the to-be-predicted pixel point according to the pre-calculated image similarity, the feature value of any pixel point, and the pixel value of the first target pixel point; wherein, the image similarity is the similarity between the upper half and the lower half of the previous frame image.

[0029] Optionally, determining the feature value of any pixel point based on the pixel value of any pixel point includes:

[0030] Using the first association relationship and based on any pixel point, determining the corresponding second target pixel point from the previous frame image;

[0031] Determine the horizontal feature value of any pixel based on the pixel value of any pixel and the pixel value of the second target pixel;

[0032] Determine the vertical feature value of any pixel based on the horizontal feature value of any pixel and the vertical feature value of the third target pixel;

[0033] Wherein, the third target pixel is the pixel in the previous pixel row of the pixel row where any pixel is located in any current frame image, and the third target pixel and any pixel are in the same pixel column in any current frame image.

[0034] Optionally, determining the horizontal feature value of any pixel based on the pixel value of any pixel and the pixel value of the second target pixel includes:

[0035] Determine a first value based on the pixel value of any pixel and the pixel value of the second target pixel;

[0036] If the first value is less than the preset threshold, set zero as the horizontal feature value of any pixel;

[0037] If the first value is not less than the preset threshold, set the first value as the horizontal feature value of any pixel.

[0038] Optionally, predicting the pixel value of the pixel to be predicted according to the pre-calculated image similarity, the feature value of any pixel, and the pixel value of the first target pixel includes:

[0039] Determine a second value based on half of the total number of pixel rows of any current frame image and the horizontal feature value of any pixel;

[0040] Determine a third value based on half of the total number of pixel columns of any current frame image and the vertical feature value of any pixel;

[0041] Predict the pixel value of the pixel to be predicted according to the image similarity, the second value, the third value, and the pixel value of the first target pixel.

[0042] Optionally, superimposing the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain the current superimposed image, including:

[0043] Perform bilinear interpolation on the current images to be superimposed corresponding to the first data source and the second data source respectively using the resolution of the display to scale the current images to be superimposed, and obtain the current scaled images corresponding to the first data source and the second data source respectively;

[0044] Superimpose the current scaled images corresponding to the first data source and the second data source respectively to obtain the current superimposed image, and output the current superimposed image to the display for display.

[0045] Optionally, output the current superimposed image to a display for display, including:

[0046] Determine the display timing parameters of the display based on the resolution of the display;

[0047] Output the current superimposed image to the display for display according to the display timing parameters of the display.

[0048] In a second aspect, the present invention provides an electronic device, including:

[0049] A memory for storing a computer program;

[0050] A processor for executing the computer program to implement the steps of the foregoing image superimposing method.

[0051] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the foregoing image superimposing method are implemented.

[0052] In a fourth aspect, the present invention provides a computer program product, including computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of the foregoing image superimposing method are implemented.

[0053] In the present invention, an image processor receives a first current frame image and a second current frame image respectively transmitted by a first data source and a second data source, and when receiving the upper half image of any current frame image, predicts the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when receiving the upper half image of any current frame image; determine a complete current image to be superimposed based on the received upper half image of any current frame image and the predicted lower half image of any current frame image; superimpose the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image; wherein, the previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image bisect any current frame image.

[0054] Beneficial effects: Through dual-channel image sampling, the present invention receives the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively at the same time, thereby improving the speed of image reception. Moreover, when receiving the upper half of any current frame image, the present invention can predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image from the same data source. Thus, when the reception of the upper half of any current frame image is completed, the prediction of the lower half of any current frame image is completed. At this time, a complete current image to be superimposed can be pieced together. Compared with the time spent receiving the entire image, the present invention only needs to spend half of the time to obtain the complete image, improving the speed of image acquisition. After that, the present invention superimposes the current images to be superimposed corresponding to the two data sources respectively to obtain the current superimposed image. It can be found that the present invention improves the speed of image acquisition to superimpose the images to be superimposed within a shorter time delay, thereby improving the speed of image superimposition and the real-time performance of image superimposed display. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] To more clearly illustrate the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of an image superimposing method provided by an embodiment of the present invention;

[0057] Figure 2 It is a flowchart of an image prediction provided by an embodiment of the present invention;

[0058] Figure 3 It is a flowchart of an image superimposing provided by an embodiment of the present invention;

[0059] Figure 4 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0061] As used in the description of the present invention and the above-mentioned drawings, the terms "comprising" and "having", as well as any variations related to "comprising" and "having", are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units not listed.

[0062] To enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0063] For image superposition, currently, it is generally achieved by externally connecting a DDR chip to two display controllers to write the two images before superposition into the DDR chip, then rereading the two images in the DDR chip through internal control logic for superposition processing, and finally outputting the superposed image to the display for display. However, this way of writing - rereading - superposing - displaying is extremely likely to cause the problem of slow image superposition speed, thus affecting the real-time performance of image superposition display. Therefore, the present invention provides an image superposition method that can improve the acquisition speed of the images to be superposed based on dual-channel image sampling and image prediction, so as to superpose the images to be superposed within a short time delay, thereby improving the speed of image superposition and the real-time performance of image superposition display.

[0064] See Figure 1 As shown, an embodiment of the present invention discloses an image superposition method applied to an image processor, including:

[0065] Step S11: Receive the first current frame image and the second current frame image respectively transmitted by the first data source and the second data source, and when receiving the upper half image of any current frame image, predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when receiving the upper half image of any current frame image; wherein, the previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image bisect any current frame image.

[0066] The image processor proposed in the embodiments of the present invention can be either an independently operating processor or integrated into other chips, thereby providing image processing capabilities for other chips. Taking the integration of the image processor into the BMC (Baseboard Management Controller) and the first data source being the VGA (Video Graphics Array) display controller in the baseboard management controller and the second data source being the SOC (System on Chip) display controller in the baseboard management controller as an example, an external image processor is connected to the VGA display controller and the SOC display controller in the baseboard management controller to receive the images transmitted by the VGA display controller and the SOC display controller through the image processor, and perform image prediction and image overlay, so as to achieve efficient and fast image overlay without additionally increasing the DDR (Double Data Rate) chip.

[0067] Specifically, the image processor simultaneously receives the first current frame image and the second current frame image respectively transmitted by the first data source and the second data source through dual-channel image sampling. When receiving the upper half of any current frame image, based on the upper half of any current frame image and the previous frame image from the same data source as any current frame image, the lower half of any current frame image is predicted. Further, in the embodiments of the present invention, when receiving the upper half of any current frame image, the prediction of the lower half of any current frame image can be completed, that is, compared with the time spent receiving the entire image, the present invention only needs to spend half of the time to obtain a complete image, improving the image acquisition speed.

[0068] It should be noted that any current frame image is any one of the first current frame image and the second current frame image, and the upper half and the lower half of any current frame image divide any current frame image equally. It should also be noted that the previous frame image is from the same data source as any current frame image and is the previous frame of any current frame image.

[0069] For any current frame image among the first current frame image and the second current frame image, it is necessary to use the target association relationship and based on the upper half of any current frame image and the previous frame image to predict the lower half of any current frame image. That is, in the embodiments of the present invention, it is necessary to use the target association relationship and based on the upper half of any current frame image and the previous frame image from the same data source as any current frame image to predict the lower half of any current frame image.

[0070] Among them, the target association relationship includes a first association relationship between any current frame image and the previous frame image, and a second association relationship between pixel points in the same frame image. In this way, based on the association relationship between adjacent two frame images and the association relationship between pixel points in the same frame image in the embodiments of the present invention, the prediction of the lower half image of any current frame image can be realized, which can not only ensure the accuracy of image prediction, but also improve the image acquisition speed, and then realize the superposition of images with a short time delay.

[0071] Step S12: Determine a complete current image to be superimposed based on the upper half image of any received current frame image and the predicted lower half image of any current frame image.

[0072] In the embodiments of the present invention, after predicting the lower half image of any current frame image, the upper half image of any received current frame image and the predicted lower half image of any current frame image are spliced to obtain a complete current image to be superimposed.

[0073] It should be noted that in the embodiments of the present invention, by adopting dual-channel image sampling, the prediction of the lower half image of the first current frame image and the prediction of the lower half image of the second current frame image are also dual-channel parallel, so as to improve the image prediction speed to a certain extent.

[0074] Step S13: Superimpose the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image.

[0075] In the embodiments of the present invention, after obtaining the complete current images to be superimposed corresponding to the first data source and the second data source respectively, the current images to be superimposed corresponding to the first data source and the second data source are superimposed respectively to obtain a current superimposed image.

[0076] Furthermore, considering that the resolutions of the first current frame image, the second current frame image, and the display may be different from each other, that is, the resolutions of the current images to be superimposed corresponding to the first data source, the current images to be superimposed corresponding to the second data source, and the display may be different from each other. Therefore, in order to enable the superimposed image to be displayed on the display, it is necessary to first scale the current image to be superimposed to the resolution size of the display, and then perform image superposition on the scaled image.

[0077] Specifically, bilinear interpolation is used for the current images to be superimposed corresponding to the first data source and the second data source respectively according to the resolution of the display, so as to scale the current images to be superimposed, thereby obtaining the current scaled images corresponding to the first data source and the second data source respectively. At this time, the resolutions of the current scaled images corresponding to the first data source and the second data source respectively are equal to the resolution of the display. Then, the current scaled images corresponding to the first data source and the second data source respectively are superimposed to obtain the current superimposed image, and the current superimposed image is output to the display for display. In this way, in the embodiments of the present invention, by scaling the current images to be superimposed and then superimposing them, the superimposed image can be adapted to the display, avoiding the problem of mismatch between the superimposed image and the display.

[0078] It should be noted that for the scaling of the current images to be superimposed, in addition to bilinear interpolation, nearest neighbor interpolation, bicubic interpolation, etc. can also be used. Specific limitations are not made here, and users can choose according to the processor performance and their own needs.

[0079] Further, for outputting the current superimposed image to the display for display, it may specifically include: determining the display timing parameters of the display based on the resolution of the display, and then outputting the current superimposed image to the display for display according to the display timing parameters of the display. Among them, the display timing parameters of the display can also be understood as the line-field timing parameters of the display, mainly including two parts: line timing parameters and field timing parameters; and, the line timing parameters mainly include line synchronization, line blanking, line video valid, line front porch, etc., and the field timing parameters mainly include field synchronization, field blanking, field video valid, field front porch, etc.

[0080] It should be noted that during the process of superimposing the current scaled images corresponding to the first data source and the second data source respectively, considering that the image brightness, image contrast, etc. of the current scaled images corresponding to the first data source and the second data source may be different. Therefore, in order to make the superimposed image look more natural, the present invention can first analyze the brightness and contrast of the current scaled images corresponding to the first data source and the second data source respectively to calculate the mean and standard deviation between the two scaled images, and then based on the mean and standard deviation between the two scaled images, adjust the brightness and contrast of the two scaled images to a similar range. After that, the two adjusted scaled images are superimposed to obtain the initially superimposed image, and finally, the brightness and contrast of the initially superimposed image are finely adjusted to obtain the final current superimposed image. In this way, through the above method, the superimposed image can have a more natural transition and be more visually coordinated, thereby further improving the user experience.

[0081] Beneficial effects: Through dual-channel image sampling, the present invention receives the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively at the same time, thereby improving the speed of image reception. Moreover, when receiving the upper half image of any current frame image, the present invention can predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image from the same data source. Thus, when the reception of the upper half image of any current frame image is completed, the prediction of the lower half image of any current frame image is completed. At this time, a complete current image to be superimposed can be pieced together. Compared with the time spent on receiving the whole image, the present invention only needs to spend half of the time to obtain the complete image, which improves the speed of image acquisition. After that, the present invention superimposes the current images to be superimposed corresponding to the two data sources respectively to obtain the current superimposed image. It can be found that the present invention improves the speed of image acquisition to superimpose the images to be superimposed within a shorter time delay, thereby improving the speed of image superimposition and the real-time performance of image superimposed display.

[0082] Based on the previous embodiment, the previous embodiment describes the complete process from image reception and image prediction to image superimposition. Next, this embodiment will elaborate in detail on how to predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image. Refer to Figure 2 As shown, the embodiment of the present invention discloses a process of image prediction, including:

[0083] Step S21: Using the second correlation relationship and based on any pixel point in the upper half image of any current frame image, determine the pixel point to be predicted in the lower half image of any current frame image.

[0084] In the embodiment of the present invention, the reception of the upper half image of any current frame image is generally carried out by receiving each pixel row by row for the upper half image of any current frame image. Therefore, for the prediction of the lower half image of any current frame image, the prediction of the lower half image of any current frame image is also carried out by predicting each pixel one by one.

[0085] Based on this, when receiving the upper half image of any current frame image, for any pixel point in the upper half image of any current frame image, it is necessary to first determine the pixel point to be predicted in the lower half image of any current frame image based on any pixel point.

[0086] Specifically, using the second correlation relationship between pixel points in the same frame image and based on any pixel point in the upper half image of any current frame image, a to-be-predicted pixel point located in the lower half image of any current frame image is determined. Moreover, any pixel point and the to-be-predicted pixel point are symmetric about the image center point of any current frame image.

[0087] According to one embodiment, for any pixel point in the upper half image of any current frame image, first, determine the first row number of the pixel row where any pixel point is located and the first column number of the pixel column where it is located in any current frame image; then, based on the total number of pixel rows and the first row number of any current frame image, determine the second row number of the pixel row where the to-be-predicted pixel point is located in any current frame image, where the sum of the first row number and the second row number is equal to the total number of pixel rows plus one; based on the total number of pixel columns and the first column number of any current frame image, determine the second column number of the pixel column where the to-be-predicted pixel point is located in any current frame image, where the sum of the first column number and the second column number is equal to the total number of pixel columns plus one; finally, according to the second row number and the second column number, determine the to-be-predicted pixel point located in the lower half image of any current frame image. In this way, the determined to-be-predicted pixel point and any pixel point can be made symmetric about the image center point of any current frame image through the above method.

[0088] Exemplarily, if any current frame image is an image with a resolution of 640*480, then the total number of pixel rows of any current frame image is 480 rows, and the total number of pixel columns is 640 columns. Taking any pixel point as the pixel point in the 2nd row and 1st column of any current frame image as an example, determine the first row number of the pixel row where any pixel point is located and the first column number of the pixel column where it is located in any current frame image. At this time, the first row number is 2, and the first column number is 1; then, based on the total number of pixel rows and the first row number of any current frame image, determine the second row number of the pixel row where the to-be-predicted pixel point is located in any current frame image. At this time, the second row number = total number of pixel rows + 1 - first row number = 480 + 1 - 2 = 479; then, based on the total number of pixel columns and the first column number of any current frame image, determine the second column number of the pixel column where the to-be-predicted pixel point is located in any current frame image. At this time, the second column number = total number of pixel columns + 1 - first column number = 640 + 1 - 1 = 640; finally, according to the second row number and the second column number, determine the to-be-predicted pixel point located in the lower half image of any current frame image, that is, the to-be-predicted pixel point is the pixel point in the 479th row and 640th column of any current frame image.

[0089] Step S22: Using the first correlation relationship and based on the to-be-predicted pixel point, determine the corresponding first target pixel point from the previous frame image.

[0090] In an embodiment of the present invention, after determining a to-be-predicted pixel point in the lower half of an image of any current frame, a corresponding first target pixel point is determined from the previous frame by using a first correlation relationship between any current frame and the previous frame and based on the to-be-predicted pixel point.

[0091] Specifically, a first pixel position of the to-be-predicted pixel point in any current frame is determined, and a pixel point at a corresponding position is determined from the previous frame based on the first pixel position to obtain a first target pixel point. That is, the pixel position of the to-be-predicted pixel point in any current frame is the same as the pixel position of the first target pixel point in the previous frame.

[0092] Exemplarily, if the to-be-predicted pixel point is a pixel point at the 479th row and the 640th column in any current frame, the first pixel position of the to-be-predicted pixel point in any current frame is (479th row, 640th column), and then the pixel point at the 479th row and the 640th column is determined from the previous frame to obtain a first target pixel point.

[0093] Step S23: Predict the pixel value of the to-be-predicted pixel point based on the pixel value of any pixel point and the pixel value of the first target pixel point.

[0094] In an embodiment of the present invention, after determining the first target pixel point from the previous frame, the pixel value of the to-be-predicted pixel point is predicted based on the pixel value of any pixel point and the pixel value of the first target pixel point.

[0095] Specifically, a feature value of any pixel point is determined based on the pixel value of any pixel point; wherein, the feature value of any pixel point includes a horizontal feature value and a vertical feature value; then, according to a pre-calculated image similarity, the feature value of any pixel point, and the pixel value of the first target pixel point, the pixel value of the to-be-predicted pixel point is predicted; wherein, the image similarity is the similarity between the upper half and the lower half of the previous frame.

[0096] It should be noted that the image similarity between the upper half and the lower half of the previous frame can be calculated using SSIM (Structural Similarity), and the specific calculation method is as follows:

[0097] ;

[0098] wherein, represents the upper half of the previous frame; represents the lower half of the previous frame; represents the image similarity between the upper half and the lower half of the previous frame; represents the mean of the upper half of the previous frame image; represents the mean of the lower half of the previous frame image; represents the variance of the upper half of the previous frame image; represents the variance of the lower half of the previous frame image; represents the covariance between the upper half and the lower half of the previous frame image; and represents a constant used to avoid the denominator being zero.

[0099] For the determination of the horizontal eigenvalue and the vertical eigenvalue of any pixel point, it specifically may include: First, use the first correlation relationship between any current frame image and the previous frame image and based on any pixel point, determine the corresponding second target pixel point from the previous frame image; then, based on the pixel value of any pixel point and the pixel value of the second target pixel point, determine the horizontal eigenvalue of any pixel point; and based on the horizontal eigenvalue of any pixel point and the vertical eigenvalue of the third target pixel point, determine the vertical eigenvalue of any pixel point; where the third target pixel point is the pixel point in the previous pixel row of the pixel row where any pixel point is located in any current frame image, and the third target pixel point and any pixel point are in the same pixel column in any current frame image.

[0100] Among them, for the second target pixel point, specifically, it is necessary to first determine the second pixel position of any pixel point in any current frame image, and based on the second pixel position, determine the pixel point at the corresponding position from the previous frame image to obtain the second target pixel point. That is, the pixel position of any pixel point in any current frame image is the same as the pixel position of the second target pixel point in the previous frame image.

[0101] And, the position of the third target pixel point in any current frame image is in the same column as the previous row of any pixel point. Taking any pixel point as the pixel point at the 2nd row and 1st column in any current frame image as an example, the third target pixel point is the pixel point at the 1st row and 1st column in any current frame image.

[0102] For the determination of the horizontal eigenvalue of any pixel point, it specifically may include: Based on the pixel value of any pixel point and the pixel value of the second target pixel point, determine the first value; determine whether the first value is less than the preset threshold. If the first value is less than the preset threshold, then set zero as the horizontal eigenvalue of any pixel point; if the first value is not less than the preset threshold, then set the first value as the horizontal eigenvalue of any pixel point. It should be noted that the preset threshold is a configurable value.

[0103] Among them, for the determination of the first value, taking the pixel value including RGB (Red, Green, Blue, the three primary colors of red, green, and blue) values as an example, the difference between the pixel value of any pixel point on any channel and the pixel value of the second target pixel point on any channel is determined. Here, any channel is any one of the three channels R, G, and B. Then, the first value is determined based on the differences corresponding to the three channels R, G, and B respectively.

[0104] In the first specific implementation manner, the first value is equal to the sum of the differences corresponding to the three channels R, G, and B respectively, that is, the first value = (Rc - Rp) + (Gc - Gp) + (Bc - Bp). Here, Rc represents the pixel value of any pixel point on the R channel, Rp represents the pixel value of the second target pixel point on the R channel, Gc represents the pixel value of any pixel point on the G channel, Gp represents the pixel value of the second target pixel point on the G channel, Bc represents the pixel value of any pixel point on the B channel, and Bp represents the pixel value of the second target pixel point on the B channel.

[0105] In the second specific implementation manner, by determining the sum of the differences corresponding to the three channels R, G, and B respectively, and then taking the average of the sum to obtain the first value, that is, the first value = [(Rc - Rp) + (Gc - Gp) + (Bc - Bp)] / 3. It should be noted that the preset threshold corresponding to the first specific implementation manner = three times the preset threshold corresponding to the second specific implementation manner.

[0106] For the determination of the vertical feature value of any pixel point, it may specifically include: based on the horizontal feature value of any pixel point, the vertical feature value of the third target pixel point, and a configurable parameter, determining the vertical feature value of any pixel point; where the configurable parameter is a value not less than 0 and not greater than 1. That is, the vertical feature value of any pixel point = a × the horizontal feature value of any pixel point + (1 - a) × the vertical feature value of the third target pixel point; where a represents the configurable parameter, and the configurable parameter is used to characterize the correlation characteristics between the pixels in the upper and lower rows of the same frame of image.

[0107] It should be noted that since the position of the third target pixel point in any current frame of image is in the same column of the previous row of any pixel point, therefore, if any pixel point is in the first row of any current frame of image, there is no corresponding third target pixel point for any pixel point, and at this time, the vertical feature value of any pixel point = a × the horizontal feature value of any pixel point.

[0108] Further, for the prediction of the pixel value of the pixel point to be predicted, it may specifically include: determining a second value based on half of the total number of pixel rows of any current frame image and the horizontal feature value of any pixel point; determining a third value based on half of the total number of pixel columns of any current frame image and the vertical feature value of any pixel point; and then predicting the pixel value of the pixel point to be predicted according to the image similarity between the upper half image and the lower half image of the previous frame image calculated in advance, the second value, the third value, and the pixel value of the first target pixel point. The prediction formula for predicting the pixel value of the pixel point to be predicted is as follows:

[0109] ;

[0110] Wherein, represents the pixel value of the pixel point to be predicted, and at the same time represents that the pixel point to be predicted is located at the I-th row and the n-th column of any current frame image; represents the pixel value of the first target pixel point, and at the same time represents that the first target pixel point is located at the I-th row and the n-th column of the previous frame image; represents the image similarity between the upper half image and the lower half image of the previous frame image; A represents the total number of pixel rows of any current frame image; B represents the total number of pixel columns of any current frame image; represents the horizontal feature value of any pixel point; represents the vertical feature value of any pixel point, and at the same time represents that any pixel point is located at the J-th row and the m-th column of any current frame image. It should be noted that since any pixel point is symmetric with the pixel point to be predicted about the image center point of any current frame image, therefore, I + J = A + 1, and n + m = B + 1.

[0111] Based on this, when all the pixel points in the upper half image of any current frame image are received, the prediction of all the pixel points in the lower half image of any current frame image can be completed, so as to obtain the lower half image of any current frame image.

[0112] Beneficial effects: When receiving the upper half image of any current frame image, the present invention can predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image from the same data source, so that when all the pixel points in the upper half image of any current frame image are received, the prediction of the lower half image of any current frame image is completed. At this time, a complete current image to be superimposed can be pieced together. Compared with the time spent receiving the whole image, the present invention only needs to spend half of the time to obtain the complete image, which improves the acquisition speed of the image to be superimposed, and thus the image to be superimposed can be superimposed with a shorter time delay, so as to improve the speed of image superposition and the real-time performance of image superposition display.

[0113] SeeFigure 3 As shown in Figure 3 , an embodiment of the present invention discloses an image overlay method, which is applied to an image processor. The image processor includes two Random Access Memories (RAMs), two image similarity calculation modules, two image prediction modules, two image scaling modules, a display timing generation module, and a Multiplexer (MUX); and the image overlay method proposed by the present invention is as follows:

[0114] The image processor receives the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively through dual-channel image sampling, and stores the received images in their respective corresponding random access memories.

[0115] For any one of the first current frame image and the second current frame image, when the image processor receives the upper half of any one of the current frame images, the image prediction module predicts the lower half of any one of the current frame images based on the upper half of any one of the current frame images and the previous frame image. Among them, when predicting the lower half of any one of the current frame images, the image similarity between the upper half and the lower half of the previous frame image calculated by the image similarity calculation module is required.

[0116] Further, when the image prediction module finishes receiving the upper half of any one of the current frame images, it also completes the prediction of the lower half of any one of the current frame images, and then determines the complete current image to be overlaid based on the received upper half of any one of the current frame images and the predicted lower half of any one of the current frame images, and transmits the current image to be overlaid to the image scaling module.

[0117] The image scaling module uses the resolution of the display to perform bilinear interpolation on the received current image to be overlaid, so as to scale the current image to be overlaid, thereby obtaining the current scaled image, and storing the current scaled image in the corresponding random access memory.

[0118] After obtaining the current scaled images corresponding to the first data source and the second data source respectively, the display timing generation module overlays the current scaled images corresponding to the first data source and the second data source respectively to obtain the current overlaid image, determines the display timing parameters of the display based on the resolution of the display, and then outputs the current overlaid image to the display through the multiplexer according to the display timing parameters of the display for display.

[0119] It can be found that through dual-channel image sampling, dual-channel image prediction, and dual-channel image scaling, the present invention can significantly improve the image acquisition speed, image prediction speed, and image scaling speed, thereby performing image superposition within a short time delay to increase the image superposition speed.

[0120] Advantageous effects: Through dual-channel image sampling, the present invention simultaneously receives the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively, thereby improving the image reception speed. Moreover, when receiving the upper half of any current frame image, the present invention can predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image from the same data source. Thus, when the reception of the upper half of any current frame image is completed, the prediction of the lower half of any current frame image is completed. At this time, a complete current image to be superposed can be pieced together. Compared with the time spent receiving the entire image, the present invention only needs to spend half of the time to obtain a complete image, improving the image acquisition speed. Subsequently, the present invention superposes the current images to be superposed corresponding to the two data sources respectively to obtain the current superposed image. It can be found that by improving the acquisition speed of the images to be superposed, the present invention superposes the images to be superposed within a short time delay, thereby increasing the image superposition speed and the real-time performance of image superposition display.

[0121] Furthermore, the embodiment of the present application also discloses an electronic device. Figure 4 It is a structural diagram of an electronic device shown according to an exemplary embodiment. The content in the figure should not be regarded as any limitation on the scope of use of the present application. The electronic device may specifically include: at least one processor 11, at least one memory 12, a power supply 13, a communication interface 14, an input / output interface 15, and a communication bus 16. Among them, the memory 12 is used to store a computer program, and the computer program is loaded and executed by the processor 11 to implement the relevant steps in the image superposition method disclosed in any of the foregoing embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0122] In this embodiment, the power supply 13 is used to provide working voltage for each hardware device on the electronic device; the communication interface 14 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed here; the input / output interface 15 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and specific limitations are not imposed here.

[0123] In addition, as a carrier for storing resources, the memory 12 can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include an operating system 121, a computer program 122, etc., and the storage method can be transient storage or permanent storage.

[0124] Among them, the operating system 121 is used to manage and control each hardware device and the computer program 122 on the electronic device, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the image overlay method executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 122 can further include computer programs that can be used to complete other specific tasks.

[0125] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing disclosed image overlay method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0126] Furthermore, the present application also discloses a computer program product, including a computer program / instructions; wherein, when the computer program / instructions are executed by a processor, the foregoing disclosed image overlay method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0127] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method part.

[0128] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0129] The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination thereof. The software modules may be located in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0130] Finally, it should also be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0131] The technical solutions provided in this application have been introduced in detail above. Specific examples are used in this document to illustrate the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. An image superposition method, characterized in that: Applied to image processors, including: Receiving the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively, and when receiving the upper half image of any current frame image, predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when the upper half image of any current frame image is received; Determine a complete current image to be superimposed based on the received upper half image of any current frame image and the predicted lower half image of any current frame image; Superimposing the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image; Wherein, the previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image equally divide the any current frame image; The step of predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image includes: Predicting the lower half image of any current frame image by using the target association relationship and based on the upper half image of any current frame image and the previous frame image; The target association relationship includes a first association relationship between any current frame image and the previous frame image and a second association relationship between pixels in the same frame image.

2. The image superposition method according to claim 1, characterized in that: The method of using the target association relationship and predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image includes: Determine a pixel point to be predicted in a lower half image of any current frame image by using the second association relationship and based on any pixel point in an upper half image of any current frame image; Determine a corresponding first target pixel from the previous frame image by using the first association relationship and based on the pixel to be predicted; Based on the pixel value of any one pixel and the pixel value of the first target pixel, the pixel value of the pixel to be predicted is predicted.

3. The image superposition method according to claim 2, characterized in that: The any pixel point and the pixel point to be predicted are symmetrical about the image center point of any current frame image.

4. The image superposition method according to claim 3, characterized in that: The step of using the second association relationship and based on any pixel in the upper half of the image of any current frame image to determine the pixel to be predicted in the lower half of the image of any current frame image includes: Determine the first row number of the pixel row and the first column number of the pixel column where any pixel point is located in any current frame image; Based on the total number of pixel rows of any current frame image and the first row number, determine the second row number of the pixel row where the pixel to be predicted is located in any current frame image; the sum of the first row number and the second row number is equal to the total number of pixel rows plus one; Based on the total number of pixel columns of any current frame image and the first column number, determining a second column number of the pixel column where the pixel to be predicted is located in any current frame image; the sum of the first column number and the second column number is equal to the total number of pixel columns plus one; According to the second row number and the second column number, the pixel point to be predicted located in the lower half image of any current frame image is determined.

5. The image superposition method according to claim 2, characterized in that: The step of using the first association relationship and based on the pixel to be predicted to determine a corresponding first target pixel from the previous frame image includes: Determine a first pixel position of the pixel to be predicted in any current frame image; Based on the first pixel position, a pixel point at a corresponding position is determined from the previous frame image to obtain a first target pixel point.

6. The image superposition method according to claim 2, characterized in that: The predicting the pixel value of the pixel to be predicted based on the pixel value of any pixel and the pixel value of the first target pixel includes: Determine a feature value of any pixel point based on the pixel value of any pixel point; wherein the feature value of any pixel point includes a horizontal feature value and a vertical feature value; The pixel value of the pixel to be predicted is predicted based on the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel; wherein the image similarity is the similarity between the upper half image and the lower half image of the previous frame image.

7. The image superposition method according to claim 6, characterized in that: The determining the characteristic value of any pixel point based on the pixel value of any pixel point comprises: Determine a corresponding second target pixel point from the previous frame image by using the first association relationship and based on the any pixel point; Determine a horizontal feature value of any pixel point based on a pixel value of any pixel point and a pixel value of the second target pixel point; Determine a vertical eigenvalue of any pixel point based on the horizontal eigenvalue of any pixel point and the vertical eigenvalue of a third target pixel point; Among them, the third target pixel point is a pixel point in the previous pixel row of the pixel row where the any pixel point is located in any current frame image, and the third target pixel point and the any pixel point are located in the same pixel column in any current frame image.

8. The image superposition method according to claim 7, characterized in that: The determining the horizontal characteristic value of any pixel point based on the pixel value of any pixel point and the pixel value of the second target pixel point includes: Determine a first value based on the pixel value of any one pixel and the pixel value of the second target pixel; If the first value is less than a preset threshold, setting zero as the horizontal characteristic value of any pixel; If the first value is not less than a preset threshold, the first value is set as the horizontal characteristic value of any pixel point.

9. The image superposition method according to claim 6, characterized in that: The step of predicting the pixel value of the pixel to be predicted according to the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel includes: Determine a second value based on half of the total number of pixel rows of any current frame image and the horizontal characteristic value of any pixel point; Determine a third value based on half of the total number of pixel columns of any current frame image and the vertical characteristic value of any pixel point; The pixel value of the pixel to be predicted is predicted according to the image similarity, the second value, the third value and the pixel value of the first target pixel.

10. The image superposition method according to any one of claims 1 to 9, characterized in that: The step of superimposing the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image includes: Performing bilinear interpolation on the current images to be superimposed corresponding to the first data source and the second data source respectively using the resolution of the display, so as to scale the current images to be superimposed, and obtain current scaled images corresponding to the first data source and the second data source respectively; The current scaled images corresponding to the first data source and the second data source are superimposed to obtain a current superimposed image, and the current superimposed image is output to the display for display.

11. The image superposition method according to claim 10, characterized in that: Outputting the current superimposed image to the display for display includes: Determining display timing parameters of the display based on the resolution of the display; According to the display timing parameters of the display, the current superimposed image is output to the display for display.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the image overlay method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image overlay method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the image superposition method according to any one of claims 1 to 11 are implemented.

Images