Image superposition method and device, medium and product

Abstract

The invention discloses an image superposition method and device, a medium and a product, and relates to the field of image processing, and the method comprises the steps that an image processor receives a first current frame image and a second current frame image which are transmitted by two data sources respectively, and when the upper half part image of any current frame image is received, the first current frame image and the second current frame image are overlapped; predicting a lower half part image of any current frame image based on the upper half part image of any current frame image and the previous frame image, so as to complete the prediction of the lower half part image of any current frame image when the upper half part image of any current frame image is received, thereby obtaining a complete current to-be-superposed image; and then superposing the current to-be-superposed images respectively corresponding to the two data sources to obtain a current superposed image. According to the invention, on the basis of dual-channel image sampling and image prediction, the acquisition speed of the to-be-superposed image is improved, so that the to-be-superposed image is superposed in a short time delay, and thus the image superposition speed and the real-time performance of image superposition display are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image superposition method, device, medium and product. Background Art

[0002] The VGA (Video Graphic Array) display controller integrated in the BMC (Baseboard Management Controller) can process the local display screen of the host and output it to the monitor for display. At this time, the BMC is used as the graphics card of the host, and the graphics card is only controlled by the host. The SoC (System on Chip) display controller integrated in the BMC can process some custom screens and output them to the monitor for display. For example, the custom screen can be a screen generated based on the real-time status monitoring information of the host, or a screen generated based on the real-time status monitoring information of the BMC, etc. At this time, the BMC is no longer used as the graphics card of the host.

[0003] Since two display controllers are integrated inside the BMC, there are three ways to display the screen of the BMC. The first is to output screen 1 to the display for display through the VGA display controller, the second is to output screen 2 to the display for display through the SoC display controller, and the third is to superimpose screen 1 and screen 2, and output the superimposed screen to the display for display. As for the third method, currently, it is generally to write the two images before superposition into the DDR chip by connecting the two display controllers externally, and then re-read the two images in the DDR chip through the internal control logic for superposition processing, and then output the superimposed image to the display for display. However, this method of writing first, reading second, superimposing second, and displaying second is very likely to cause the problem of slow image superposition speed, thereby affecting the real-time performance of image superposition display. It can be seen that how to improve the speed of image superposition is a technical problem that technicians in this field need to solve. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an image superposition method, device, medium and product, which can improve the acquisition speed of the image to be superimposed based on dual-channel image sampling and image prediction, so as to superimpose the image to be superimposed within a shorter time delay, thereby improving the speed of image superposition and the real-time performance of image superposition display. The specific scheme is as follows:

[0005] In a first aspect, the present invention provides an image superposition method, which is applied to an image processor, comprising:

[0006] Receiving the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively, and predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image when receiving the upper half image of any current frame image, so as to complete the prediction of the lower half image of any current frame image when the upper half image of any current frame image is received;

[0007] Determine a complete current image to be superimposed based on the received upper half image of any current frame image and the predicted lower half image of any current frame image;

[0008] Superimposing the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image;

[0009] The previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image equally divide any current frame image.

[0010] Optionally, predicting a lower half image of any current frame image based on an upper half image of any current frame image and a previous frame image includes:

[0011] Predicting the lower half of any current frame image by using the target association relationship and based on the upper half of any current frame image and the previous frame image;

[0012] The target association relationship includes a first association relationship between any current frame image and a previous frame image and a second association relationship between pixels in the same frame image.

[0013] Optionally, using the target association relationship and based on the upper half image of any current frame image and the previous frame image, predicting the lower half image of any current frame image includes:

[0014] Determine a pixel point to be predicted in a lower half image of any current frame image by using the second association relationship and based on any pixel point in an upper half image of any current frame image;

[0015] Determine a corresponding first target pixel from a previous frame of image by using the first association relationship and based on the pixel to be predicted;

[0016] Based on the pixel value of any pixel and the pixel value of the first target pixel, the pixel value of the pixel to be predicted is predicted.

[0017] Optionally, any pixel point and the pixel point to be predicted are symmetrical about an image center point of any current frame image.

[0018] Optionally, using the second association relationship and based on any pixel in the upper half of any current frame image, determining a pixel to be predicted in the lower half of any current frame image includes:

[0019] Determine the first row number of the pixel row and the first column number of the pixel column where any pixel point is located in any current frame image;

[0020] Based on the total number of pixel rows and the first row number of any current frame image, determine the second row number of the pixel row where the pixel to be predicted is located in any current frame image; the sum of the first row number and the second row number is equal to the total number of pixel rows plus one;

[0021] Based on the total number of pixel columns and the first column number of any current frame image, determine the second column number of the pixel column where the pixel to be predicted is located in any current frame image; the sum of the first column number and the second column number is equal to the total number of pixel columns plus one;

[0022] According to the second row number and the second column number, a pixel point to be predicted located in the lower half image of any current frame image is determined.

[0023] Optionally, using the first association relationship and based on the pixel to be predicted, determining a corresponding first target pixel from the previous frame image includes:

[0024] Determine the first pixel position of the pixel to be predicted in any current frame image;

[0025] Based on the first pixel position, a pixel point at a corresponding position is determined from the previous frame image to obtain a first target pixel point.

[0026] Optionally, predicting the pixel value of the pixel to be predicted based on the pixel value of any pixel and the pixel value of the first target pixel includes:

[0027] Determine a feature value of any pixel point based on a pixel value of any pixel point; wherein the feature value of any pixel point includes a horizontal feature value and a vertical feature value;

[0028] The pixel value of the pixel to be predicted is predicted according to the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel; wherein the image similarity is the similarity between the upper half image and the lower half image of the previous frame image.

[0029] Optionally, determining a characteristic value of any pixel point based on a pixel value of any pixel point includes:

[0030] Determine a corresponding second target pixel point from the previous frame image by using the first association relationship and based on any pixel point;

[0031] Determine a horizontal feature value of any pixel based on a pixel value of any pixel and a pixel value of a second target pixel;

[0032] Determine a vertical eigenvalue of any pixel based on the horizontal eigenvalue of any pixel and the vertical eigenvalue of a third target pixel;

[0033] The third target pixel point is a pixel point located in a pixel row before a pixel row where any pixel point is located in any current frame image, and the third target pixel point and any pixel point are located in the same pixel column in any current frame image.

[0034] Optionally, determining the horizontal feature value of any pixel based on the pixel value of any pixel and the pixel value of the second target pixel includes:

[0035] Determine a first value based on a pixel value of any pixel point and a pixel value of a second target pixel point;

[0036] If the first value is less than a preset threshold, zero is set as the horizontal feature value of any pixel;

[0037] If the first value is not less than the preset threshold, the first value is set as the horizontal characteristic value of any pixel point.

[0038] Optionally, predicting the pixel value of the pixel to be predicted according to the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel includes:

[0039] Determine a second value based on half of the total number of pixel rows of any current frame image and a horizontal characteristic value of any pixel point;

[0040] Determine a third value based on half of the total number of pixel columns of any current frame image and a vertical characteristic value of any pixel point;

[0041] The pixel value of the pixel to be predicted is predicted according to the image similarity, the second value, the third value and the pixel value of the first target pixel.

[0042] Optionally, superimposing the current to-be-superimposed images corresponding to the first data source and the second data source respectively to obtain the current superimposed image includes:

[0043] Using the resolution of the display, bilinear interpolation is performed on the current images to be superimposed corresponding to the first data source and the second data source, respectively, so as to scale the current images to be superimposed, and obtain current scaled images corresponding to the first data source and the second data source, respectively;

[0044] The current scaled images corresponding to the first data source and the second data source are superimposed to obtain a current superimposed image, and the current superimposed image is output to a display for display.

[0045] Optionally, outputting the current superimposed image to a display for display, including:

[0046] Determine display timing parameters of the display based on the resolution of the display;

[0047] According to the display timing parameters of the display, the current superimposed image is output to the display for display.

[0048] In a second aspect, the present invention provides an electronic device, comprising:

[0049] Memory for storing computer programs;

[0050] A processor is used to execute a computer program to implement the steps of the aforementioned image overlay method.

[0051] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the aforementioned image overlay method are implemented.

[0052] In a fourth aspect, the present invention provides a computer program product, comprising a computer program / instruction, which implements the steps of the aforementioned image overlay method when executed by a processor.

[0053] In the present invention, an image processor receives a first current frame image and a second current frame image transmitted by a first data source and a second data source respectively, and when receiving an upper half image of any current frame image, predicts a lower half image of any current frame image based on the upper half image of any current frame image and a previous frame image, so as to complete the prediction of the lower half image of any current frame image when the upper half image of any current frame image is received; determines a complete current image to be superimposed based on the received upper half image of any current frame image and the predicted lower half image of any current frame image; superimposes the current images to be superimposed corresponding to the first data source and the second data source respectively, so as to obtain a current superimposed image; wherein the previous frame image and any current frame image correspond to the same data source; and the upper half image and the lower half image of any current frame image divide any current frame image equally.

[0054] Beneficial effects: The present invention uses dual-channel image sampling to simultaneously receive the first current frame image and the second current frame image transmitted by the first data source and the second data source, respectively, so as to improve the speed of image reception. In addition, when receiving the upper half of any current frame image, the present invention can predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image from the same data source, so that when the upper half of any current frame image is received, the prediction of the lower half of any current frame image is completed, and a complete current image to be superimposed can be pieced together at this time. Compared with the time spent on receiving the entire image, the present invention only takes half the time to obtain the complete image, thereby improving the image acquisition speed. Afterwards, the present invention superimposes the current images to be superimposed corresponding to the two data sources to obtain the current superimposed image. It can be found that the present invention superimposes the superimposed images within a shorter time delay by increasing the acquisition speed of the superimposed images to be superimposed, thereby improving the speed of image superposition and the real-time performance of image superposition display. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A flow chart of an image superposition method provided by an embodiment of the present invention;

[0057] Figure 2 An image prediction flow chart provided by an embodiment of the present invention;

[0058] Figure 3 An image overlay flow chart provided by an embodiment of the present invention;

[0059] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0062] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0063] For image superposition, currently, two display controllers are generally connected to an external DDR chip to write the two images before superposition into the DDR chip, and then the two images in the DDR chip are re-read through the internal control logic for superposition processing, and then the superimposed image is output to the display for display. However, this method of first writing-then reading-then superimposing-then displaying is very likely to cause the problem of slow image superposition speed, thereby affecting the real-time performance of image superposition display. To this end, the present invention provides an image superposition method, which can improve the acquisition speed of the image to be superimposed based on dual-channel image sampling and image prediction, so as to superimpose the image to be superimposed within a shorter time delay, thereby improving the speed of image superposition and the real-time performance of image superposition display.

[0064] See also Figure 1 As shown, an embodiment of the present invention discloses an image overlay method, which is applied to an image processor, comprising:

[0065] Step S11, receiving the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively, and when receiving the upper half image of any current frame image, predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when the upper half image of any current frame image is received; wherein the previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image divide any current frame image equally.

[0066] The image processor proposed in the embodiment of the present invention can be a processor that runs independently or integrated into other chips, thereby providing image processing capabilities for other chips. Taking the case where the image processor is integrated into the BMC baseboard management controller, and the first data source is the VGA display controller in the baseboard management controller, and the second data source is the SOC display controller in the baseboard management controller as an example, an image processor is externally connected to the VGA display controller and the SOC display controller in the baseboard management controller, so that the image processor receives the image transmitted by the VGA display controller and the image transmitted by the SOC display controller, and performs image prediction and image superposition, thereby achieving efficient and fast image superposition without adding an additional DDR chip.

[0067] Specifically, the image processor uses dual-channel image sampling to simultaneously receive the first current frame image and the second current frame image transmitted by the first data source and the second data source, respectively, and when receiving the upper half of any current frame image, the lower half of any current frame image is predicted based on the upper half of any current frame image and the previous frame image from the same data source as any current frame image. Furthermore, the embodiment of the present invention can complete the prediction of the lower half of any current frame image when the upper half of any current frame image is received, that is, compared with the time spent on receiving the entire image, the present invention only takes half the time to obtain a complete image, thereby improving the image acquisition speed.

[0068] It should be noted that any current frame image is any one of the first current frame image and the second current frame image, and the upper half image and the lower half image of any current frame image equally divide any current frame image. It should also be noted that the previous frame image and any current frame image come from the same data source and are located in the previous frame of any current frame image.

[0069] For any current frame image of the first current frame image and the second current frame image, it is necessary to use the target association relationship and predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image. That is, the embodiment of the present invention needs to use the target association relationship and predict the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image from the same data source as any current frame image.

[0070] The target association relationship includes a first association relationship between any current frame image and a previous frame image and a second association relationship between pixels in the same frame image. In this way, the embodiment of the present invention can predict the lower half of any current frame image based on the association relationship between two adjacent frames and the association relationship between pixels in the same frame image, which not only ensures the accuracy of image prediction, but also improves the image acquisition speed, thereby realizing image superposition within a shorter time delay.

[0071] Step S12: determining a complete current image to be superimposed based on the received upper half image of any current frame image and the predicted lower half image of any current frame image.

[0072] In an embodiment of the present invention, after predicting the lower half of any current frame image, the received upper half of any current frame image and the predicted lower half of any current frame image are spliced ​​to obtain a complete current image to be superimposed.

[0073] It should be noted that the embodiment of the present invention adopts dual-channel image sampling, and the corresponding prediction of the lower half of the image of the first current frame image and the prediction of the lower half of the image of the second current frame image are also dual-channel and parallel, which can improve the speed of image prediction to a certain extent.

[0074] Step S13: superimpose the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image.

[0075] In the embodiment of the present invention, after obtaining the complete current to-be-overlapped images corresponding to the first data source and the second data source respectively, the current to-be-overlapped images corresponding to the first data source and the second data source respectively are overlaid to obtain the current overlaid image.

[0076] Furthermore, considering that the resolution of the first current frame image, the resolution of the second current frame image and the resolution of the display may be different from each other, that is, the resolution of the current image to be superimposed corresponding to the first data source, the resolution of the current image to be superimposed corresponding to the second data source and the resolution of the display may be different from each other, therefore, in order to enable the superimposed image to be displayed through the display, it is necessary to first scale the current image to be superimposed so as to scale the current image to be superimposed to the resolution size of the display, and then perform image superposition on the scaled image.

[0077] Specifically, bilinear interpolation is performed on the current images to be superimposed corresponding to the first data source and the second data source respectively using the resolution of the display to scale the current images to be superimposed, thereby obtaining the current scaled images corresponding to the first data source and the second data source respectively. At this time, the resolutions of the current scaled images corresponding to the first data source and the second data source are equal to the resolution of the display. Then, the current scaled images corresponding to the first data source and the second data source are superimposed to obtain the current superimposed image, and the current superimposed image is output to the display for display. In this way, the embodiment of the present invention can make the superimposed image adapt to the display by scaling the current image to be superimposed and then superimposing it, thereby avoiding the problem of the superimposed image not matching the display.

[0078] It should be noted that, for scaling the current image to be superimposed, in addition to bilinear interpolation, nearest neighbor interpolation, bicubic interpolation, etc. can also be used, which is not specifically limited here and can be selected by the user according to processor performance and own needs.

[0079] Furthermore, outputting the current superimposed image to a display for display may specifically include: determining the display timing parameters of the display based on the resolution of the display, and then outputting the current superimposed image to the display for display according to the display timing parameters of the display. The display timing parameters of the display may also be understood as the row and field timing parameters of the display, which mainly include row timing parameters and field timing parameters; and the row timing parameters mainly include row synchronization, row blanking, row video validity, row front shoulder, etc., and the field timing parameters mainly include field synchronization, field blanking, field video validity, field front shoulder, etc.

[0080] It should be noted that, in the process of superimposing the current scaled images corresponding to the first data source and the second data source, respectively, considering that the image brightness, image contrast, etc. of the current scaled images corresponding to the first data source and the second data source may be different, therefore, in order to make the superimposed image look more natural, the present invention can first analyze the brightness and contrast of the current scaled images corresponding to the first data source and the second data source, respectively, to calculate the mean and standard deviation between the two scaled images, and then adjust the brightness and contrast of the two scaled images to a similar range based on the mean and standard deviation between the two scaled images, and then superimpose the adjusted two scaled images to obtain the initial superimposed image, and finally fine-tune the brightness and contrast of the initial superimposed image to obtain the final current superimposed image. In this way, the above method can make the transition of the superimposed image more natural and more visually coordinated, thereby further improving the user's perception.

[0081] Beneficial effects: The present invention uses dual-channel image sampling to simultaneously receive the first current frame image and the second current frame image transmitted by the first data source and the second data source, respectively, so as to improve the speed of image reception. In addition, when receiving the upper half of any current frame image, the present invention can predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image from the same data source, so that when the upper half of any current frame image is received, the prediction of the lower half of any current frame image is completed, and a complete current image to be superimposed can be pieced together at this time. Compared with the time spent on receiving the entire image, the present invention only takes half the time to obtain the complete image, thereby improving the image acquisition speed. Afterwards, the present invention superimposes the current images to be superimposed corresponding to the two data sources to obtain the current superimposed image. It can be found that the present invention superimposes the superimposed images within a shorter time delay by increasing the acquisition speed of the superimposed images to be superimposed, thereby improving the speed of image superposition and the real-time performance of image superposition display.

[0082] Based on the previous embodiment, it can be known that the previous embodiment describes the complete process from image reception and image prediction to image superposition. Next, this embodiment will explain in detail how to predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image. Figure 2 As shown, an embodiment of the present invention discloses a process of image prediction, including:

[0083] Step S21: Determine a pixel to be predicted in a lower half image of any current frame image by using the second association relationship and based on any pixel in an upper half image of any current frame image.

[0084] The embodiments of the present invention generally receive the upper half of any current frame image row by row and pixel by pixel. Therefore, the prediction of the lower half of any current frame image is also performed pixel by pixel.

[0085] Based on this, when receiving the upper half image of any current frame image, for any pixel point in the upper half image of any current frame image, it is necessary to first determine the pixel point to be predicted in the lower half image of any current frame image based on any pixel point.

[0086] Specifically, the second association relationship between the pixels in the same frame image is used and based on any pixel in the upper half of any current frame image, a pixel to be predicted located in the lower half of any current frame image is determined. Moreover, any pixel and the pixel to be predicted are symmetrical about the image center point of any current frame image.

[0087] According to one of the embodiments, for any pixel point in the upper half of any current frame image, first determine the first row number of the pixel row and the first column number of the pixel column where any pixel point is located in any current frame image; then determine the second row number of the pixel row where the pixel point to be predicted is located in any current frame image based on the total number of pixel rows and the first row number of any current frame image, wherein the sum of the first row number and the second row number is equal to the total number of pixel rows plus one; determine the second column number of the pixel column where the pixel point to be predicted is located in any current frame image based on the total number of pixel columns and the first column number of any current frame image, wherein the sum of the first column number and the second column number is equal to the total number of pixel columns plus one; finally, determine the pixel point to be predicted in the lower half of any current frame image based on the second row number and the second column number. In this way, the determined pixel point to be predicted can be symmetrical with any pixel point about the image center point of any current frame image in the above manner.

[0088] Exemplarily, if any current frame image is an image with a resolution of 640*480, the total number of pixel rows of any current frame image is 480 rows, and the total number of pixel columns is 640 columns. Taking any pixel point as a pixel point in the second row and the first column of any current frame image as an example, determine the first row number of the pixel row and the first column number of the pixel column where any pixel point is located in any current frame image, at this time, the first row number is 2 and the first column number is 1; then based on the total number of pixel rows and the first row number of any current frame image, determine the second row number of the pixel row where the pixel point to be predicted is located in any current frame image, at this time, the second row number=the total number of pixel rows+1-the first row number=480+1-2=479; then based on the total number of pixel columns and the first column number of any current frame image, determine the second column number of the pixel column where the pixel point to be predicted is located in any current frame image, at this time, the second column number=the total number of pixel columns+1-the first column number=640+1-1=640; finally, according to the second row number and the second column number, determine the pixel point to be predicted in the lower half image of any current frame image, that is, the pixel point to be predicted is the pixel point in the 479th row and the 640th column of any current frame image.

[0089] Step S22: using the first association relationship and based on the pixel to be predicted, determine a corresponding first target pixel from the previous frame image.

[0090] In an embodiment of the present invention, after determining the pixel point to be predicted located in the lower half of any current frame image, the corresponding first target pixel point is determined from the previous frame image by utilizing the first association relationship between any current frame image and the previous frame image and based on the pixel point to be predicted.

[0091] Specifically, the first pixel position of the pixel to be predicted in any current frame image is determined, and based on the first pixel position, the pixel at the corresponding position is determined from the previous frame image to obtain the first target pixel. That is, the pixel position of the pixel to be predicted in any current frame image is the same as the pixel position of the first target pixel in the previous frame image.

[0092] Exemplarily, if the pixel to be predicted is the pixel at the 479th row and 640th column in any current frame image, then the first pixel position of the pixel to be predicted in any current frame image is (479th row, 640th column), and then the pixel at the 479th row and 640th column is determined from the previous frame image to obtain the first target pixel.

[0093] Step S23: predicting the pixel value of the pixel to be predicted based on the pixel value of any pixel and the pixel value of the first target pixel.

[0094] In the embodiment of the present invention, after the first target pixel is determined from the previous frame image, the pixel value of the pixel to be predicted is predicted based on the pixel value of any pixel and the pixel value of the first target pixel.

[0095] Specifically, the feature value of any pixel is determined based on the pixel value of any pixel; wherein the feature value of any pixel includes a horizontal feature value and a vertical feature value; then the pixel value of the pixel to be predicted is predicted based on the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel; wherein the image similarity is the similarity between the upper half image and the lower half image of the previous frame image.

[0096] It should be noted that the image similarity between the upper half image and the lower half image of the previous frame image can be calculated using SSIM (Structural Similarity), and the specific calculation method is as follows:

[0097] ;

[0098] in, Represents the upper half of the previous frame image; Represents the lower half of the previous frame image; Indicates the image similarity between the upper half image and the lower half image of the previous frame image; Represents the mean value of the upper half of the previous frame image; Represents the mean value of the lower half of the previous frame image; Represents the variance of the upper half of the previous frame image; Represents the variance of the lower half of the previous frame image; Represents the covariance between the upper half and the lower half of the previous frame image; and Represents a constant, used to avoid the denominator being 0.

[0099] As for determining the horizontal eigenvalue and the vertical eigenvalue of any pixel point, it may specifically include: first utilizing the first association relationship between any current frame image and the previous frame image and based on any pixel point, determining the corresponding second target pixel point from the previous frame image; then determining the horizontal eigenvalue of any pixel point based on the pixel value of any pixel point and the pixel value of the second target pixel point; and determining the vertical eigenvalue of any pixel point based on the horizontal eigenvalue of any pixel point and the vertical eigenvalue of a third target pixel point; wherein the third target pixel point is a pixel point in the previous pixel row of the pixel row where any pixel point is located in any current frame image, and the third target pixel point and any pixel point are located in the same pixel column in any current frame image.

[0100] Among them, for the second target pixel point, it is specifically necessary to first determine the second pixel position of any pixel point in any current frame image, and determine the pixel point at the corresponding position from the previous frame image based on the second pixel position to obtain the second target pixel point. That is, the pixel position of any pixel point in any current frame image is the same as the pixel position of the second target pixel point in the previous frame image.

[0101] Moreover, the position of the third target pixel point in any current frame image is located in the same column of the previous row of any pixel point. Taking any pixel point as a pixel point in the second row and the first column of any current frame image as an example, the third target pixel point is a pixel point in the first row and the first column of any current frame image.

[0102] Determining the horizontal characteristic value of any pixel point may specifically include: determining a first value based on the pixel value of any pixel point and the pixel value of a second target pixel point; judging whether the first value is less than a preset threshold, and if the first value is less than the preset threshold, setting zero as the horizontal characteristic value of any pixel point; and if the first value is not less than the preset threshold, setting the first value as the horizontal characteristic value of any pixel point. It should be noted that the preset threshold is a configurable value.

[0103] Among them, for determining the first numerical value, taking the pixel value including RGB (Red, Green, Blue, three primary colors) value as an example, determine the difference between the pixel value of any pixel point on any channel and the pixel value of the second target pixel point on any channel, wherein any channel is any one of the three channels R, G, and B, and then determine the first numerical value based on the difference corresponding to the three channels R, G, and B respectively.

[0104] In a first specific implementation, the first numerical value is equal to the sum of the differences corresponding to the three channels R, G, and B, that is, the first numerical value = (Rc-Rp) + (Gc-Gp) + (Bc-Bp), wherein Rc represents the pixel value of any pixel point on the R channel, Rp represents the pixel value of the second target pixel point on the R channel, Gc represents the pixel value of any pixel point on the G channel, Gp represents the pixel value of the second target pixel point on the G channel, Bc represents the pixel value of any pixel point on the B channel, and Bp represents the pixel value of the second target pixel point on the B channel.

[0105] In the second specific implementation, the sum of the differences corresponding to the three channels R, G, and B is determined, and then the sum is averaged to obtain the first value, that is, the first value = [(Rc-Rp) + (Gc-Gp) + (Bc-Bp)] / 3. It should be noted that the preset threshold corresponding to the first specific implementation = three times the preset threshold corresponding to the second specific implementation.

[0106] The determination of the vertical eigenvalue of any pixel may specifically include: determining the vertical eigenvalue of any pixel based on the horizontal eigenvalue of any pixel, the vertical eigenvalue of the third target pixel, and a configurable parameter; wherein the configurable parameter is a value not less than 0 and not greater than 1. That is, the vertical eigenvalue of any pixel = a × the horizontal eigenvalue of any pixel + (1-a) × the vertical eigenvalue of the third target pixel; wherein a represents the configurable parameter, and the configurable parameter is used to characterize the correlation characteristics between the pixels of the upper and lower rows in the same frame image.

[0107] It should be noted that since the position of the third target pixel point in any current frame image is located in the same column of the previous row of any pixel point, if any pixel point is located in the first row of any current frame image, there is no corresponding third target pixel point for any pixel point. At this time, the vertical eigenvalue of any pixel point = a×the horizontal eigenvalue of any pixel point.

[0108] Furthermore, the prediction of the pixel value of the pixel to be predicted may specifically include: determining the second value based on half of the total number of rows of pixels in any current frame image and the horizontal characteristic value of any pixel; determining the third value based on half of the total number of columns of pixels in any current frame image and the vertical characteristic value of any pixel; and then predicting the pixel value of the pixel to be predicted based on the pre-calculated image similarity between the upper half image and the lower half image of the previous frame image, the second value, the third value and the pixel value of the first target pixel. Among them, the prediction formula for predicting the pixel value of the pixel to be predicted is as follows:

[0109] ;

[0110] in, represents the pixel value of the pixel to be predicted, and also represents that the pixel to be predicted is located in the Ith row and the nth column of any current frame image; represents the pixel value of the first target pixel, and also represents that the first target pixel is located in the Ith row and the nth column of the previous frame image; Indicates the image similarity between the upper half and the lower half of the previous frame image; A indicates the total number of pixel rows in any current frame image; B indicates the total number of pixel columns in any current frame image; Represents the horizontal feature value of any pixel; It represents the vertical eigenvalue of any pixel point, and also represents that any pixel point is located in the Jth row and the mth column of any current frame image. It should be noted that since any pixel point and the pixel point to be predicted are symmetrical about the image center point of any current frame image, I+J=A+1, and n+m=B+1.

[0111] Based on this, when all the pixels in the upper half of any current frame image are received, the prediction of all the pixels in the lower half of any current frame image can be completed, thereby obtaining the lower half of any current frame image.

[0112] Beneficial effect: When receiving the upper half of any current frame image, the present invention can predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image from the same data source, so that when the upper half of any current frame image is received, the prediction of the lower half of any current frame image is completed, and a complete current image to be superimposed can be pieced together. Compared with the time spent on receiving the entire image, the present invention only takes half the time to obtain the complete image, thereby improving the acquisition speed of the image to be superimposed, so that the image to be superimposed can be superimposed within a shorter time delay, so as to improve the speed of image superposition and the real-time performance of image superposition display.

[0113] See also Figure 3 As shown, an embodiment of the present invention discloses an image superposition method, which is applied to an image processor, wherein the image processor includes two random access memories (Random Access Memory, RAM), two image similarity calculation modules, two image prediction modules, two image scaling modules, a display timing generation module and a multiplexer (MUX); and the image superposition method proposed by the present invention is specifically as follows:

[0114] The image processor receives the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively at the same time through dual-channel image sampling, and stores the received images in their corresponding random access memories respectively.

[0115] For any current frame image of the first current frame image and the second current frame image, when the image processor receives the upper half image of any current frame image, the image processor predicts the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image through the image prediction module. When predicting the lower half image of any current frame image, the image similarity between the upper half image and the lower half image of the previous frame image calculated by the image similarity calculation module is required.

[0116] Furthermore, when the upper half of any current frame image is received, the image prediction module also completes the prediction of the lower half of any current frame image, and then determines the complete current image to be superimposed based on the received upper half of any current frame image and the predicted lower half of any current frame image, and transmits the current image to be superimposed to the image scaling module.

[0117] The image scaling module performs bilinear interpolation on the received current image to be superimposed using the resolution of the display to scale the current image to be superimposed, thereby obtaining the current scaled image, and stores the current scaled image in a corresponding random access memory.

[0118] After obtaining the current scaled images corresponding to the first data source and the second data source respectively, the display timing generation module superimposes the current scaled images corresponding to the first data source and the second data source respectively to obtain the current superimposed image, and determines the display timing parameters of the display based on the resolution of the display, and then outputs the current superimposed image to the display through a multiplexer for display according to the display timing parameters of the display.

[0119] It can be found that the present invention can significantly improve the image acquisition speed, image prediction speed and image scaling speed through dual-channel image sampling, dual-channel image prediction and dual-channel image scaling, thereby performing image superposition within a shorter time delay to improve the speed of image superposition.

[0120] Beneficial effects: The present invention uses dual-channel image sampling to simultaneously receive the first current frame image and the second current frame image transmitted by the first data source and the second data source, respectively, so as to improve the speed of image reception. In addition, when receiving the upper half of any current frame image, the present invention can predict the lower half of any current frame image based on the upper half of any current frame image and the previous frame image from the same data source, so that when the upper half of any current frame image is received, the prediction of the lower half of any current frame image is completed, and a complete current image to be superimposed can be pieced together at this time. Compared with the time spent on receiving the entire image, the present invention only takes half the time to obtain the complete image, thereby improving the image acquisition speed. Afterwards, the present invention superimposes the current images to be superimposed corresponding to the two data sources to obtain the current superimposed image. It can be found that the present invention superimposes the superimposed images within a shorter time delay by increasing the acquisition speed of the superimposed images to be superimposed, thereby improving the speed of image superposition and the real-time performance of image superposition display.

[0121] Furthermore, the present application also discloses an electronic device. Figure 4 This is a structural diagram of an electronic device according to an exemplary embodiment, and the content in the diagram cannot be considered as any limitation on the scope of use of this application. The electronic device may specifically include: at least one processor 11, at least one memory 12, a power supply 13, a communication interface 14, an input / output interface 15, and a communication bus 16. Among them, the memory 12 is used to store a computer program, and the computer program is loaded and executed by the processor 11 to implement the relevant steps in the image overlay method disclosed in any of the aforementioned embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0122] In this embodiment, the power supply 13 is used to provide working voltage for each hardware device on the electronic device; the communication interface 14 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 15 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0123] In addition, the memory 12, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 121, a computer program 122, etc., and the storage method can be temporary storage or permanent storage.

[0124] The operating system 121 is used to manage and control various hardware devices on the electronic device and the computer program 122, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the image overlay method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 122 can further include a computer program that can be used to complete other specific tasks.

[0125] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed image overlay method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0126] Furthermore, the present application also discloses a computer program product, including a computer program / instruction; wherein the computer program / instruction, when executed by a processor, implements the aforementioned disclosed image overlay method. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0127] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0128] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0129] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0130] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0131] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image superposition method, characterized in that: Applied to image processors, including: Receiving the first current frame image and the second current frame image transmitted by the first data source and the second data source respectively, and when receiving the upper half image of any current frame image, predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image, so as to complete the prediction of the lower half image of any current frame image when the upper half image of any current frame image is received; Determine a complete current image to be superimposed based on the received upper half image of any current frame image and the predicted lower half image of any current frame image; Superimposing the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image; The previous frame image and any current frame image correspond to the same data source; the upper half image and the lower half image of any current frame image equally divide the any current frame image.

2. The image superposition method according to claim 1, characterized in that: The predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image includes: Predicting the lower half image of any current frame image by using the target association relationship and based on the upper half image of any current frame image and the previous frame image; The target association relationship includes a first association relationship between any current frame image and the previous frame image and a second association relationship between pixels in the same frame image.

3. The image superposition method according to claim 2, characterized in that: The method of using the target association relationship and predicting the lower half image of any current frame image based on the upper half image of any current frame image and the previous frame image includes: Determine a pixel point to be predicted in a lower half image of any current frame image by using the second association relationship and based on any pixel point in an upper half image of any current frame image; Determine a corresponding first target pixel from the previous frame image by using the first association relationship and based on the pixel to be predicted; Based on the pixel value of any one pixel and the pixel value of the first target pixel, the pixel value of the pixel to be predicted is predicted.

4. The image superposition method according to claim 3, characterized in that: The any pixel point and the pixel point to be predicted are symmetrical about the image center point of any current frame image.

5. The image superposition method according to claim 4, characterized in that: The step of using the second association relationship and based on any pixel in the upper half of the image of any current frame image to determine the pixel to be predicted in the lower half of the image of any current frame image includes: Determine the first row number of the pixel row and the first column number of the pixel column where any pixel point is located in any current frame image; Based on the total number of pixel rows of any current frame image and the first row number, determine the second row number of the pixel row where the pixel to be predicted is located in any current frame image; the sum of the first row number and the second row number is equal to the total number of pixel rows plus one; Based on the total number of pixel columns of any current frame image and the first column number, determining a second column number of the pixel column where the pixel to be predicted is located in any current frame image; the sum of the first column number and the second column number is equal to the total number of pixel columns plus one; According to the second row number and the second column number, the pixel point to be predicted located in the lower half image of any current frame image is determined.

6. The image superposition method according to claim 3, characterized in that: The step of using the first association relationship and based on the pixel to be predicted to determine a corresponding first target pixel from the previous frame image includes: Determine a first pixel position of the pixel to be predicted in any current frame image; Based on the first pixel position, a pixel point at a corresponding position is determined from the previous frame image to obtain a first target pixel point.

7. The image superposition method according to claim 3, characterized in that: The predicting the pixel value of the pixel to be predicted based on the pixel value of any pixel and the pixel value of the first target pixel includes: Determine a feature value of any pixel point based on the pixel value of any pixel point; wherein the feature value of any pixel point includes a horizontal feature value and a vertical feature value; The pixel value of the pixel to be predicted is predicted based on the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel; wherein the image similarity is the similarity between the upper half image and the lower half image of the previous frame image.

8. The image superposition method according to claim 7, characterized in that: The determining the characteristic value of any pixel point based on the pixel value of any pixel point comprises: Determine a corresponding second target pixel point from the previous frame image by using the first association relationship and based on the any pixel point; Determine a horizontal feature value of any pixel point based on a pixel value of any pixel point and a pixel value of the second target pixel point; Determine a vertical eigenvalue of any pixel point based on the horizontal eigenvalue of any pixel point and the vertical eigenvalue of a third target pixel point; Among them, the third target pixel point is a pixel point in the previous pixel row of the pixel row where the any pixel point is located in any current frame image, and the third target pixel point and the any pixel point are located in the same pixel column in any current frame image.

9. The image superposition method according to claim 8, characterized in that: The determining the horizontal characteristic value of any pixel point based on the pixel value of any pixel point and the pixel value of the second target pixel point includes: Determine a first value based on the pixel value of any one pixel and the pixel value of the second target pixel; If the first value is less than a preset threshold, setting zero as the horizontal characteristic value of any pixel; If the first value is not less than a preset threshold, the first value is set as the horizontal characteristic value of any pixel point.

10. The image superposition method according to claim 7, characterized in that: The step of predicting the pixel value of the pixel to be predicted according to the pre-calculated image similarity, the feature value of any pixel and the pixel value of the first target pixel includes: Determine a second value based on half of the total number of pixel rows of any current frame image and the horizontal characteristic value of any pixel point; Determine a third value based on half of the total number of pixel columns of any current frame image and the vertical characteristic value of any pixel point; The pixel value of the pixel to be predicted is predicted according to the image similarity, the second value, the third value and the pixel value of the first target pixel.

11. The image superposition method according to any one of claims 1 to 10, characterized in that: The step of superimposing the current images to be superimposed corresponding to the first data source and the second data source respectively to obtain a current superimposed image includes: Performing bilinear interpolation on the current images to be superimposed corresponding to the first data source and the second data source respectively using the resolution of the display, so as to scale the current images to be superimposed, and obtain current scaled images corresponding to the first data source and the second data source respectively; The current scaled images corresponding to the first data source and the second data source are superimposed to obtain a current superimposed image, and the current superimposed image is output to the display for display.

12. The image superposition method according to claim 11, characterized in that: Outputting the current superimposed image to the display for display includes: Determining display timing parameters of the display based on the resolution of the display; According to the display timing parameters of the display, the current superimposed image is output to the display for display.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the image overlay method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image superposition method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the image superposition method according to any one of claims 1 to 12 are implemented.

Images