Method, device, equipment and medium for displaying projection images

Abstract

The disclosed embodiments relate to a method, device, equipment and medium for displaying a projection screen, wherein the method comprises: obtaining a current projection video frame, determining at least one reference pixel column in the current projection video frame; determining the current display direction of the current projection video frame according to at least one reference pixel column; if the display direction is a vertical screen direction, identifying the edge column position of the screen content in the current projection video frame; cropping the current projection video frame according to the edge column position to obtain a projection screen frame containing the screen content, and controlling the projection target screen to display the projection screen frame. Thus, a simple and efficient method for determining the projection display direction is realized, and the "black edge" area that does not contain the screen content is cropped in the vertical screen direction, thereby improving the visual display effect.

Description

Technical Field

[0001] The present disclosure relates to the field of communication application technology, and in particular to a method, device, equipment and medium for displaying a projection image. Background Art

[0002] Screen projection is a technology that projects screen images onto other display devices for display. It is widely used in daily production and life. For example, the video frames you are watching on your mobile phone can be viewed on a TV screen through screen projection technology.

[0003] When a mobile phone or other screen casting initiator initiates screen casting to a TV or other screen casting target device, the screen casting target device displays the screen in the same direction as the screen casting initiator. Figure 1A As shown in the figure, if the screen display direction of the screen projection initiating device is horizontal, the screen projection display direction on the screen projection target device is also horizontal. In the horizontal screen display mode, the screen projection image can relatively fill the screen of the screen projection target device and does not affect the visual display effect; however, if Figure 1B As shown in the figure, if the screen display direction of the screen projection initiating device is vertical, the screen projection display direction on the screen projection target device is also vertical. In the vertical display mode, the screen projection contains a lot of black "black border" filling area, which affects the visual display effect. In the related art, due to the limitation of the screen projection protocol, it is impossible to directly read the screen display direction of the screen projection initiating device, and thus, it is impossible to directly know the screen projection display direction on the screen projection target device.

[0004] Therefore, there is an urgent need for a method that can efficiently determine the projection display direction on the projection target device, so as to crop the "black border" area in the vertical screen display mode to ensure the visual display effect. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, device, equipment and medium for displaying a projection screen, which realizes a simple and efficient method for determining the projection display direction, and cuts out the "black border" area that does not contain the screen content in the vertical screen orientation, thereby improving the visual display effect.

[0006] An embodiment of the present disclosure provides a method for displaying a projection screen, the method comprising the following steps: obtaining a current projection video frame, and determining at least one reference pixel column in the current projection video frame; determining a current display direction of the current projection video frame based on the at least one reference pixel column; if the current display direction is a vertical screen direction, identifying an edge column position of a screen content in the current projection video frame; cropping the current projection video frame according to the edge column position to obtain a projection screen frame containing the screen content, and controlling the projection target screen to display the projection screen frame.

[0007] The presently disclosed embodiment also provides a device for displaying a projected screen, the device comprising: a first determination module, used to obtain a current projected screen video frame, and determine at least one reference pixel column in the current projected screen video frame; a second determination module, used to determine a current display direction of the current projected screen video frame based on the at least one reference pixel column; an identification module, used to identify an edge column position of a screen content in the current projected screen video frame when the current display direction is a vertical screen direction; a cropping module, used to crop the current projected screen video frame according to the edge column position to obtain a projected screen frame containing the screen content; and a projection module, used to control a projection target screen to display the projection screen frame.

[0008] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a method for displaying a projection screen as provided in an embodiment of the present disclosure.

[0009] An embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the method for displaying a projection screen as provided in the embodiment of the present disclosure.

[0010] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0011] The display scheme of the projection screen provided by the embodiment of the present disclosure obtains the current projection screen video frame, determines at least one reference pixel column in the current projection screen video frame, determines the current display direction of the current projection screen video frame according to the at least one reference pixel column, and then, if the current display direction is the portrait screen direction, identifies the edge column position of the screen content in the current projection screen video frame, crops the current projection screen video frame according to the edge column position to obtain the projection screen frame containing the screen content, and controls the projection target screen to display the projection screen frame. Thus, a simple and efficient method for determining the projection screen display direction is realized, and the "black border" area that does not contain the screen content is cropped in the portrait screen direction, thereby improving the visual display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0013] Figure 1A A schematic diagram of a display scene of a projection screen in the prior art provided in an embodiment of the present disclosure;

[0014] Figure 1B A schematic diagram of another display scene of a projection screen in the prior art provided in an embodiment of the present disclosure;

[0015] Figure 2 A schematic diagram of a flow chart of a method for displaying a projection screen provided in an embodiment of the present disclosure;

[0016] Figure 3A A schematic diagram of determining the position of a reference pixel column provided in an embodiment of the present disclosure;

[0017] Figure 3B A schematic diagram of another reference pixel column position determination provided by an embodiment of the present disclosure;

[0018] Figure 3C A schematic diagram of another reference pixel column position determination provided by an embodiment of the present disclosure;

[0019] Figure 4 A schematic diagram of a display scenario of a projection screen provided in an embodiment of the present disclosure;

[0020] Figure 5 A schematic diagram of a flow chart of another method for displaying a projection screen provided by an embodiment of the present disclosure;

[0021] Figure 6 A schematic diagram of another display scenario of a projection screen provided in an embodiment of the present disclosure;

[0022] Figure 7 A schematic diagram of a flow chart of another method for displaying a projection screen provided by an embodiment of the present disclosure;

[0023] Figure 8 A schematic diagram of another display scenario of a projection screen provided in an embodiment of the present disclosure;

[0024] Fig. 9 A schematic diagram of a flow chart of another method for displaying a projection screen provided by an embodiment of the present disclosure;

[0025] Fig.10 A schematic diagram of a flow chart of another method for displaying a projection screen provided by an embodiment of the present disclosure;

[0026] Fig.11 A schematic diagram of another display scenario of a projection screen provided in an embodiment of the present disclosure;

[0027] Fig.12 A schematic diagram of the structure of a display device for projecting a screen provided by an embodiment of the present disclosure;

[0028] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] In order to solve the above problems, the embodiments of the present disclosure provide a method for displaying a projected screen. In this method, not only the display direction of the projected screen on the target device is confirmed, but also in the method for confirming the display direction of the projected screen, only a reference pixel column is needed as a sampling pixel column, and the display direction is determined based on the sampling pixel column. No complex algorithm is required, thereby realizing a simple and efficient method for determining the display direction of the projected screen, which provides convenience for cutting out the "black border" area that does not contain the screen content in the vertical screen orientation, thereby improving the visual display effect.

[0036] The method is introduced below in conjunction with specific embodiments.

[0037] Figure 2 The present invention provides a flowchart of a method for displaying a projection screen according to an embodiment of the present invention. The method can be executed by a display device for the projection screen, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 2 As shown, the method includes:

[0038] Step 201, obtain the current projection video frame, and determine at least one reference pixel column in the current projection video frame.

[0039] Among them, the current projection video frame is generated to adapt to the logical resolution corresponding to the projection protocol of the projection target device, and is a video frame containing the screen content projected by the projection initiating device; for example, if the logical resolution corresponding to the projection protocol of the projection target device is 1920*1080, then the logical resolution of the current projection video frame is 1920*1080, so that the current projection video frame can be displayed on the corresponding projection target device. Therefore, in order to adapt to the logical resolution based on the projection protocol, the corresponding current projection video frame may contain a "black border" area in addition to the screen content projected by the projection initiating device, where the "black border" area is usually an area filled with black meaningless pixels, and in the vertical screen display direction, it is usually symmetrically distributed on the left and right sides of the current projection video frame.

[0040] Considering that the current projection video frame in the vertical screen display direction may contain a large number of "black edge" areas, it is necessary to identify the display direction of the current projection video frame.

[0041] In one embodiment of the present disclosure, at least one reference pixel column is determined in the current projection video frame, and the display direction of the current projection video frame is determined with reference to the reference pixel column, wherein the reference pixel column may be located at the positions where the "black border" areas are usually located, such as the two sides of the current video frame.

[0042] It should be emphasized that, as mentioned above, in the vertical screen display direction, the "black border" area is usually symmetrically distributed on the left and right sides of the current projection video frame. Therefore, if the reference pixel column is extracted close to the edge column position of the current projection video frame, it can be guaranteed that the reference pixel column is in the "black border" area in the vertical screen direction, which is convenient for further determining the display direction of the current projection video frame based on the reference pixel column. Conversely, if the reference pixel column is extracted far away from the edge column position of the current projection video frame, since the reference pixel column may be on the picture content, the reference pixel column cannot be used to confirm the image characteristics of the "black border" area.

[0043] In one embodiment of the present disclosure, in order to ensure the accuracy of display direction confirmation, the ratio of the number of columns between each reference pixel column and the start column or the end column of the current projection video frame to the total number of columns is less than a first preset ratio threshold, wherein the first preset ratio threshold can be determined according to the logical resolution of the current projection video frame mentioned above. Usually, the first preset ratio threshold is less than one-third, for example, the first preset ratio can be one-fifth, etc. In this embodiment, each reference pixel column needs to be close to the start column of the current projection video frame, or close to the end column of the current projection video frame, wherein the start column and the end column can be regarded as the left and right edge columns of the current projection video frame. If extracted from left to right, the leftmost edge column is the start column and the rightmost edge column is the end column. If extracted from right to left, the leftmost edge column is the end column and the left and right edge columns are the start column.

[0044] It should be understood that, since the “black border” area is usually distributed symmetrically in the vertical screen display direction, the at least one reference pixel column may be only one, for example, Figure 3A As shown, the reference pixel column may be located at one fifth of the total number of columns of the current projection video frame; for another example, in order to improve the accuracy of determining the display direction, at least one reference pixel column may be multiple, for example, Figure 3B As shown, the reference pixel column may be located at one fifth and four fifths of the total number of columns of the current projection video frame, or, as shown in Figure 3C As shown, there can be three reference pixel columns, which are located at one-fifth, one-sixth, and four-fifths of the total number of columns of the current projection video frame.

[0045] Step 202, determining a current display direction of a current projection video frame based on at least one reference pixel column.

[0046] In one embodiment of the present disclosure, after determining at least one reference pixel column, since at least one reference pixel column is usually in the "black border" area in the vertical screen display direction, in order to determine the display direction of the current projected video frame, the current display direction of the current projected video frame is determined according to at least one reference pixel column. The specific determination method refers to the subsequent embodiments and will not be repeated here.

[0047] Step 203, if the current display direction is the vertical screen direction, identify the edge column position of the picture content in the current projection video frame.

[0048] In one embodiment of the present disclosure, when it is determined that the current display direction is a portrait screen direction, that is, there are "black border" areas distributed on the left and right in the current projected video frame, the edge column position of the picture content is determined, wherein the picture content corresponds to the video frame picture projected on the projection initiating device, and therefore the black side content is retained and other areas are cropped to enhance the visual display effect, and the edge column position coincides with the left and right boundaries of the picture content.

[0049] Of course, in some possible embodiments, considering that the current display direction may be caused by a brief jitter of the screen projection initiating device, in order to eliminate the misjudgment caused by such jitter, in this embodiment, anti-shake processing is also performed before identifying the edge column position of the picture content in the current screen projection video frame.

[0050] In this embodiment, it is obtained whether the current display direction is consistent with the historical display direction of the previous projection video frame. If they are consistent, it is considered that no jitter occurs at present, and thus, the current display direction is determined to be the vertical screen direction.

[0051] If they are inconsistent, further determine whether the vertical screen direction is caused by the jitter of the projection initiating device, and obtain the continuous multiple-frame projection video frames adjacent to the current projection video frame, where the number of continuous multiple-frame projection video frames can be calibrated according to experiments, for example, it can be 10, etc., and identify multiple reference display directions corresponding to the continuous multiple-frame projection video frames, where the method for determining the reference display direction can refer to the method for determining the display direction of the current projection video frame, which will not be repeated here. If it is determined that the multiple reference display directions are consistent with the current display direction, it is considered that the vertical screen direction is not caused by device jitter. Therefore, further black edge removal processing is performed. Otherwise, the current display direction is corrected according to the display direction of the previous projection video frame.

[0052] Step 204, cropping the current projection video frame according to the edge column position to obtain a projection picture frame containing the picture content, and controlling the projection target screen to display the projection picture frame.

[0053] In this embodiment, after determining the edge column position, the video frame area outside the edge column position is cropped to obtain a projection screen frame containing the screen content, thereby achieving the removal of the "black border" area and improving the visual display effect.

[0054] It is not difficult to understand that the resolution of the projection screen frame after cutting off the "black border" area will inevitably be reduced. Therefore, after the projection screen frame can be displayed on the projection target screen on the projection target device, the remaining screen area is exposed to meet other display needs.

[0055] In one embodiment of the present disclosure, a projection display area corresponding to the projection picture frame on the projection target screen is determined, and Figure 4 As shown, the projection screen frame is displayed in the projection display area, wherein the projection display area can be located on the left side of the projection target screen, or on the right side, etc. In response to the display request carrying the target display information, continue Figure 4 As shown, the target display information can also be displayed in the remaining display area on the target screen (the target display information in the figure is the application operation interface on the target device of the application projection). In order to better realize the display of the target display information and the projection screen frame, two threads can also be set, one of which is used to process the display of the projection screen frame, and the other thread is used to expose the monitoring event of the display request, and display the corresponding target display information after the display request is monitored.

[0056] The above-mentioned display request may be sent by other devices, such as a screen projection display request for a picture or album sent by another screen projection initiating device. In this example, the target display information is an image or photo sent by other screen projection initiating devices. The above-mentioned display request may also be triggered by the user's opening operation of the relevant application on the screen projection target device. Thus, in this example, the target display information may include application interface display elements, etc.

[0057] In summary, the method for displaying the projection screen of the embodiment of the present disclosure obtains the current projection video frame, determines at least one reference pixel column in the current projection video frame, determines the current display direction of the current projection video frame based on the at least one reference pixel column, and then, if the current display direction is a portrait screen direction, identifies the edge column position of the screen content in the current projection video frame, crops the current projection video frame according to the edge column position to obtain a projection screen frame containing the screen content, and controls the projection target screen to display the projection screen frame. Thus, a simple and efficient method for determining the projection display direction is realized, and the "black border" area that does not contain the screen content is cropped in the portrait screen direction, thereby improving the visual display effect.

[0058] In different application scenarios, the above-mentioned method of determining the current display direction of the current projection video frame according to at least one reference pixel column is different, and the examples are as follows:

[0059] In one embodiment of the present disclosure, the pixel values ​​of each pixel point in the reference pixel column are traversed one by one. If the pixel value of each pixel point satisfies the preset cropping condition, it is considered that the pixel point at the location of the reference pixel column satisfies the pixel value characteristics of the "black border" area. Therefore, it is determined that there are "black border" areas on both sides of the current projection video frame, and thus, the current display direction of the current projection video frame is determined to be the vertical screen display direction. In this embodiment, the pixel value of each pixel point in the reference pixel column is traversed to determine the current display direction, thereby ensuring the accuracy of determining the current display direction.

[0060] Among them, the preset cropping conditions in the disclosed embodiment are used to determine whether the pixel value of the corresponding pixel point meets the pixel value characteristics of the "black border" area. For example, if the pixel value characteristic of the "black border" area is that the pixel value is black, then the preset cropping condition is that if the pixel value of the pixel point is in the YUV reference system, if the brightness of the pixel point in the Y channel is 16, the chromaticity value in the U channel is greater than or equal to 128 and less than or equal to 129, and the chromaticity value of the V channel is greater than or equal to 128 and less than or equal to 129, then the corresponding pixel point is considered to meet the preset cropping conditions.

[0061] In one embodiment of the present disclosure, considering that the efficiency of traversing the pixel value of each pixel point in the reference pixel column may be relatively low, in order to further improve the efficiency of determining the current display direction, the sampling column pixel points in the reference pixel column are selected based on sampling to determine the current display direction.

[0062] In this embodiment, if Figure 5 As shown, determining the current display direction of the current projection video frame according to the at least one reference pixel column includes:

[0063] Step 501: determine a first sampling quantity interval corresponding to each reference pixel column.

[0064] Among them, the first sampling number interval can be determined according to the logical resolution of the front projection video frame in the height direction, wherein the higher the logical resolution in the height direction, the larger the corresponding first sampling number interval can be, and so on.

[0065] In this embodiment, in order to further improve the efficiency of determining the current display direction, when multiple reference pixel columns are included, the first sampling number intervals corresponding to different reference pixel columns may be different, and a smaller first sampling number interval is used to make a judgment on the degree of refinement, and a larger first sampling number interval is used for auxiliary judgment, etc.

[0066] For example, when the reference pixel column includes two columns, which are located at one-fifth and four-fifths of the width of the current projection video frame respectively, the first sampling number interval of the reference pixel column corresponding to one-fifth can be 30, and the first sampling number interval of the reference pixel column corresponding to four-fifths can be 60, and so on.

[0067] Step 502 : Determine pixel values ​​of sampling column pixels in a corresponding reference pixel column according to a first sampling number interval in a height direction.

[0068] Among them, the height direction is the column direction of the current projection video frame. The height direction can be the direction from top to bottom of the current projection video frame, or it can be the direction from bottom to top of the current projection video frame, and there is no limitation here.

[0069] In this embodiment, according to the height direction, the pixel value of the sampling column pixel point in the corresponding reference pixel column is determined according to the first sampling number interval, that is, starting from the first pixel point in the height direction, a sampling column pixel point is determined every first sampling number interval to identify the pixel value of the sampling column pixel point.

[0070] Step 503 : judging, according to the pixel values ​​of the sampling column pixels, whether at least one reference pixel column includes sampling column pixels that do not meet a preset cropping condition.

[0071] Step 504: If the sampling row contains pixel points that do not meet the preset cropping condition, determine that the current display direction is the horizontal screen direction.

[0072] It should be understood that if Figure 6 As shown on the left (there is one reference pixel column in the figure), if the current display direction of the current projection video frame is vertical, the reference pixel column is located on the "black edge area", so the pixel points of the sampling column all meet the preset cropping conditions. Otherwise, Figure 6 As shown on the right, if the current display direction of the current projection video frame is horizontal, not all reference pixel columns are located on the "black border area", and therefore, there are preset cropping conditions that do not meet the preset cropping conditions.

[0073] Therefore, based on the above principle, in the embodiment of the present disclosure, according to the pixel values ​​of the sampling column pixels, it is determined whether at least one reference pixel column contains sampling column pixels that do not meet the preset cropping condition, and if the sampling column pixels that do not meet the preset cropping condition are included, it is determined that the current display direction is the horizontal screen direction. In the sampling judgment process, if there are sampling column pixels that do not meet the preset cropping condition, the sampling judgment will not be continued, thereby improving the judgment efficiency.

[0074] For example, Figure 7As shown, if there is one reference pixel column, the first sampling number interval step is 30, and the total number of pixels contained in the reference pixel column is maxPos, then the preset cropping condition is the pixel value corresponding to the black pixel, and each sampling pixel is marked with i in the order of the reference pixel column. Then, starting from the 0th pixel of the reference pixel column, it is determined whether the current i is less than maxPos. If it is less, the pixel value of the i-th sampling column pixel is extracted to determine whether the sampling column pixel is a black pixel. If it is a black pixel, the next round of judgment is performed based on the pixel value of the sampling column pixel of i=0+step=30. If the i-th pixel is not a black pixel, the judgment ends, and it is determined that there is a sampling column pixel that does not meet the preset cropping condition.

[0075] In some other possible examples, if at least one reference pixel column does not include sampling column pixels that do not meet the preset cropping condition, that is, all sampling column pixels meet the preset cropping condition, the current display direction can be directly considered to be the vertical screen direction.

[0076] In the actual implementation process, if all the pixels in the sampling column meet the preset cropping conditions, it may be misjudged if the current display direction is directly considered to be the vertical screen direction. For example, Figure 8 As shown, although the current display direction of the current projection video frame is landscape, the pixel values ​​of the pixels in at least one reference pixel column in the current projection video frame meet the preset cropping conditions. For example, although the current projection video frame is displayed in landscape, the displayed content is pure black. The pixel values ​​of the pixel points in the sampling column extracted from at least one reference pixel column are all black, which is also easily determined to be portrait.

[0077] Therefore, if the sampling column pixel points that do not meet the preset cropping conditions are not included, at least one first reference pixel row is further determined in the current projection video frame, that is, the ratio of the number of rows between the first reference pixel row and the starting row and the ending row of the current projection video frame to the total number of rows is greater than the second preset ratio threshold, and the first preset ratio threshold is greater than the first preset ratio threshold. In order to ensure that the first reference pixel row will definitely pass through the picture content, each first reference pixel row is different from the reference pixel column and should be as close to the middle row of the current projection video frame as possible. For example, in some possible embodiments, the first reference pixel row can be located at the middle row position of the current projection video frame, etc.

[0078] Further, after determining the first reference pixel row, the current display direction of the current projection video frame is determined according to at least one first reference pixel row.

[0079] It should be noted that the method of determining the current display direction of the current projection video frame according to at least one first reference pixel row is different in different application scenarios:

[0080] In some possible examples, the pixel values ​​of each sampling row pixel point in at least one first reference pixel row are traversed to determine whether the pixel values ​​of the sampling row pixel points include sampling row pixel points that do not meet the preset cropping conditions. If the sampling row pixel points do not meet the preset cropping conditions, it is considered that the reason why at least one reference pixel column does not include sampling column pixel points that do not meet the preset cropping conditions is not due to reasons such as the display content of the current projection video frame being displayed as pure black. Therefore, it is determined that the current display direction is a vertical screen direction.

[0081] On the contrary, if the sampling row pixel points do not include the sampling row pixel points that do not meet the preset cropping conditions, it is considered that the reason why at least one reference pixel column does not include the sampling column pixel points that do not meet the preset cropping conditions may be that the display content of the current projection video frame is displayed as pure black. Therefore, it is not determined whether the current display direction is the vertical screen direction, and the current display direction can be determined in combination with the previous and next frames.

[0082] In other possible examples, in order to improve the efficiency of determining the current display direction, the pixel values ​​of the sampling row pixels corresponding to the first reference pixel row can be extracted based on a sampling method, and the current display direction can be determined based on the pixel values ​​of the sampling row pixels.

[0083] In this embodiment, if Fig. 9 As shown, determining a current display direction of a current projection video frame according to at least one first reference pixel row includes:

[0084] Step 901: determine a second sampling quantity interval corresponding to each first reference pixel row.

[0085] In an embodiment of the present disclosure, a second sampling quantity interval corresponding to each first reference pixel row is determined, wherein the second sampling quantity intervals corresponding to the first reference pixel rows may be the same or different, which is not limited herein.

[0086] The second sampling number interval can be determined according to the logical resolution of the front projection video frame in the width direction, wherein the higher the logical resolution in the width direction, the larger the corresponding second sampling number interval can be, and so on.

[0087] Step 902 : Determine pixel values ​​of sampling row pixels in the corresponding first reference pixel row according to a second sampling number interval in the width direction.

[0088] Among them, the width direction is the row direction of the current projection video frame. The width direction can be the direction from left to right of the current projection video frame, or it can be the direction from left to right of the current projection video frame. There is no limitation here.

[0089] In this embodiment, the pixel values ​​of the sampling row pixels in the corresponding first reference pixel row are determined according to the second sampling number interval in the width direction, that is, starting from the first pixel in the width direction, a sampling row pixel is determined every second sampling number interval to identify the pixel values ​​of the sampling row pixels.

[0090] Step 903: judging, according to the pixel values ​​of the pixel points in the sampling row, whether at least one first reference pixel row includes pixel points in the sampling row that do not meet the preset cropping condition.

[0091] Step 904: If the sample row contains pixel points that do not meet the preset cropping condition, determine that the current display direction is the vertical screen direction.

[0092] It should be understood that if the current display direction of the current projection video frame is the portrait direction, the first reference pixel row passes through the screen content (the first reference pixel row in the figure is one), therefore, the pixel value characteristics of the screen content in the current projection video frame can be determined based on the sampling row pixels. For example, if the screen content is pure black, the acquired sampling row pixels are all black pixel values. At this time, even if the pixel points of the sampling column pixels are all black values, it does not mean that the current display direction is the portrait direction. On the contrary, if the screen content is not pure black, the acquired sampling row pixels contain non-black pixel values. At this time, the pixel points of the sampling column pixels are all black values, which may indicate that the current display direction is the portrait direction.

[0093] Therefore, based on the above principle, in the embodiment of the present disclosure, according to the pixel values ​​of the sampling row pixels, it is determined whether at least one first reference pixel row contains sampling row pixels that do not meet the preset cropping condition, and if it contains sampling row pixels that do not meet the preset cropping condition, it is determined that the current display direction is the vertical screen direction. In the sampling judgment process, if there are sampling row pixels that do not meet the preset cropping condition, the sampling judgment will not be continued, thereby improving the judgment efficiency.

[0094] In summary, the method for displaying the projection screen in the embodiment of the present invention extracts the sampling column pixel points in the reference pixel column based on the sampling method as a reference for determining the current display direction of the projection video frame. In some scenarios, in order to ensure the judgment accuracy, the sampling row pixel points are also determined, and the current display direction of the projection video frame is determined in combination with the pixel values ​​of the sampling row pixel points and the sampling column pixel points. On the basis of ensuring the determination of the current display direction of the projection video frame, the efficiency of determining the current display direction of the projection video frame is improved.

[0095] Based on the above embodiment, different methods can be used to determine the edge column position of the picture content in the current projection video frame based on different application scenarios. The examples are as follows:

[0096] In one embodiment of the present disclosure, the edge column positions of the screen contents may be extracted based on an image recognition algorithm.

[0097] In another embodiment of the present disclosure, in order to reduce computing power consumption, the edge column position of the picture content may also be determined based on simple pixel value extraction.

[0098] like Fig.10 As shown, the edge column positions of the picture content in the current projection video frame include:

[0099] Step 1001, determining at least one second reference pixel row in the current projection video frame, wherein:

[0100] The ratio of the number of rows different between the second reference pixel row and the start row and the end row of the current projection video frame to the total number of rows is greater than the third preset ratio threshold, and the third preset ratio threshold is greater than the first preset ratio threshold.

[0101] The second reference pixel row may be the same as or different from the first reference pixel row, and the third preset ratio threshold may be the same as or different from the second preset ratio threshold.

[0102] In an embodiment of the present disclosure, in order to ensure that the second reference pixel row passes through the area where the picture content is located, the third preset ratio threshold is greater than the first preset ratio threshold, wherein the third preset ratio threshold can be calibrated according to the scene.

[0103] Step 1002: traverse the pixel value of each pixel in the corresponding second reference pixel row in the width direction, and determine the edge column position according to the pixel value of each pixel.

[0104] In an embodiment of the present disclosure, in order to ensure the accuracy of edge column position determination and avoid cropping of picture information in the picture content, the pixel value of each pixel point in the corresponding second reference pixel row is traversed in the width direction to determine the edge column position according to the pixel value of each pixel point.

[0105] In this embodiment, the target column where the first pixel point that does not meet the preset cropping condition in each second reference pixel row is located can be determined, such as Fig.11 As shown, considering that some picture contents are displayed at the edge position as pixel values ​​that meet the preset cropping conditions, for example, displayed as black pixel values, therefore, the target column where the first pixel point that does not meet the preset cropping conditions in each second reference pixel row is located is determined, the minimum value in at least one target column corresponding to at least one second reference pixel row is determined, and the column where the minimum value is located is determined as the edge column position.

[0106] In the actual execution process, considering that the picture content is usually in the middle position of the current projection video frame, it is symmetrically distributed with the center line of the current projection video frame as the symmetry axis. Therefore, if the edge column position is one, for example, the left edge line of the picture content, the other side, that is, the right edge line, can be determined based on the principle of symmetry. In this embodiment, the first cropping column position and the second cropping column position of the current projection video frame can be determined according to the edge column position. If the column where the determined edge column position is located is W1, and the total number of columns is Wtotal, the difference between Wtotal and 2 times W1 can be used as the width S of the picture content, Wtotal-W1 can be used as the first cropping column position, and (Wtotal-W1)+S can be used as the second cropping column position.

[0107] If there are two edge column positions, that is, traverse the pixel value of each pixel in the corresponding second reference pixel row in the width direction, determine the first target column where the first pixel that does not meet the preset cropping condition in each second reference pixel row is located, and the second target column where the last pixel that does not meet the preset cropping condition is located, determine the minimum value in at least one first target column corresponding to at least one second reference pixel row, determine the column where the minimum value is located as an edge column position, determine the maximum value in at least one second target column corresponding to at least one second reference pixel row, determine the column where the maximum value is located as another edge column position, use one edge column position as the first cropping column position, and use the other edge column position as the second cropping column position.

[0108] In summary, the display method of the projected screen embodiment of the present invention extracts the sampling row pixel points in the reference pixel row based on the sampling method to serve as a reference for determining the edge column position of the picture content in the current projected screen video frame, thereby achieving the simplicity and feasibility of determining the edge column position.

[0109] In order to implement the above embodiments, the present disclosure also proposes a display device for projecting images.

[0110] Fig.12 The present invention provides a schematic diagram of a device for displaying a projection screen in accordance with an embodiment of the present invention. The device can be implemented by software and / or hardware and can generally be integrated in an electronic device to display a projection screen. Fig.12 As shown, the device includes: a first determination module 1210, a second determination module 1220, an identification module 1230, a cropping module 1240 and a screen projection module 1250, wherein:

[0111] A first determination module 1210 is used to obtain a current projection video frame and determine at least one reference pixel column in the current projection video frame;

[0112] A second determination module 1220, configured to determine a current display direction of a current projection video frame according to at least one reference pixel column;

[0113] The identification module 1230 is used to identify the edge column position of the picture content in the current projection video frame when the current display direction is the vertical screen direction;

[0114] A cropping module 1240 is used to crop the current projection video frame according to the edge column position to obtain a projection picture frame containing picture content;

[0115] The screen projection module 1250 is used to control the projection target screen to display the projection picture frame.

[0116] The display device of the projection screen provided in the embodiment of the present disclosure can execute the display method of the projection screen provided in any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method, which will not be repeated here.

[0117] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program / instruction, which implements the method for displaying the projection screen in the above embodiments when the computer program / instruction is executed by a processor.

[0118] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0119] The following specific reference Fig.13 , which shows a schematic diagram of the structure of an electronic device 1300 suitable for implementing the embodiment of the present disclosure. The electronic device 1300 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0120] like Fig.13 As shown, the electronic device 1300 may include a processor (e.g., a central processing unit, a graphics processor, etc.) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a memory 1308 to a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the electronic device 1300 are also stored. The processor 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0121] Typically, the following devices may be connected to the I / O interface 1305: an input device 1306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1309. The communication device 1309 may allow the electronic device 1300 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.13 The electronic device 1300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0122] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1309, or installed from the memory 1308, or installed from the ROM 1302. When the computer program is executed by the processor 1301, the above-mentioned functions defined in the display method of the projection screen of the embodiment of the present disclosure are executed.

[0123] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0124] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0126] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0127] Get the current projection video frame, determine at least one reference pixel column in the current projection video frame, determine the current display direction of the current projection video frame based on at least one reference pixel column, and then, if the current display direction is a portrait screen direction, identify the edge column position of the picture content in the current projection video frame, crop the current projection video frame according to the edge column position to obtain a projection screen frame containing the picture content, and control the projection target screen to display the projection screen frame. In this way, a simple and efficient method for determining the projection display direction is realized, and the "black border" area that does not contain the picture content is cropped in the portrait screen direction, thereby improving the visual display effect.

[0128] The electronic device may be written in one or more programming languages ​​or a combination thereof to write computer program code for performing the operations of the present disclosure, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0131] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0133] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0134] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0135] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for displaying a projection screen, characterized in that: The following steps are involved: Obtain a current projection video frame, determine a plurality of reference pixel columns in the current projection video frame, and a ratio of a difference in the number of columns between each of the reference pixel columns and a start column or an end column of the current projection video frame to a total number of columns is less than a first preset ratio threshold; Determine a plurality of first sampling number intervals corresponding to the plurality of reference pixel columns, wherein the plurality of first sampling number intervals are different from each other; Determine, in the height direction, pixel values ​​of sampling column pixels in the corresponding reference pixel column according to each of the first sampling number intervals; According to the pixel values ​​of the sampling column pixels, determining whether each of the reference pixel columns includes sampling column pixels that do not meet the preset cropping condition; If the sampling column pixel points that do not meet the preset cropping condition are not included, at least one first reference pixel row is determined in the current projection video frame, wherein the ratio of the number of rows between each of the first reference pixel rows and the start row and the end row of the current projection video frame to the total number of rows is greater than a second preset ratio threshold, and the second preset ratio threshold is greater than the first preset ratio threshold, Determining a current display direction of the current projection video frame according to the at least one first reference pixel row; If the current display direction is a vertical screen direction, identifying the edge column position of the picture content in the current projection video frame; The current projection video frame is cropped according to the edge column position to obtain a projection picture frame containing the picture content, and the projection target screen is controlled to display the projection picture frame.

2. The method according to claim 1, characterized in that Also includes: If the sampling row of pixels does not satisfy the preset cropping condition, it is determined that the current display direction is a horizontal screen direction.

3. The method according to claim 1, characterized in that The determining, according to the at least one first reference pixel row, a current display direction of the current projection video frame includes: Determine a second sampling number interval corresponding to each of the first reference pixel rows; Determine, in the width direction, pixel values ​​of the sampling row pixels in the corresponding first reference pixel row according to the second sampling number intervals; Determining, according to pixel values ​​of the sampling row pixels, whether the at least one first reference pixel row includes sampling row pixels that do not meet the preset cropping condition; If the sampling row includes pixel points that do not meet the preset cropping condition, it is determined that the current display direction is a vertical screen direction.

4. The method according to claim 1, characterized in that The identifying the edge column position of the picture content in the current projection video frame includes: At least one second reference pixel row is determined in the current projection video frame, wherein: The ratio of the number of rows different between the second reference pixel row and the start row and the end row of the current projection video frame to the total number of rows is greater than a third preset ratio threshold, and the third preset ratio threshold is greater than the first preset ratio threshold; The pixel value of each pixel point in the corresponding second reference pixel row is traversed in the width direction, and the edge column position is determined according to the pixel value of each pixel point.

5. The method according to claim 4, characterized in that The step of determining the edge column position according to the pixel value of each pixel point comprises: Determine a target column where a first pixel point that does not meet the preset cropping condition in each of the second reference pixel rows is located; A minimum value in at least one target column corresponding to the at least one second reference pixel row is determined, and a column where the minimum value is located is determined as the edge column position.

6. The method according to claim 1, characterized in that The step of cropping the current projection video frame according to the edge column position to obtain a projection picture frame containing the picture content includes: Determine a first cropping column position and a second cropping column position of the current projection video frame according to the edge column position; The current projection video frame is cut according to the first cropping column position and the second cropping column position to obtain a projection screen frame containing the screen content.

7. The method according to any one of claims 1 to 6, characterized in that: Before identifying the edge column position of the picture content in the current projection video frame, the method further includes: Obtain whether the current display direction is consistent with the historical display direction of the previous projection video frame; If they are inconsistent, then obtaining multiple consecutive projection video frames adjacent to the current projection video frame; Identify multiple reference display directions corresponding to the continuous multiple projection video frames, and determine that the multiple reference display directions are consistent with the current display direction.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: Determine a projection display area on the projection target screen corresponding to the projection picture frame, and display the projection picture frame in the projection display area; In response to a display request carrying target display information, the target display information is displayed in the remaining display area on the projection target screen.

9. A display device for projecting a screen, characterized in that: include: A first determination module is used to obtain a current projection video frame and determine a plurality of reference pixel columns in the current projection video frame; A second determination module is used to determine a plurality of first sampling number intervals corresponding to the plurality of reference pixel columns, wherein the plurality of first sampling number intervals are different from each other, and according to the height direction, the pixel value of the sampling column pixel point in the corresponding reference pixel column is determined according to each of the first sampling number intervals, and according to the pixel value of the sampling column pixel point, it is judged whether each of the reference pixel columns contains sampling column pixel points that do not meet the preset cropping condition, and if it does not contain sampling column pixel points that do not meet the preset cropping condition, at least one first reference pixel row is determined in the current projection video frame, wherein the ratio of the number of rows that differ between each of the first reference pixel rows and the starting row and the ending row of the current projection video frame to the total number of rows is greater than a second preset ratio threshold, and the second preset ratio threshold is greater than the first preset ratio threshold, and the current display direction of the current projection video frame is determined according to the at least one first reference pixel row; An identification module, used for identifying the edge column position of the picture content in the current projection video frame when the current display direction is the vertical screen direction; A cropping module, used for cropping the current projection video frame according to the edge column position to obtain a projection picture frame containing the picture content; The screen projection module is used to control the projection target screen to display the projection picture frame.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the method for displaying the projection image described in any one of claims 1-8 above.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to execute the method for displaying the projection image as described in any one of claims 1-8 above.

Images