Screen state detection method and system for screen projection sending end, and storage medium

Abstract

This application relates to the field of information and communication technology, and discloses a method, system, and storage medium for detecting the screen state of a screen projection sending end. The method includes: acquiring a first screen image, wherein the first screen image includes the screen image of the screen projection sending end; performing binarization processing on the first screen image to determine an effective display area, wherein the effective display area is the position of the screen image of the screen projection sending end in the first screen image; acquiring the pixel coordinates of the effective display area; calculating the width and length of the effective display area based on the pixel coordinates of the effective display area; and determining the screen state of the screen projection sending end based on the width and length. This application can realize the automatic identification of the screen orientation of the screen projection sending end at the screen projection display end, so as to automatically adjust the display screen of the screen projection display end and further improve the control efficiency of the screen projection display end.

Description

Technical Field

[0001] This application relates to the field of information and communication technology, and in particular to a method, system and storage medium for detecting the screen status of a screen projection sending end. Background Technology

[0002] Currently, when casting a mobile phone screen, it is necessary to obtain the Extended Display Identification Data (EDID) of the casting display to determine the resolution of the casting display. Based on the resolution of the casting display, the mobile phone adjusts the parameters of the image to be sent to the casting display according to its own screen rotation to adapt to different resolutions. However, in actual casting, only the mobile phone can recognize its own screen rotation, while the casting display cannot obtain the mobile phone's screen rotation. When controlling the mobile phone through the casting display, it is necessary to frequently set the latest rotation state of the mobile phone on the casting display, resulting in low control efficiency. Summary of the Invention

[0003] To address the aforementioned technical issues, this application provides a method, system, and storage medium for detecting the screen state of a projection sending end. This method can automatically identify the latest flip state of the projection sending end, thereby quickly adjusting the display screen of the projection display end to improve the control efficiency of the projection display end.

[0004] To address the aforementioned technical problems, the embodiments of this application provide the following technical solutions:

[0005] In a first aspect, embodiments of this application provide a method for detecting the screen state of a screen projection sending end. The method is applied to a screen projection display end, which communicates with the screen projection sending end. The screen projection display end displays the screen image of the screen projection sending end. The method includes:

[0006] Acquire a first screen image, wherein the first screen image includes the screen image of the screen projection sending end;

[0007] The first screen image is binarized to determine the effective display area, where the effective display area is the position of the screen image of the projection sending end in the first screen image;

[0008] Obtain the pixel coordinates of the valid display area;

[0009] Calculate the width and length of the effective display area based on the pixel coordinates of the effective display area;

[0010] The screen state of the projection sending end is determined based on the width and length, including landscape or portrait mode.

[0011] In some embodiments, binarizing the first screen image to determine the effective display area includes:

[0012] Obtain the color value of each pixel in the first screen image;

[0013] The color value of each pixel is weighted to obtain the grayscale value of each pixel;

[0014] The effective display area is determined based on the grayscale value.

[0015] In some embodiments, determining the effective display area based on grayscale values ​​includes:

[0016] Obtain the coordinates of pixels whose grayscale value is greater than the grayscale threshold;

[0017] Based on the coordinates of pixels with gray values ​​greater than a gray threshold, a first coordinate and a second coordinate are determined. The first coordinate includes a first abscissa and a first ordinate, and the second coordinate includes a second abscissa and a second ordinate. The first abscissa is the minimum abscissa among all pixel coordinates with gray values ​​greater than the gray threshold, the first ordinate is the maximum ordinate among all pixel coordinates with gray values ​​greater than the gray threshold, the second abscissa is the maximum abscissa among all pixel coordinates with gray values ​​greater than the gray threshold, and the second ordinate is the minimum ordinate among all pixel coordinates with gray values ​​greater than the gray threshold.

[0018] The effective display area is determined based on the first and second coordinates.

[0019] In some embodiments, before binarizing the first screen image, the method further includes:

[0020] The first screen image is filtered, specifically including:

[0021] Obtain the noise tolerance value;

[0022] The first screen image is filtered based on the noise tolerance value.

[0023] In some embodiments, obtaining a noise tolerance value includes:

[0024] Initialize a noise value;

[0025] Within a preset time period, the first screen image is binarized multiple times based on the noise value to obtain multiple second screen images. The second screen images are screen images after binarization, and the screen images after binarization include the effective display area.

[0026] If the width and length of the effective display area in each second screen image are both greater than the preset threshold, and the variation values ​​of the width and length of the effective display area in multiple second screen images are all within the range, then the noise value is determined to be the noise tolerance value.

[0027] In some embodiments, the width and length of the effective display area are calculated based on the pixel coordinates of the effective display area, including:

[0028] Calculate the first difference between the first horizontal coordinate and the second horizontal coordinate, and use the absolute value of the first difference as the width of the effective display area;

[0029] Calculate the second difference between the first and second ordinates, and use the absolute value of the second difference as the length of the effective display area.

[0030] In some embodiments, determining the screen orientation (portrait or landscape) of the projection sending end based on its width and length includes:

[0031] If the width of the effective display area is greater than its length, then the screen projection sending end is determined to be in landscape mode;

[0032] If the width of the effective display area is less than its length, then the screen projection sending end is in portrait mode.

[0033] In some embodiments, the method further includes:

[0034] Determine the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end;

[0035] Based on the mapping relationship, the touch coordinates corresponding to the coordinates of the pixels in the effective display area are determined, where the touch coordinates are the coordinates of the touch point on the screen of the projection sending end.

[0036] In some embodiments, determining the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end includes:

[0037] Calculate the first scaling ratio between the width of the first screen image and the width of the screen image at the projection sending end;

[0038] Calculate a second scaling factor between the length of the first screen image and the length of the screen image at the projection sending end;

[0039] Based on the first scaling ratio and the second scaling ratio, a mapping relationship is determined between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end. Specifically, the horizontal coordinate of the pixel in the screen image of the projection sending end = the first scaling ratio * the horizontal coordinate of the pixel in the effective display area, and the vertical coordinate of the pixel in the screen image of the projection sending end = the second scaling ratio * the vertical coordinate of the pixel in the effective display area.

[0040] Secondly, embodiments of this application provide a screen state detection system for a screen projection sending end, the system comprising:

[0041] The screen projection display end is used to perform the screen projection sending end screen state detection method of the first aspect;

[0042] The screen mirroring sender is used to send the screen image from the screen mirroring sender to the screen mirroring display.

[0043] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed, implements the screen state detection method of the screen projection sending end as described in the first aspect.

[0044] The beneficial effects of the embodiments of this application are as follows: Unlike the prior art, the embodiments of this application provide a method for detecting the screen state of a screen projection sending end. The method includes: acquiring a first screen image, wherein the first screen image includes the screen image of the screen projection sending end; performing binarization processing on the first screen image to determine an effective display area, wherein the effective display area is the position of a second screen image in the first screen image; acquiring the pixel coordinates of the effective display area; calculating the width and length of the effective display area based on the pixel coordinates of the effective display area; and determining the screen state of the screen projection sending end based on the width and length.

[0045] By binarizing the first screen image, the effective display area is determined, and then the length and width of the effective display area are calculated. Based on the length and width of the effective display area, the screen state of the projection sending end is determined. This allows for automatic identification of the screen state of the projection sending end, enabling rapid adjustment of the display screen on the projection display end and improving the control efficiency of the projection display end. Attached Figure Description

[0046] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0047] Figure 1AThis is an example schematic diagram illustrating the location of a touch point on a projection display in portrait mode on a mobile phone, as provided in an embodiment of this application.

[0048] Figure 1B This is an example schematic diagram of a touch point on a mobile phone screen provided in an embodiment of this application;

[0049] Figure 1C This is an example schematic diagram illustrating the location of a touch point on a screen projection display in landscape mode, as provided in an embodiment of this application.

[0050] Figure 1D This is an example schematic diagram of another touch point on a mobile phone screen provided in an embodiment of this application;

[0051] Figure 2 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0052] Figure 3 This is a flowchart illustrating a method for detecting the screen status of a screen projection sending end, as provided in an embodiment of this application.

[0053] Figure 4 This is an example schematic diagram of a first screen image provided in an embodiment of this application;

[0054] Figure 5 yes Figure 3 A detailed flowchart of step S302 in the process;

[0055] Figure 6 yes Figure 5 A detailed flowchart of step S3023 in the process;

[0056] Figure 7 This is a schematic diagram of a process for filtering a first screen image according to an embodiment of this application;

[0057] Figure 8 yes Figure 7 A detailed flowchart of step S701 in the process;

[0058] Figure 9 yes Figure 3 A detailed flowchart of step S304 in the process;

[0059] Figure 10 yes Figure 3 A detailed flowchart of step S305 in the process;

[0060] Figure 11 This is a schematic diagram of a process for determining touch coordinates provided in an embodiment of this application;

[0061] Figure 12 yes Figure 11A detailed flowchart of step S1101 in the process;

[0062] Figure 13 This is a schematic diagram of the overall process of a screen state detection method for a screen projection sending end provided in an embodiment of this application;

[0063] Figure 14 This is a schematic diagram of the structure of a screen state detection system for a screen projection sending end provided in an embodiment of this application;

[0064] Figure 15 This is a schematic diagram of the structure of a screen state detection device for a screen projection sending end provided in an embodiment of this application;

[0065] Figure 16 This is a schematic diagram of the structure of a screen projection display terminal provided in an embodiment of this application.

[0066] Explanation of icon numbers:

[0067] Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. In addition, the terms "first" and "second" used in this application do not limit the data, but only distinguish the same or similar items with basically the same function and effect.

[0070] Before introducing the embodiments of this application, a brief introduction will be given to the screen state detection methods known to the inventors of this application, so as to facilitate the understanding of the embodiments of this application later.

[0071] Currently, when projecting a mobile phone screen onto a projection display, if the phone's screen state changes, the phone itself can only adjust the screen image before sending it to the projection display for display. When operating the phone's screen on the projection display, the screen parameters of the projection display need to be manually adjusted to adapt to the changes in the phone's screen state. This requires manually adjusting the relative relationship between the phone's touch coordinates and the projection display's touch coordinates to achieve reverse control of the phone. This results in a certain lag in the reverse control function, leading to low control efficiency.

[0072] For example, see Figure 1A , Figure 1A This is an example schematic diagram illustrating the location of a touch point on a projection display in portrait mode on a mobile phone, as provided in an embodiment of this application. Figure 1A As shown, area L is the screen image of the projection display, and area M is the screen image of the mobile phone. When any point in area M is clicked on the projection display, such as point F, the projection display will send the coordinates of point F to the mobile phone to prompt the mobile phone to perform the corresponding operation at the point corresponding to point F.

[0073] Please refer to the following: Figure 1B , Figure 1B This is an example schematic diagram of a touch point on a mobile phone screen provided in an embodiment of this application. Area N represents the screen image of the mobile phone. When point F is clicked on the projection display, the point on the mobile phone corresponding to point F is... Figure 1B Point E in the diagram.

[0074] When the phone screen is switched to landscape mode, for example, see Figure 1C , Figure 1C This is an example schematic diagram illustrating the location of a touch point on a screen projection display in landscape mode, as provided in an embodiment of this application. Figure 1C As shown, when the phone screen is in landscape mode, the phone will automatically adjust the screen image to send to the projection display. When you click on any point in area H, such as point G, the projection display will send the coordinates of point G to the phone to prompt the phone to perform the corresponding operation at the point corresponding to point G.

[0075] When the phone screen state changes, for example from Figure 1A The portrait mode in the middle is converted to Figure 1C In landscape mode, the coordinates of points F and G appear the same on the screen of the projection display. However, because the projection display cannot detect the current screen state of the phone, when the coordinates of point G in landscape mode are sent to the phone, point G will still appear different from the displayed coordinates. Figure 1B Point E in the image corresponds to the same point on the phone screen in both landscape and portrait modes. In fact, in landscape mode... Figure 1C G point in Figure 1D Point K in the middle.

[0076] In summary, existing screen projection displays cannot automatically detect the screen status of the mobile phone, resulting in a difference in the relative relationship between the touch coordinates of the screen projection display and the touch coordinates of the mobile phone. This leads to errors when controlling the mobile phone screen on the screen projection display, resulting in low control efficiency.

[0077] To address the aforementioned issues, this application provides a method for detecting the screen status of a projection sending end. By binarizing a first screen image to determine the effective display area, and then calculating the length and width of the effective display area, the screen status of the projection sending end is determined based on the length and width of the effective display area. This method can automatically identify the screen status of the projection sending end, thereby quickly adjusting the display screen of the projection display end and improving the control efficiency of the projection display end.

[0078] The technical solution of this application is described in detail below with reference to the accompanying drawings:

[0079] Please see Figure 2 , Figure 2 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0080] like Figure 2 As shown, the application environment 200 includes: a screen projection display terminal 10 and a screen projection sending terminal 20. The screen projection display terminal 10 and the screen projection sending terminal 20 are connected via a wireless network or a wired network. The wireless network includes 2G, 3G, 4G, 5G, wireless LAN, Bluetooth and other wireless networks, and the wired network includes serial cable, network cable and other wired networks.

[0081] The screen projection display terminal 10 is used to display the screen image of the screen projection sending terminal 20 and to control the screen of the screen projection sending terminal 20 from the screen of the screen projection display terminal 10. The screen projection display terminal 10 may include display devices such as smart TVs, projectors, and computer monitors.

[0082] The screen projection sending end 20 is used to send the screen image of the screen projection sending end 20 to the screen projection display end 10. The screen projection sending end 20 may include smartphones, tablets, laptops, desktop computers, etc.

[0083] Please see Figure 3 , Figure 3 This is a flowchart illustrating a method for detecting the screen status of a screen projection sending end, as provided in an embodiment of this application.

[0084] The screen state detection method of the screen projection sending end is applied to the screen projection display end. Specifically, the execution subject of the screen state detection method of the screen projection sending end is one or at least two processors of the screen projection display end.

[0085] like Figure 3 As shown, the method for detecting the screen status of the screen projection sending end includes:

[0086] Step S301: Obtain the first screen image, wherein the first screen image includes the screen image of the screen projection sending end;

[0087] Specifically, the screen image sent by the screen projection sending end is received, and the screen image of the screen projection sending end is displayed on the screen of the screen projection display end. The current screen image of the screen projection display end is obtained to obtain the first screen image, wherein the first screen image is the screen image of the screen projection display end and includes the screen image of the screen projection sending end.

[0088] Please refer to the following: Figure 4 , Figure 4 This is an example schematic diagram of a first screen image provided in an embodiment of this application;

[0089] like Figure 4 As shown, area A is the area where the first screen image is located, that is, area A is the area where the screen image of the projection display end is located. Area A includes area B, which is the area where the screen image of the projection sending end is located.

[0090] Step S302: Binarize the first screen image to determine the effective display area, wherein the effective display area is the position of the screen image of the projection sending end in the first screen image;

[0091] Specifically, the first screen image is binarized to convert it into a binary image. Based on this binary image, the effective display area is determined, where the effective display area is the position of the screen image from the projection sending end within the first screen image. For detailed steps of the image binarization process, please refer to [link to relevant documentation]. Figure 5 .

[0092] In this embodiment of the application, the first screen image is binarized so that the first screen image contains only two colors, namely black and white, so as to identify and analyze the target image, which is the screen image of the projection sending end.

[0093] Please refer to the following: Figure 5 , Figure 5 yes Figure 3 A detailed flowchart of step S302 in the process;

[0094] like Figure 5 As shown, step S302 includes:

[0095] Step S3021: Obtain the color value of each pixel in the first screen image;

[0096] Specifically, the color value of each pixel in the first screen image is obtained. The color value is an RGB value, which is composed of the values ​​corresponding to red (R), green (G), and blue (B).

[0097] It is understandable that an image is composed of multiple pixels, each containing specific color and brightness information. Pixels in an image are usually represented as one or more numerical values. For color images, each pixel typically contains values ​​for three color channels: red (R), green (G), and blue (B). The values ​​of R, G, and B all range from [0, 255]. The number of pixels is determined by the image resolution. The higher the resolution, the more pixels there are in the image, and the clearer the image. Conversely, the lower the resolution, the fewer pixels there are, and the blurrier the image appears.

[0098] Step S3022: Perform weighted processing on the color value of each pixel to obtain the grayscale value of each pixel;

[0099] Specifically, the color value of each pixel is weighted to obtain the grayscale value of each pixel, including:

[0100] Gray value = aR + βG + γB

[0101] Where R is the red value, G is the green value, B is the blue value, α is the weighting coefficient of R, β is the weighting coefficient of G, and γ is the weighting coefficient of B.

[0102] It is understandable that red, green, and blue are distinguished based on the human eye's sensitivity to different colors. From a biological perspective, red, green, and blue correspond to different wavelengths of light. The human eye is most sensitive to green light, followed by red light, and has relatively low sensitivity to blue light. Therefore, the weight coefficients of R, G, and B are ordered as β > α > γ. In actual calculations, the specific values ​​of β, α, and γ can be determined based on the open-source cross-platform computer vision library (OpenCV), for example, β = 0.587, α = 0.299, and γ = 0.114.

[0103] OpenCV includes an image processing module, which contains a series of functions and classes to implement specific functions, such as image filtering, geometric transformation, and color space conversion. By using the image processing module in OpenCV to binarize screen images, the color value of each pixel in the screen image can be quickly converted into a grayscale value.

[0104] Step S3023: Determine the effective display area based on the grayscale values;

[0105] Specifically, each pixel corresponds to a coordinate. The grayscale value of each pixel is determined, that is, the color of each pixel is determined to be black or white. Then the coordinates of the white pixel are obtained to determine the effective display area. The effective display area is the position of the screen image of the projection sending end in the first screen image.

[0106] Please refer to the following: Figure 6 , Figure 6 yes Figure 5 A detailed flowchart of step S3023 in the process;

[0107] like Figure 6 As shown, step S3023 includes:

[0108] Step S3231: Obtain the coordinates of pixels whose grayscale values ​​are greater than the grayscale threshold;

[0109] In this embodiment, the grayscale value ranges from [0, 255], where 0 represents the darkest (black) and 255 represents the brightest (white). To clearly distinguish between black and white, a grayscale threshold needs to be set to separate black pixels from white pixels.

[0110] Specifically, the grayscale value of each pixel is obtained, and it is determined whether the grayscale value of each pixel is greater than a grayscale threshold. If the grayscale value of a pixel is greater than the grayscale threshold, the color corresponding to that pixel is determined to be white; if the grayscale value of a pixel is less than or equal to the grayscale threshold, the color corresponding to that pixel is determined to be black. For example, if the grayscale threshold is set to 1, then the color corresponding to a pixel with a grayscale value greater than 1 is white, and the color corresponding to a pixel with a grayscale value less than or equal to 1 is black.

[0111] In this embodiment of the application, before obtaining the coordinates of pixels with gray values ​​greater than the gray value threshold, a coordinate system is constructed based on the first screen image after binarization. The coordinates of each pixel can be obtained through this coordinate system.

[0112] Specifically, the coordinates of each pixel whose grayscale value is greater than the grayscale threshold are read to obtain the coordinates of all pixels whose grayscale value is greater than the grayscale threshold.

[0113] Please refer to the following: Figure 4 ,like Figure 4 As shown, A is the first screen image, and B is the screen image of the projection sending end. Taking the O point at the top left corner of the first screen image as the origin, and using pixels as the unit, a horizontal axis x and a vertical axis y are constructed to obtain an xOy coordinate system. Based on this coordinate system, the coordinates of each pixel point can be directly read.

[0114] Step S3232: Determine the first coordinate and the second coordinate based on the coordinates of pixels with gray values ​​greater than the gray value threshold;

[0115] Specifically, among all pixel coordinates with gray values ​​greater than the gray threshold, the pixel with the smallest x-coordinate and the largest y-coordinate is selected. Then, the pixel with the largest x-coordinate and the smallest y-coordinate is selected. The coordinates of the pixel with the smallest x-coordinate and the largest y-coordinate are used as the first coordinates, and the coordinates of the pixel with the largest x-coordinate and the smallest y-coordinate are used as the second coordinates. The first coordinate includes a first x-coordinate and a first y-coordinate, and the second coordinate includes a second x-coordinate and a second y-coordinate. The first x-coordinate is less than the second x-coordinate, and the first y-coordinate is greater than the second y-coordinate.

[0116] Wherein, the first horizontal axis is the minimum horizontal axis among all the horizontal axes of the pixels whose gray values ​​are greater than the gray threshold, the first vertical axis is the maximum vertical axis among all the vertical axes of the pixels whose gray values ​​are greater than the gray threshold, the second horizontal axis is the maximum horizontal axis among all the horizontal axes of the pixels whose gray values ​​are greater than the gray threshold, and the second vertical axis is the minimum vertical axis among all the vertical axes of the pixels whose gray values ​​are greater than the gray threshold.

[0117] Step S3233: Determine the effective display area based on the first and second coordinates;

[0118] Specifically, a rectangle is constructed based on the first and second coordinates. The area containing this rectangle is the effective display area. The pixel coordinates of the upper left corner of the effective display area are the first coordinates, and the pixel coordinates of the lower right corner of the effective display area are the second coordinates.

[0119] In this embodiment of the application, the effective display area is composed of multiple pixels with gray values ​​greater than the gray value threshold.

[0120] Please refer to it again. Figure 4 , Figure 4 Point 'a' is the first coordinate point, point 'b' is the second coordinate point, and area B is the effective display area.

[0121] In this embodiment, by binarizing the first screen image, the first screen image is made to appear as only black and white, thereby reducing the complexity of image data processing, making it easier to determine the position of the screen image of the projection sending end in the first screen image, and improving the speed of image processing.

[0122] In this embodiment of the application, before binarizing the first screen image, it is also necessary to filter the first screen image to remove image noise. Image noise includes various types of noise, such as Gaussian noise, salt and pepper noise, etc. Image noise can cause the image to become blurry. Therefore, it is necessary to filter the first screen image.

[0123] Please see Figure 7 , Figure 7 This is a schematic diagram of a process for filtering a first screen image according to an embodiment of this application;

[0124] like Figure 7 As shown, filtering processing is performed on the first screen image, including:

[0125] Step S701: Obtain the noise tolerance value;

[0126] Noise margin refers to the maximum allowable noise level in digital communication or digital circuits to ensure correct signal transmission or normal circuit operation. A suitable noise margin serves as a benchmark for filtering processes.

[0127] For details on obtaining noise tolerance values, please refer to [link / reference]. Figure 8 The steps.

[0128] Please see Figure 8 , Figure 8 yes Figure 7 A detailed flowchart of step S701 in the process;

[0129] like Figure 8 As shown, step S701 includes:

[0130] Step S711: Initialize a noise value;

[0131] Specifically, set a noise value to initialize a noise value.

[0132] It should be noted that the initial noise value was set based on experimental experience.

[0133] Step S712: Within a preset time period, the first screen image is binarized multiple times based on the noise value to obtain multiple second screen images;

[0134] Specifically, within a preset time period, the first screen image is binarized multiple times based on the currently set noise value to obtain multiple second screen images. The second screen images are the binarized screen images, and the binarized screen images include the effective display area.

[0135] For example, with a preset time period of 200 milliseconds (ms), the first screen image is subjected to multiple binarization processes.

[0136] Step S713: Determine whether the width and length of the effective display area in the second screen image are both greater than a preset threshold;

[0137] In this embodiment, a coordinate system is constructed for each second screen image to determine the effective display area in each second screen image. For details, please refer to [link / reference needed]. Figure 5 , Figure 6 This will not be explained in detail here.

[0138] In this embodiment of the application, the calculation process of the width and length of the effective display area in the second screen image includes: obtaining the coordinates of the upper left corner and the lower right corner of the effective display area in each second screen image; calculating the difference between the horizontal coordinates of the upper left corner and the lower right corner; taking the absolute value of the difference corresponding to the horizontal coordinate as the width of the effective display area; and calculating the difference between the vertical coordinates of the upper left corner and the lower right corner; taking the absolute value of the difference corresponding to the vertical coordinate as the length of the effective display area.

[0139] Specifically, it is determined whether the width and length of the effective display area in each second screen image are greater than a preset threshold. If the width and length of the effective display area in each second screen image are greater than the preset threshold, the process jumps to step S714. If the width and length of the effective display area in each second screen image are less than or equal to the preset threshold, the process jumps to step S711.

[0140] For example, a preset threshold of 256 is set to determine whether the width and length of the effective display area in each second screen image are greater than 256.

[0141] In this embodiment of the application, if the width and length of the effective display area in the second screen image do not exceed the preset threshold, it means that the screen of the projection display end does not display the screen image of the projection sending end. The above reason may be that the noise value is set too low. By readjusting the noise value and re-binarizing the first screen image, the effective display area can be re-determined.

[0142] Step S714: Determine whether the change range values ​​of the width and length of the effective display area in the second screen image are both within the range;

[0143] Specifically, when the width and length of the effective display area in each second screen image are greater than a preset threshold, it is determined whether the change range of the width and length of the effective display area in the second screen image is within the range. If the change range of the width and length of the effective display area in all second screen images is within the range, the process jumps to step S715. If the change range of the width and length of the effective display area in all second screen images is not within the range, the process jumps to step S711.

[0144] In this embodiment, the amplitude range is [-5, +5]. The maximum and minimum widths of the effective display areas in all second screen images are obtained, and the difference between the maximum and minimum widths is calculated. Similarly, the maximum and minimum lengths of the effective display areas in all second screen images are obtained, and the difference between the maximum and minimum lengths is calculated. If either the difference between the maximum and minimum widths or the difference between the maximum and minimum lengths is outside the amplitude range of [-5, +5], the current noise value is reset, and the process continues sequentially. Figure 8 The steps in the process. The variation range values ​​include the difference between the maximum and minimum widths, and the difference between the maximum and minimum lengths.

[0145] Step S715: Determine the noise value as the noise tolerance value;

[0146] Specifically, when the width and length of the effective display area in each second screen image are greater than the preset threshold, and the variation values ​​of the width and length of the effective display area in all second screen images are within the range, the currently set noise value is determined to be the noise tolerance value.

[0147] It should be noted that the noise value gradually increases during the adjustment process.

[0148] In this embodiment of the application, the first screen image is binarized multiple times, and it is determined whether the length and width of the effective display area in the multiple second screen images are stable. If they are unstable, the noise value is dynamically adjusted until the length and width of the effective display area in the multiple second screen images remain stable, so as to accurately identify the effective display area.

[0149] Step S702: Filter the first screen image based on the noise tolerance value;

[0150] Specifically, after determining the noise tolerance value, the first screen image is filtered based on the noise tolerance value.

[0151] In this embodiment of the application, filtering the first screen image includes: selecting a filter and performing operations such as smoothing, sharpening, or edge enhancement on the image through the filter. Here, a filter is a mathematical function, and the types of filters include linear filters and nonlinear filters. Linear filters include box filtering, mean filtering, Gaussian filtering, Laplacian filtering, etc., while nonlinear filters include median filters and bilateral filters.

[0152] In this embodiment of the application, filtering the first screen image can improve image quality and facilitate the rapid determination of the effective display area in the first screen image.

[0153] Step S303: Obtain the pixel coordinates of the effective display area;

[0154] Specifically, based on the coordinate system constructed in the first screen image, the first coordinate and the second coordinate of the effective display area are read to obtain the pixel coordinates of the effective display area, which include the first coordinate and the second coordinate.

[0155] For example, Figure 4 The coordinates of points a and b in the diagram are given, with point a being the first coordinate and point b being the second coordinate.

[0156] Step S304: Calculate the width and length of the effective display area based on the pixel coordinates of the effective display area;

[0157] Specifically, the width and length of the effective display area are calculated based on the pixel coordinates of the effective display area, i.e., based on the first and second coordinates. Please refer to [link / reference] for the detailed calculation method. Figure 9 .

[0158] Please see Figure 9 , Figure 9 yes Figure 3 A detailed flowchart of step S304 in the process;

[0159] like Figure 9 As shown, step S304 includes:

[0160] Step S341: Calculate the first difference between the first horizontal coordinate and the second horizontal coordinate, and use the absolute value of the first difference as the width of the effective display area;

[0161] In the embodiments of this application, the first coordinate includes a first horizontal coordinate and a first vertical coordinate, and the second coordinate includes a second horizontal coordinate and a second vertical coordinate.

[0162] Specifically, the first difference between the first horizontal coordinate and the second horizontal coordinate is calculated, and the absolute value of the first difference is used as the width of the effective display area.

[0163] Step S342: Calculate the second difference between the first ordinate and the second ordinate, and use the absolute value of the second difference as the length of the effective display area;

[0164] Specifically, the second difference between the first and second ordinates is calculated, and the absolute value of the second difference is used as the length of the effective display area.

[0165] Step S305: Determine the screen status of the projection sending end based on the width and length;

[0166] Specifically, to determine whether the width of the effective display area is greater than its length, the screen state of the projection sending end can be determined based on the difference between the width and length of the effective display area. The screen state of the projection sending end includes either landscape or portrait mode. For detailed determination steps, please refer to [link to relevant documentation]. Figure 10 .

[0167] Please see Figure 10 , Figure 10 yes Figure 3 A detailed flowchart of step S305 in the process;

[0168] like Figure 10 As shown, step S305 includes:

[0169] Step S351: Determine whether the width of the effective display area is greater than its length;

[0170] Specifically, determine whether the width of the effective display area is greater than the length of the effective display area. If the width of the effective display area is greater than the length of the effective display area, proceed to step S352. If the width of the effective display area is less than the length of the effective display area, proceed to step S353.

[0171] Step S352: Determine that the screen projection sending end is in landscape mode;

[0172] Specifically, when the width of the effective display area is greater than the length of the effective display area, the screen projection sending end is determined to be in landscape mode.

[0173] Step S353: Determine that the screen projection sending end is in portrait mode;

[0174] Specifically, if the width of the effective display area is less than the length of the effective display area, then the screen projection sending end is determined to be in portrait mode.

[0175] In this embodiment of the application, the screen status of the current projection sending end can be automatically detected by measuring the size between the width and length of the effective display area.

[0176] In this embodiment, the screen state of the projection sending end is automatically detected on the projection display end, so as to realize the rapid adjustment of the screen display on the projection display end to adapt to the changes in the screen state of the projection sending end, so as to further realize the control of the screen of the projection sending end on the projection display end, that is, to realize the control of the screen of the projection sending end by clicking on the screen of the projection display end.

[0177] In this embodiment of the application, in order to control the screen of the screen projection sending end, the application obtains touch coordinates and sends the touch coordinates to the screen projection sending end to instruct the screen projection sending end to perform corresponding operations according to the touch coordinates, such as opening a certain software on the screen projection sending end.

[0178] Please see Figure 11 , Figure 11 This is a schematic diagram of a process for determining touch coordinates provided in an embodiment of this application;

[0179] like Figure 11 As shown, the process for determining touch coordinates includes:

[0180] Step S1101: Determine the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end;

[0181] The screen state of the screen sending end includes landscape or portrait mode.

[0182] Specifically, when the screen of the projection sending end is in landscape mode, the mapping relationship between the coordinates of the pixels in the effective display area and the coordinates of the pixels in the screen image of the projection sending end is determined based on the landscape mode of the projection sending end.

[0183] Specifically, when the screen of the projection sending end is in portrait mode, the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end is determined based on the portrait mode of the projection sending end. For the specific steps of determining the mapping relationship, please refer to [link to documentation / reference]. Figure 12 .

[0184] Please refer to the following: Figure 12 , Figure 12 yes Figure 11 A detailed flowchart of step S1101 in the process;

[0185] like Figure 12 As shown, step S1101 includes:

[0186] Step S1111: Calculate the first scaling ratio between the width of the first screen image and the width of the screen image of the projection sending end;

[0187] The first screen image refers to the screen image of the projection display device. Specifically, the width of the first screen image is the same as the width of the projection display device's screen image. The width of the projection display device is determined by its screen size, which is usually fixed. For example, if the projection display device's screen size is 1920*1080 pixels, then the width of the first screen image is 1920 pixels. It should be noted that the screen size of projection devices from different manufacturers may vary.

[0188] Specifically, the first scaling ratio = the width of the screen image of the projection sending end / the width of the first screen image.

[0189] Step S1112: Calculate the second scaling ratio between the length of the first screen image and the length of the screen image at the projection sending end;

[0190] The length of the first screen image is the length of the screen image on the projection display end. The length of the projection display end is determined by the screen size of the projection display end, and its actual screen size is usually fixed. For example, if the screen size of the projection display end is 1920*1080 pixels, then the length of the first screen image is 1080.

[0191] Specifically, the second scaling ratio = the length of the screen image of the projection sending end / the length of the first screen image.

[0192] Step S1113: Based on the first scaling ratio and the second scaling ratio, determine the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end;

[0193] Specifically, the coordinates of the pixels in the effective display area are obtained. The coordinates of the pixels in the effective display area include the horizontal and vertical coordinates. The horizontal coordinate of the pixel in the screen image of the projection sending end = the first scaling ratio * the horizontal coordinate of the pixel in the effective display area. The vertical coordinate of the pixel in the screen image of the projection sending end = the second scaling ratio * the vertical coordinate of the pixel in the effective display area.

[0194] For example, if the coordinates of a pixel in the effective display area are (x1, y1) and the touch coordinates are (x2, y2), and the values ​​of x1 and y1 are known, then x2 = x1 * first scaling factor, and y2 = y1 * second scaling factor.

[0195] In this embodiment of the application, the one-to-one correspondence between the coordinates of each pixel in the effective display area and the coordinates of each pixel in the screen image of the projection sending end can be determined through the above calculation method, that is, the mapping relationship between the coordinates of the pixels in the effective display area and the coordinates of the pixels in the screen image of the projection sending end is determined.

[0196] Step S1102: Based on the mapping relationship, determine the touch coordinates corresponding to the coordinates of the pixels in the effective display area;

[0197] Specifically, based on the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end, when the screen of the projection display end is clicked, the coordinates corresponding to the current click are identified. Based on the mapping relationship, the touch coordinates corresponding to the current click coordinates are determined, so as to determine the touch coordinates corresponding to the coordinates of pixels in the effective display area.

[0198] In this embodiment of the application, after obtaining the touch coordinates, the touch coordinates are sent to the screen projection sending end to control the screen of the screen projection sending end.

[0199] Please see Figure 13 , Figure 13 This is a schematic diagram of the overall process of a screen state detection method for a screen projection sending end provided in an embodiment of this application;

[0200] like Figure 13 As shown, the overall flow of the screen status detection method of the screen projection sending end includes:

[0201] Step S1301: Acquire the first screen image;

[0202] The first screen image is the screen image of the projection display terminal, and the first screen image includes the screen image of the projection sending terminal.

[0203] Step S1302: Binarize the first screen image to determine the effective display area;

[0204] Specifically, the first screen image is binarized so that it only displays white and black. The coordinates of the pixels displaying white are obtained. Based on the coordinates of all the pixels displaying white, a rectangle is constructed. This rectangle is used as the effective display area, which is the position of the screen image of the projection sending end in the first screen image.

[0205] Step S1303: Calculate the width and length of the effective display area;

[0206] Specifically, a coordinate system is constructed in the first screen image, and the pixel coordinates of the upper left corner and the lower right corner of the effective display area are obtained based on the coordinate system. Based on the pixel coordinates of the upper left corner and the lower right corner, the width and length of the effective display area are calculated.

[0207] Step S1304: Determine whether the width of the effective display area is greater than its length;

[0208] Specifically, it is determined whether the width of the effective display area is greater than the length of the effective display area. If the width of the effective display area is greater than the length of the effective display area, the process jumps to step S1305. If the width of the effective display area is less than the length of the effective display area, the process jumps to step S1306.

[0209] Step S1305: Determine that the screen state of the screen projection sending end is landscape mode, and determine the touch coordinates based on the landscape mode;

[0210] Specifically, when the width of the effective display area is greater than the length of the effective display area, the screen state of the projection sending end is determined to be landscape mode. Based on the landscape mode, the scaling ratio of the width of the effective display area and the screen image of the projection sending end is calculated, as well as the scaling ratio of the length of the effective display area and the screen image of the projection sending end is calculated. Based on the scaling ratio of the width and the scaling ratio of the length, the touch coordinates are determined.

[0211] Step S1306: Determine that the screen state of the screen projection sending end is portrait mode, and determine the touch coordinates based on the portrait mode;

[0212] Specifically, when the width of the effective display area is less than the length of the effective display area, the screen state of the projection sending end is determined to be portrait mode. Based on the portrait mode, the scaling ratio of the width of the effective display area and the screen image of the projection sending end is calculated, as well as the scaling ratio of the length of the effective display area and the screen image of the projection sending end is calculated. Based on the scaling ratio of the width and the scaling ratio of the length, the touch coordinates are determined.

[0213] Step S1307: Send the touch coordinates to the projection sending end;

[0214] Specifically, the obtained touch coordinates are sent to the projection sending end, so that the screen of the projection sending end can be controlled from the projection display end through the touch coordinates.

[0215] In this embodiment, by automatically detecting the screen state of the projection sending end at the projection display end, the screen parameters of the projection display end are automatically adjusted, and a mapping relationship is established to quickly obtain touch coordinates, thereby realizing control of the projection sending end and improving control efficiency.

[0216] Please see Figure 14 , Figure 14 This is a schematic diagram of the structure of a screen state detection system for a screen projection sending end provided in an embodiment of this application;

[0217] like Figure 14 As shown, the screen status detection system 1400 of the screen projection sending end includes a screen projection display end 10 and a screen projection sending end 20, wherein the screen projection display end 10 and the screen projection sending end 20 communicate with each other.

[0218] The screen projection display terminal 10 is used to display the screen image of the screen projection sending terminal 20 and to control the screen of the screen projection sending terminal 20.

[0219] The screen projection sending end 20 is used to send the screen image of the screen projection sending end 20 to the screen projection display end 10.

[0220] Please see Figure 15 , Figure 15This is a schematic diagram of the structure of a screen state detection device for a screen projection sending end provided in an embodiment of this application;

[0221] like Figure 15 As shown, the screen status detection device 1500 of the screen projection sending end includes:

[0222] The first acquisition unit 1501 is used to acquire a first screen image, wherein the first screen image includes the screen image of the projection sending end;

[0223] The binarization processing unit 1502 is used to perform binarization processing on the first screen image to determine the effective display area, wherein the effective display area is the position of the second screen image in the first screen image;

[0224] The second acquisition unit 1503 is used to acquire the pixel coordinates of the effective display area;

[0225] The calculation unit 1504 is used to calculate the width and length of the effective display area based on the pixel coordinates of the effective display area;

[0226] The screen state determination unit 1505 is used to determine the screen state of the projection sending end based on the width and length, wherein the screen state includes landscape state or portrait state.

[0227] In this embodiment, the screen state detection device of the screen projection sending end can be a software module. The software module includes several instructions, which are stored in a memory. The processor can access the memory and call the instructions to execute them, so as to complete the screen state detection method of the screen projection sending end in the above embodiments.

[0228] In the embodiments of this application, the screen status detection device of the screen projection sending end can also be built from hardware devices. For example, the screen status detection device of the screen projection sending end can be built from one or more chips, and the chips can work together to complete the screen status detection method of the screen projection sending end described in the above embodiments. As another example, the screen status detection device of the screen projection sending end can also be built from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0229] The screen status detection device of the screen projection sending end in this application embodiment can be a device, or it can be a component, integrated circuit, or chip in the terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not make specific limitations.

[0230] The screen state detection device of the screen projection sending end in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit it.

[0231] The screen status detection device for the screen projection sending end provided in this application embodiment can achieve... Figure 3 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0232] It should be noted that the above-described device can execute the screen state detection method of the screen projection sending end provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the device embodiments can be found in the screen state detection method of the screen projection sending end provided in the embodiments of this application.

[0233] In this embodiment, the various modules of the screen state detection device of the screen projection sending end cooperate with each other to automatically identify the screen orientation of the screen projection sending end at the screen projection display end, so as to automatically adjust the display screen of the screen projection display end and further improve the control efficiency of the screen projection display end.

[0234] Please see Figure 16 , Figure 16 This is a schematic diagram of the structure of a screen projection display terminal provided in an embodiment of this application;

[0235] like Figure 16 As shown, the projection display terminal 10 includes one or more processors 11 and a memory 12. Among them, Figure 16 Take a processor 11 as an example.

[0236] Processor 11 and memory 12 can be connected via a bus or other means. Figure 16 Taking the example of a connection between China and Israel via a bus.

[0237] A processor is configured to execute a screen state detection method for a screen projection sending end according to any embodiment of this application, comprising: acquiring a first screen image, wherein the first screen image includes a screen image of the screen projection sending end; performing binarization processing on the first screen image to determine an effective display area, wherein the effective display area is the position of a second screen image in the first screen image; acquiring pixel coordinates of the effective display area; calculating the width and length of the effective display area based on the pixel coordinates of the effective display area; and determining the screen state of the screen projection sending end according to the width and length, wherein the screen state includes a landscape state or a portrait state.

[0238] By binarizing the first screen image, the effective display area is determined, and then the length and width of the effective display area are calculated. Based on the length and width of the effective display area, the screen state of the projection sending end is determined. This allows for automatic identification of the screen state of the projection sending end, enabling rapid adjustment of the display screen on the projection display end and improving the control efficiency of the projection display end.

[0239] The memory 12, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the screen state detection method of the screen projection sending end in the embodiments of the present invention. The processor 11 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions, and modules stored in the memory 12, thereby implementing the screen state detection method of the screen projection sending end in the above method embodiments.

[0240] Memory 12 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 12 may optionally include memory remotely located relative to processor 11. Examples of the above-described networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0241] One or more modules are stored in memory 12. When executed by one or more processors 11, they perform the screen state detection method of the projection sending end in any of the above method embodiments, for example, performing the above-described... Figure 3 The steps shown.

[0242] This application also provides a computer program product, which includes one or more lines of program code stored in a non-volatile computer-readable storage medium. The processor of the electronic device reads the program code from the non-volatile computer-readable storage medium and executes the program code to complete the steps of the screen state detection method of the screen projection sending end provided in the above embodiments.

[0243] Based on the above description of the embodiments, those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a non-volatile computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0244] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The non-volatile computer-readable storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0245] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations as described above in different aspects of this application, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting the screen status of a screen projection sending end, characterized in that, The method is applied to a screen projection display terminal, which communicates with the screen projection sending terminal. The screen projection display terminal is used to display the screen image of the screen projection sending terminal. The method includes: Acquire a first screen image, wherein the first screen image includes the screen image of the projection sending end; The first screen image is binarized to determine the effective display area, wherein the effective display area is the position of the screen image of the projection sending end in the first screen image; Obtain the pixel coordinates of the effective display area; Calculate the width and length of the effective display area based on the pixel coordinates of the effective display area; Based on the width and length, the screen state of the projection sending end is determined, wherein the screen state includes landscape mode or portrait mode.

2. The method according to claim 1, characterized in that, The binarization process of the first screen image to determine the effective display area includes: Obtain the color value of each pixel in the first screen image; The color value of each pixel is weighted to obtain the grayscale value of each pixel; The effective display area is determined based on the grayscale value.

3. The method according to claim 2, characterized in that, Determining the effective display area based on the grayscale value includes: Obtain the coordinates of pixels whose grayscale value is greater than the grayscale threshold; Based on the coordinates of pixels with gray values ​​greater than a gray threshold, a first coordinate and a second coordinate are determined. The first coordinate includes a first abscissa and a first ordinate, and the second coordinate includes a second abscissa and a second ordinate. The first abscissa is the minimum abscissa among all pixel coordinates with gray values ​​greater than a gray threshold, the first ordinate is the maximum ordinate among all pixel coordinates with gray values ​​greater than a gray threshold, the second abscissa is the maximum abscissa among all pixel coordinates with gray values ​​greater than a gray threshold, and the second ordinate is the minimum ordinate among all pixel coordinates with gray values ​​greater than a gray threshold. The effective display area is determined based on the first and second coordinates.

4. The method according to claim 3, characterized in that, Before binarizing the first screen image, the method further includes: The filtering process for the first screen image specifically includes: Obtain the noise tolerance value; Based on the noise tolerance value, the first screen image is filtered.

5. The method according to claim 4, characterized in that, The acquisition of the noise tolerance value includes: Initialize a noise value; Within a preset time period, the first screen image is binarized multiple times based on the noise value to obtain multiple second screen images. The second screen images are screen images after binarization, and the screen images after binarization include an effective display area. If the width and length of the effective display area in each of the second screen images are greater than a preset threshold, and the variation values ​​of the width and length of the effective display areas in multiple second screen images are all within the range, then the noise value is determined to be the noise tolerance value.

6. The method according to claim 4, characterized in that, The step of calculating the width and length of the effective display area based on the pixel coordinates of the effective display area includes: Calculate the first difference between the first horizontal coordinate and the second horizontal coordinate, and use the absolute value of the first difference as the width of the effective display area; Calculate the second difference between the first ordinate and the second ordinate, and use the absolute value of the second difference as the length of the effective display area.

7. The method according to claim 1, characterized in that, Determining the screen orientation (portrait or landscape) of the projection sending end based on the width and length includes: If the width of the effective display area is greater than the length, then the screen projection sending end is determined to be in landscape mode; If the width of the effective display area is less than the length, then the screen projection sending end is determined to be in portrait mode.

8. The method according to claim 1, characterized in that, The method further includes: Determine the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end; Based on the mapping relationship, the touch coordinates corresponding to the coordinates of the pixels in the effective display area are determined, wherein the touch coordinates are the coordinates of the touch point on the screen of the projection sending end.

9. The method according to claim 8, characterized in that, Determining the mapping relationship between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end includes: Calculate the first scaling ratio between the width of the first screen image and the width of the screen image at the projection sending end; Calculate a second scaling factor between the length of the first screen image and the length of the screen image at the projection sending end; Based on the first scaling ratio and the second scaling ratio, a mapping relationship is determined between the coordinates of pixels in the effective display area and the coordinates of pixels in the screen image of the projection sending end, wherein the horizontal coordinate of the pixel in the screen image of the projection sending end = the first scaling ratio * the horizontal coordinate of the pixel in the effective display area, and the vertical coordinate of the pixel in the screen image of the projection sending end = the second scaling ratio * the vertical coordinate of the pixel in the effective display area.

10. A system for detecting the screen status of a screen projection sending end, characterized in that, The system includes: The screen projection display terminal is used to execute the screen state detection method of the screen projection sending terminal as described in any one of claims 1 to 9; A screen projection sending end is used to send the screen image of the screen projection sending end to the screen projection display end.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the screen state detection method of the screen projection sending end as described in any one of claims 1 to 9.

Images