Image display method, device, micro display screen and medium based on micro display screen

Abstract

The present invention relates to the field of micro display technology, and discloses an image display method, device, micro display screen and medium based on a micro display screen, the method comprising: receiving an image to be displayed sent by a MIPI video transmitter through a MIPI interface, and receiving auxiliary display information through other interfaces; determining multiple candidate display areas in the micro display screen based on the resolution of the auxiliary display information; respectively counting pixel distribution results of the image to be displayed in each candidate display area, and determining a target display area from multiple candidate display areas; combining the image to be displayed with the auxiliary display information to obtain a target image of the target display area; and displaying the image to be displayed outside the target display area and the target image of the target display area in the micro display screen. The present invention realizes the synchronous display function of the micro display screen for video data and auxiliary information, reduces the requirements for MIPI chips, reduces the cost of the micro display screen, and improves the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of micro display screens, and in particular to an image display method, device, micro display screen and medium based on a micro display screen. Background Art

[0002] At present, micro-displays such as micro-led micro-displays are increasingly widely used and have been widely used in AR technology, such as AR glasses and other products. It sends video data to the micro-display chip of the micro-display through the Mobile Industry Processor Interface (MIPI). The micro-display screen displays the picture on the lens through the optical system. People can see the projected picture and the real world at the same time. Usually, in addition to displaying image data on the micro-display, it is also necessary to display some auxiliary information related to the image on the screen synchronously, for example: translating people's English conversations in the real world into Chinese and displaying them on the micro-display of AR glasses.

[0003] In the prior art, the image of auxiliary information is directly embedded with the MIPI image to be displayed and sent to the micro display chip, which makes the entire system very complicated. MIPI is not just a video transmitter, but also needs to process auxiliary information such as translation content and add it to the original image. This will place higher requirements on the MIPI sending chip function, greatly increasing the cost of the MIPI chip, and thus increasing the cost of the entire micro display product. Summary of the invention

[0004] In view of this, the present invention provides an image display method, device, micro display screen and medium based on a micro display screen to solve the problem in the related art that the micro display screen needs to rely on a more expensive MIPI chip when synchronously displaying video data and other auxiliary information, resulting in a higher cost for the entire micro display screen.

[0005] In a first aspect, the present invention provides an image display method based on a micro display screen, the method comprising:

[0006] Receive the image to be displayed sent by the MIPI video transmitter through the MIPI interface, and receive auxiliary display information through other interfaces;

[0007] Determine a resolution corresponding to the auxiliary display information, and determine a plurality of candidate display areas of the auxiliary display information in the micro display screen based on the resolution;

[0008] Counting pixel distribution results of the image to be displayed in each candidate display area respectively;

[0009] Determine a target display area from a plurality of candidate display areas based on the pixel distribution result;

[0010] Combining the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image in the target display area;

[0011] The image to be displayed outside the target display area and the target image in the target display area are displayed in the micro display screen.

[0012] The present invention only uses the MIPI video transmitter to send image data, receives auxiliary display information through other interfaces, and processes the auxiliary display information by using the data processing capability of the controller of the micro display screen itself, thereby screening out a target display area, and combining the image data with the auxiliary display information to display them in the target display area, thereby realizing the synchronous display function of the micro display screen for video data and other auxiliary information, and reducing the requirements for the MIPI chip, thereby reducing the cost of the entire micro display screen product and improving the user experience.

[0013] In an optional implementation, the determining, based on the resolution, a plurality of candidate display areas for the auxiliary display information in the micro display screen includes:

[0014] Determining the number of horizontal pixels and the number of vertical pixels corresponding to the auxiliary display information based on the resolution;

[0015] Based on the magnitude relationship between the number of horizontal pixels and the number of vertical pixels, a plurality of candidate display areas of the auxiliary display information in the micro display screen are determined.

[0016] The present invention determines the shape of the display area required for the auxiliary display information by using the horizontal pixel number and the vertical pixel number of the auxiliary display information, and then selects multiple suitable candidate display areas in the micro display screen, so that the display distribution of the auxiliary display information in the micro display screen is more reasonable, thereby improving the user's viewing experience.

[0017] In an optional implementation, determining a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the size relationship between the number of horizontal pixels and the number of vertical pixels includes:

[0018] Determining whether the number of horizontal pixels is greater than the number of vertical pixels;

[0019] When the number of horizontal pixels is greater than the number of vertical pixels, a first area is determined from top to bottom starting from the upper edge of the micro display screen based on the number of vertical pixels, and a second area is determined from bottom to top starting from the lower edge of the micro display screen;

[0020] The first area and the second area are divided based on the number of horizontal pixels to obtain a plurality of candidate display areas.

[0021] When the number of horizontal pixels is greater than the number of vertical pixels, that is, when the auxiliary display information is in the shape of a horizontal strip, the present invention sets the candidate display area of ​​the auxiliary display information above or below the display screen, so that the display effect of the auxiliary display information on the screen is more in line with the user's viewing habits, thereby further improving the user experience.

[0022] In an optional embodiment, the method further includes:

[0023] When the number of horizontal pixels is not greater than the number of vertical pixels, a third area is determined from left to right starting from the left edge of the micro display screen based on the number of horizontal pixels, and a fourth area is determined from right to left starting from the right edge of the micro display screen;

[0024] The third area and the fourth area are divided based on the number of vertical pixels to obtain a plurality of candidate display areas.

[0025] When the number of horizontal pixels is not greater than the number of vertical pixels, that is, when the auxiliary display information is in the shape of a vertical strip, the present invention sets the candidate display area of ​​the auxiliary display information on the left or right side of the display screen, so that the display effect of the auxiliary display information on the screen is more in line with the user's viewing habits, thereby further improving the user experience.

[0026] In an optional implementation, determining a target display area from a plurality of candidate display areas based on the pixel distribution result includes:

[0027] Respectively counting the number of pixels in each preset pixel interval in the pixel distribution result corresponding to each candidate display area;

[0028] The candidate display area having the largest number of preset pixel intervals with zero pixels in the pixel distribution result is determined as the target display area.

[0029] The present invention counts the number of pixels in each preset pixel interval in each candidate display area, screens out the candidate display area with the largest number of preset pixel intervals with zero pixels, and determines it as the target display area, that is, screens out the area with the most concentrated pixel distribution for auxiliary information display, so that the pixel fluctuation of the image to be displayed in the target display area is minimized, thereby reducing the display interference of the auxiliary display information on the image to be displayed, which can maximize the display effect of the image to be displayed and further enhance the user's viewing experience.

[0030] In an optional implementation, combining the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain the target image of the target display area includes:

[0031] Respectively from the first pixel and the second pixel at the current pixel position of the image data of the image to be displayed in the target display area and the image data corresponding to the auxiliary display information;

[0032] Combining the first pixel with the second pixel to obtain a display pixel at the current pixel position;

[0033] A target image of the target display area is generated based on display pixels at each pixel position in the target display area.

[0034] The present invention obtains the final display pixels of each pixel position by combining the pixels of the image to be displayed corresponding to each pixel position in the target display area with the pixels corresponding to the auxiliary display information, and then obtains the target image finally displayed in the entire target display area, thereby realizing the fusion display of the auxiliary display information and the image to be displayed through the controller of the micro display screen, without increasing other hardware costs, thereby improving the user experience.

[0035] In an optional implementation, the receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface and receiving the auxiliary display information through other interfaces includes:

[0036] After receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface, determining whether auxiliary display information is received through other interfaces within a preset time;

[0037] If no auxiliary display information is received through other interfaces within a preset time, the image to be displayed is displayed on the micro display screen.

[0038] If the auxiliary display information is received through other interfaces within a preset time, the steps of determining a resolution corresponding to the auxiliary display information and determining a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the resolution are performed.

[0039] The present invention determines whether there is corresponding auxiliary display information for the current image to be displayed by determining whether the auxiliary display information is received within a set time after receiving the image to be displayed. If the auxiliary display information is not received, the image to be displayed is directly displayed, thereby reducing the data processing amount of the micro display screen controller, ensuring the display efficiency of the micro display screen, and realizing the fusion display function of the auxiliary display information and the image to be displayed by utilizing the processing power of the controller when the auxiliary information is received, thereby further improving the user experience.

[0040] In a second aspect, the present invention provides an image display device based on a micro display screen, applied to a controller of a micro display screen, the device comprising:

[0041] A receiving module, used for receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface, and receiving auxiliary display information through other interfaces;

[0042] A first processing module, configured to determine a resolution corresponding to the auxiliary display information, and determine a plurality of candidate display areas of the auxiliary display information in the micro display screen based on the resolution;

[0043] A second processing module, used for respectively counting pixel distribution results of the image to be displayed in each candidate display area;

[0044] A third processing module, configured to determine a target display area from a plurality of candidate display areas based on the pixel distribution result;

[0045] A fourth processing module, configured to combine the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image of the target display area;

[0046] The fifth processing module is used to display the image to be displayed outside the target display area and the target image in the target display area in the micro display screen.

[0047] In a third aspect, the present invention provides a micro display screen, comprising: a controller, the controller comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method provided in the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method provided in the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 is a flow chart of an image display method based on a micro display screen according to an embodiment of the present invention;

[0051] Figure 2 is a flow chart of another image display method based on a micro display screen according to an embodiment of the present invention;

[0052] Figure 3 is an example diagram of a candidate display area in a micro display screen according to an embodiment of the present invention;

[0053] Figure 4 is an example diagram of a candidate display area in another micro display screen according to an embodiment of the present invention;

[0054] Figure 5 is a schematic structural diagram of a micro display screen according to an embodiment of the present invention;

[0055] Figure 6 is a schematic structural diagram of an image display device based on a micro display screen according to an embodiment of the present invention;

[0056] Figure 7 4 is a schematic diagram of the structure of a controller of a micro display screen according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0058] When the micro display screen is displaying image data, and needs to display some auxiliary information related to the image on the screen synchronously, for example, translating people's English conversations in the real world into Chinese and displaying them on the micro display screen of AR glasses. In the prior art, the image of the auxiliary information is directly embedded with the MIPI image to be displayed and sent to the micro display chip, which makes the whole system very complicated. MIPI is not just a video transmitter, but also needs to process auxiliary information such as translation content and add it to the original image. This will have higher requirements on the function of the MIPI sending chip, greatly increasing the cost of the MIPI chip, and then increasing the cost of the entire micro display product.

[0059] The technical solution provided by the present invention uses the MIPI chip only as a MIPI transmitter to send the main video signal, and does not send and process other auxiliary display information. The other auxiliary display information is sent to the controller of the micro display screen, that is, the micro display chip, using other interfaces, and the auxiliary display information is processed by using the data processing capability of the micro display chip itself to be integrated with the image to be displayed at a suitable position of the micro display screen, thereby realizing the synchronous display function of video data and auxiliary display information, and reducing the requirements for the MIPI chip, reducing the chip hardware cost, and then reducing the cost of the entire micro display screen product, thereby improving the user experience.

[0060] According to an embodiment of the present invention, an embodiment of an image display method based on a micro display screen is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] Based on the above problems, in this embodiment, a method for displaying an image based on a micro display screen is provided, which is applied to a controller in a micro display screen, such as a CPU, a single-chip microcomputer, and other micro display chips. It should be noted that in the embodiment of the present invention, a micro-led micro display screen is used as an example for description. In practical applications, the method for displaying an image based on a micro display screen is also applicable to other types of micro display screens, and the present invention is not limited thereto. Figure 1 is a flow chart of an image display method based on a micro display screen according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0062] Step S101, receiving an image to be displayed sent by a MIPI video transmitter through a MIPI interface, and receiving auxiliary display information through other interfaces.

[0063] Specifically, the MIPI video transmitter is a MIPI chip, and other interfaces are SPI / QSPI interfaces. Auxiliary information can be sent through MCU or other low-end chips to further reduce costs. The image to be displayed is a single-frame image of video data, and the auxiliary display information hereinafter referred to as auxiliary information is prompt information, voice interaction information, translation information, etc. that need to be synchronously displayed on the image to be displayed, but the present invention is not limited to this.

[0064] Step S102: determining a resolution corresponding to the auxiliary display information, and determining a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the resolution.

[0065] Specifically, the resolution corresponding to the auxiliary display information represents the number of rows and columns of pixels occupied by the display. Assuming that the resolution is x*y, it requires an area of ​​x rows and y columns of pixels to display on the micro display screen. The candidate display area is the location where the auxiliary information can be displayed.

[0066] Step S103 , counting pixel distribution results of the image to be displayed in each candidate display area respectively.

[0067] For example, assuming that the data bit width is 8 bits, the pixel range of the image pixel is 0-255, and the pixel range can be divided into several statistical intervals, such as being evenly divided into 8 intervals, that is, pixel data 0-31, 32-63... Every 32 data are divided into a group for distribution statistics, and the interval in which the pixel value of the current pixel is located is counted by 1, and the pixel data in each candidate display area is distributed and counted in this way to obtain the corresponding pixel distribution result. The present invention is only taken as an example and is not limited thereto.

[0068] Step S104: determining a target display area from a plurality of candidate display areas based on the pixel distribution result.

[0069] Specifically, since the pixel distribution result can intuitively reflect the distribution of effective image data of the image to be displayed in each candidate area, such as the distribution of pixels corresponding to the face area in the face image, the background image area without occlusion of the face area or the area to be displayed with less occlusion can be selected as the target display area to reduce the impact of the auxiliary information display on the display effect of the original image to be displayed, thereby improving the user's viewing experience. In addition, in practical applications, the target display area can also be determined based on the pixel distribution result in combination with special auxiliary information display requirements. For example, if auxiliary information needs to be displayed at some specific positions in the image to be displayed, the area where the candidate display area overlaps with the specific position can be determined as the target display area. The present invention is only taken as an example and is not limited to this.

[0070] Step S105 , combining the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image in the target display area.

[0071] Specifically, since the image to be displayed and the auxiliary information need to be displayed synchronously in the target display area, when the image is displayed in the area, the target image of the area can be obtained by combining the corresponding image data of the two, thereby realizing the fusion of the image data of the two.

[0072] Step S106, displaying the image to be displayed outside the target display area and the target image in the target display area on the micro display screen.

[0073] Specifically, the target image obtained in the above steps is directly displayed in the target display area of ​​the micro display screen, and the image to be displayed is directly displayed in the area outside the target display area, thereby realizing the synchronous display function of the auxiliary information and the frame image in the video data in the micro display screen.

[0074] The embodiment of the present invention only uses the MIPI video transmitter to send image data, receives auxiliary display information through other interfaces, and processes the auxiliary display information by using the data processing capability of the controller of the micro display screen itself, thereby screening out the target display area, and combining the image data with the auxiliary display information and displaying them in the target display area, thereby realizing the micro display screen's synchronous display function of video data and other auxiliary information, and reducing the requirements for the MIPI chip, thereby reducing the cost of the entire micro display screen product and improving the user experience.

[0075] In this embodiment, a method for displaying an image based on a micro display screen is also provided, which is applied to a controller in the micro display screen, such as a CPU, a single-chip microcomputer, or other micro display chips. Figure 2 is a flow chart of an image display method based on a micro display screen according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0076] Step S201, receiving an image to be displayed sent by a MIPI video transmitter through a MIPI interface, and receiving auxiliary display information through other interfaces.

[0077] Specifically, the above step S201 includes:

[0078] Step a1, after receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface, it is determined whether auxiliary display information is received through other interfaces within a preset time.

[0079] Specifically, the preset time can be the interval time of video frames for sending video data by the MIPI video transmitter, or it can be the sending time interval between two adjacent images to be displayed. Exemplarily, the preset time is 2s, 3s, etc., but the present invention is not limited thereto.

[0080] Step a2: if no auxiliary display information is received through other interfaces within a preset time, the image to be displayed is displayed on the micro display screen.

[0081] Specifically, if no auxiliary display information is received through other interfaces within the preset time, it means that the current image to be displayed has no corresponding auxiliary information that needs to be displayed. Therefore, the auxiliary information display function can be turned off, and the image to be displayed can be directly displayed on the micro display screen, thereby simplifying the data processing process of the micro display screen for image display and improving display efficiency.

[0082] Step a3, if the auxiliary display information is received through other interfaces within a preset time, the step of determining the resolution corresponding to the auxiliary display information and determining a plurality of candidate display areas of the auxiliary display information in the micro display screen based on the resolution is executed.

[0083] Specifically, if auxiliary display information is received through other interfaces within a preset time, it indicates that auxiliary information of the current image to be displayed needs to be displayed synchronously. At this time, the auxiliary information display function is turned on, and the image to be displayed and the auxiliary information can be displayed synchronously by executing step S202.

[0084] The embodiment of the present invention determines whether there is corresponding auxiliary display information for the current image to be displayed by determining whether the auxiliary display information is received within a set time after receiving the image to be displayed. If the auxiliary display information is not received, the image to be displayed is directly displayed, thereby reducing the data processing amount of the micro display screen controller, ensuring the display efficiency of the micro display screen, and realizing the fusion display function of the auxiliary display information and the image to be displayed by utilizing the processing power of the controller when the auxiliary information is received, thereby further improving the user experience.

[0085] Step S202: determining a resolution corresponding to the auxiliary display information, and determining a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the resolution.

[0086] Specifically, the above step S202 includes:

[0087] Step S2021, determining the number of horizontal pixels and the number of vertical pixels corresponding to the auxiliary display information based on the resolution.

[0088] Exemplarily, assuming that the resolution corresponding to the auxiliary display information is x*y, the corresponding horizontal number of pixels and vertical number of pixels on the micro display screen are x and y respectively.

[0089] Step S2022: determining a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the size relationship between the number of horizontal pixels and the number of vertical pixels.

[0090] The embodiment of the present invention determines the shape of the display area required for the auxiliary display information by utilizing the horizontal pixel number and the vertical pixel number of the auxiliary display information, and then selects multiple suitable candidate display areas in the micro display screen, so that the display distribution of the auxiliary display information in the micro display screen is more reasonable, thereby improving the user's viewing experience.

[0091] Furthermore, the above step S2022 includes:

[0092] Step b1, determining whether the number of horizontal pixels is greater than the number of vertical pixels.

[0093] Step b2, when the number of horizontal pixels is greater than the number of vertical pixels, a first area is determined from top to bottom starting from the upper edge of the micro display screen based on the number of vertical pixels, and a second area is determined from bottom to top starting from the lower edge of the micro display screen.

[0094] Specifically, when the number of horizontal pixels is greater than the number of vertical pixels, it means that the display area required to be occupied by the auxiliary information is a horizontal strip. In order to make the display effect of the auxiliary display information on the screen more in line with the user's viewing habits, it can be set above or below the micro display screen to make its distribution on the screen more reasonable.

[0095] Step b3, dividing the first area and the second area based on the number of horizontal pixels to obtain a plurality of candidate display areas.

[0096] Specifically, since the resolution of the auxiliary information is usually much smaller than the resolution of the micro display screen itself, when the auxiliary information is located above or below the micro display screen, there may be a variety of display schemes at different positions in the micro display screen. For example, assuming that the number of horizontal pixels of the auxiliary information is 1 / 3 of the number of pixels in each row of the micro display screen, the upper and lower areas of the micro display screen may be divided into the following: Figure 3 In addition to the 3 areas shown, you can also Figure 3 The priority of each candidate display area is determined by the numbering order in, but the present invention is not limited to this.

[0097] In an embodiment of the present invention, when the number of horizontal pixels is greater than the number of vertical pixels, that is, when the auxiliary display information is in the shape of a horizontal strip, the candidate display area of ​​the auxiliary display information is set above or below the display screen, so that the display effect of the auxiliary display information on the screen is more in line with the user's viewing habits, thereby further improving the user experience.

[0098] Step b4, when the number of horizontal pixels is not greater than the number of vertical pixels, a third area is determined from left to right starting from the left edge of the micro display screen based on the number of horizontal pixels, and a fourth area is determined from right to left starting from the right edge of the micro display screen.

[0099] Specifically, when the number of horizontal pixels is not greater than the number of vertical pixels, it indicates that the display area required to be occupied by the auxiliary information is a vertical strip. In order to make the display effect of the auxiliary display information on the screen more in line with the user's viewing habits, it can be set on the left or right side of the micro display screen to make its distribution on the screen more reasonable.

[0100] Step b5, dividing the third area and the fourth area based on the number of vertical pixels to obtain a plurality of candidate display areas.

[0101] Specifically, since the resolution of the auxiliary information is usually much smaller than the resolution of the micro display screen itself, when the auxiliary information is located on the left or right side of the micro display screen, there may be a variety of display schemes at different positions in the micro display screen. For example, assuming that the number of vertical pixels of the auxiliary information is 1 / 3 of the number of pixels in each column of the micro display screen, the left and right areas of the micro display screen may be divided into the following: Figure 4 In addition to the 3 areas shown, you can also Figure 4 The priority of each candidate display area is determined by the numbering order in, but the present invention is not limited to this.

[0102] In an embodiment of the present invention, when the number of horizontal pixels is not greater than the number of vertical pixels, that is, when the auxiliary display information is in the shape of a vertical strip, the candidate display area of ​​the auxiliary display information is set on the left or right side of the display screen. This makes the display effect of the auxiliary display information on the screen more in line with the user's viewing habits, thereby further improving the user experience.

[0103] Step S203, respectively count the pixel distribution results of the image to be displayed in each candidate display area. Figure 1 The relevant description of step S103 is not repeated here.

[0104] Step S204: determining a target display area from a plurality of candidate display areas based on the pixel distribution result.

[0105] Specifically, the above step S204 includes:

[0106] Step S2041 , counting the number of pixels in each preset pixel interval in the pixel distribution result corresponding to each candidate display area.

[0107] Specifically, the preset pixel interval can be flexibly set according to the screening accuracy and actual needs of the target display area for displaying auxiliary information. For example, assuming that the range of pixel data is 0-255, the preset pixel interval can be evenly divided into 8 intervals, that is, pixel data 0-31, 32-63... Every 32 pixel data are divided into a preset pixel interval for distribution statistics. This is just an example, and the present invention is not limited to this.

[0108] Step S2042: determine the candidate display area having the largest number of preset pixel intervals with zero pixels in the pixel distribution result as the target display area.

[0109] Specifically, when the number of preset pixel intervals distributed by pixel data in a candidate display area is more, it means that the image content contained in the candidate display area is richer, such as the face area in a face image, etc. On the contrary, when the number of preset pixel intervals distributed by pixel data in a candidate display area is less, it means that the image content contained in the candidate display area is simpler, such as the background area in a face image, etc. Therefore, the embodiment of the present invention reduces the interference of the auxiliary information on the display of the image to be displayed by respectively counting the number of preset pixel intervals with zero pixels and selecting the candidate display area with the most concentrated pixel distribution and the smallest pixel fluctuation for auxiliary information display.

[0110] In practical applications, if there are multiple candidate display area pixel distribution results with the same number of preset pixel intervals with zero pixels, the priority order of the candidate display areas pre-divided as above can be used as follows: Figure 3 and Figure 4 The target display area is determined by the numbering order in the , so as to ensure that the display position of the auxiliary information is more in line with the user's actual usage needs and further improve the user experience.

[0111] The embodiment of the present invention counts the number of pixels in each preset pixel interval in each candidate display area, and screens out the candidate display area with the largest number of preset pixel intervals with zero pixels, and determines it as the target display area. That is, the area with the most concentrated pixel distribution is screened out for auxiliary information display, so that the pixel fluctuation of the image to be displayed in the target display area is minimized, thereby reducing the display interference of the auxiliary display information on the image to be displayed, which can maximize the display effect of the image to be displayed and further enhance the user's viewing experience.

[0112] Step S205 , combining the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image in the target display area.

[0113] Specifically, the above step S205 includes:

[0114] Step S2051 , respectively obtaining the first pixel and the second pixel at the current pixel position from the image data of the image to be displayed in the target display area and the image data corresponding to the auxiliary display information.

[0115] Step S2052: Combine the first pixel and the second pixel to obtain a display pixel at the current pixel position.

[0116] Specifically, assuming that the pixel value of the first pixel is a and the pixel value of the second pixel is b, the display pixel c=a+b can be obtained by directly superimposing the first pixel and the second pixel. In practical applications, the pixel combination method can also be flexibly set based on the specific application scenario requirements of the micro display screen.

[0117] Application scenario 1: When the micro display screen is used in a scenario where the video data, i.e., the image to be displayed, is used as the background and the auxiliary display information data is used as the main display data, the new display pixel data can be obtained by multiplying the pixel data of the frame of video by a coefficient less than 1, such as 10%, and then adding the pixel data of the auxiliary information. For example, assuming that the video data bit width is 8 bits, the pixel data display range is 0-255. Assuming that the current pixel value of the video data is 100, multiply it by 30% to obtain the background video data of 30, and assuming that the data of the auxiliary information is 0, this pixel displays the background data of 30. Assuming that the data of the auxiliary information is 240, the new data of 270 is obtained, which is larger than 255, so the data is cut off to 255, and the pixel data of 255 is displayed at this position.

[0118] Application scenario 2: When the micro display screen is used in a scenario where the video data, i.e., the image to be displayed, is used as the background and the auxiliary display information data is used as the main display data, the new video background data can be obtained by multiplying the frame video pixel data by a coefficient less than 1, such as 10%, and then the new video background data is displayed when the auxiliary information data is 0. When the auxiliary information data is not 0, only the auxiliary information pixel data is displayed. That is, compared with application scenario 1, pixel data is not superimposed, and when the auxiliary information is 0, the reduced video pixel data is displayed, so that it can be applied to scenarios such as translated text display. The present invention is only taken as an example and is not limited thereto.

[0119] Step S2053: generating a target image of the target display area based on display pixels at each pixel position in the target display area.

[0120] Specifically, the display pixels at each pixel position in the target display area and the target image finally displayed in the target display area constitute a process known in the art and will not be described in detail herein.

[0121] The embodiment of the present invention obtains the final display pixels of each pixel position by combining the pixels of the image to be displayed corresponding to each pixel position in the target display area with the pixels corresponding to the auxiliary display information, and then obtains the target image finally displayed in the entire target display area, thereby realizing the fusion display of the auxiliary display information and the image to be displayed through the controller of the micro display screen, without increasing other hardware costs, thereby improving the user experience.

[0122] Step S206, displaying the image to be displayed outside the target display area and the target image in the target display area on the micro display screen. Figure 1 The relevant description of step S106 is not repeated here.

[0123] The specific working principle and working process of image display by the micro display screen provided by the embodiment of the present invention will be described in detail below in conjunction with specific application examples.

[0124] Exemplarily, the above-mentioned image display scheme of the micro display screen is referred to as a digital driving algorithm of the micro display screen. The algorithm is deployed in the controller of the micro display screen, that is, the micro display chip, and runs. Figure 5 As shown, the basic working process of the micro display screen for image display is as follows:

[0125] 1. The MIPI video transmitter, i.e. the MIPI chip, only sends the main video and does not need to process other auxiliary information. The MIPI video signal is saved by the video data frame buffer circuit of the micro display.

[0126] 2. MCU or other devices that can process simple signals, such as translated text, send data through the SPI / QSPI interface. The micro display screen auxiliary information data frame buffer circuit receives the auxiliary information data sent by SPI / QSPI. In this way, the MCU can be a more general and low-end MCU chip.

[0127] 3. The display position of auxiliary information can be set through registers. For example, if the translated text is to be displayed in the lower 30 lines of the screen, or if a small window pops up, it is to be displayed in the upper 120x60 pixel area of ​​the screen, etc. The area where the auxiliary information is displayed, i.e. the target display area mentioned above, is very small compared to the entire pixel area. Therefore, the data sent by SPI / QSPI is much less, so the SPI / QSPI transmission rate can be lower.

[0128] 4. When the digital drive algorithm deployed in the micro display chip reads the video data, it displays the area at the auxiliary position and reads the auxiliary information data at the same time. The auxiliary information and video data are combined at the auxiliary information display position and output to the subsequent row / column drive circuit and pixel array. The video data is displayed normally in the non-auxiliary display area of ​​the micro display screen.

[0129] It should be noted that Figure 5 The specific circuit structure, operating principle and working process of the video data frame buffer circuit, auxiliary information data frame buffer circuit, microprocessing chip, row / column driving circuit and pixel array in the microdisplay for realizing the basic function of microdisplay image display are prior arts. For details, reference can be made to the descriptions of relevant prior arts and will not be elaborated here. The main inventive point of the embodiments of the present invention lies in the digital driving algorithm deployed and running on the microprocessing chip. Through this digital driving algorithm, the synchronous display function of video data and auxiliary information on the microdisplay is realized, so that it is not necessary to rely on a high-performance MIPI chip, and the hardware cost is reduced.

[0130] Specifically, the digital driving algorithm confirms that the resolution of the auxiliary information is x rows by y columns, and this resolution is less than the actual resolution of the microdisplay. When x < y, it is a horizontally long-strip-shaped auxiliary information. When x > y, it is a vertically long-strip-shaped auxiliary information. When the auxiliary information is a horizontal long strip, the auxiliary information is displayed above or below the screen. When the auxiliary information is a vertical long strip, the auxiliary information is displayed on the left or right side of the screen. That is, the horizontal direction has the highest priority to be displayed in the last x rows and the middle y columns, the second priority is to be displayed in the first x rows and the middle y columns, the third choice is to be displayed in the last x rows and the first y columns, the fourth choice is to be displayed in the last x rows and the last y columns, the fifth choice is to be displayed in the first x rows and the first y columns, and the sixth choice is to be displayed in the first x rows and the last y columns. Similarly, when displaying a vertical bar, the first choice is to be displayed in the middle x rows and the first y columns, the second choice is to be displayed in the middle x rows and the last y columns, and so on.

[0131] Exemplarily, assuming that when displaying a horizontal bar, according to the Figure 3 horizontal order shown, the pixel data distribution results of the normal video data are extracted in sequence. For example, the pixel data distribution results of area 1 are extracted first. The pixel data range is usually 0 - 255, which is evenly divided into 8 intervals, that is, pixel data 0 - 31, 32 - 63... Every 32 data are divided into a group for distribution statistics. That is, the number of pixels in these 8 intervals where the pixel data of area 1 are distributed is counted. For example, the chip sequentially reads the pixel data of area 1. When the read pixel data is in which interval, the count of that interval is incremented by 1. After reading the pixel data of area 1 and saving the distribution counts corresponding to the 8 intervals. Similarly, the pixel data of other areas are read and the corresponding distribution data are obtained. First, the distribution data of area 1 and area 2 are compared. The area with more regions having a count of 0 in the 8 statistical intervals wins. If the numbers are the same, area 1 is preferentially selected. Similarly, compare with area 3, 4... one by one, and finally the displayed area is obtained. The more concentrated the distribution is, the less information the video data has. Therefore, the best display area with the most concentrated data distribution and less information of the video data at the edge positions is selected to display the auxiliary information. Similarly, the best display area for the vertical bar can also be obtained. Exemplarily, the display area of the vertical bar is as Figure 4shown.

[0132] Optionally, the micro display chip monitors the auxiliary information sending interface, starts timing when the auxiliary information does not send data, and if no further sending is done within 2 seconds, it can be considered that there is no other auxiliary information to be displayed, and the auxiliary information display function is turned off until the auxiliary information sending data is detected next time, and then this function is turned on.

[0133] Application example 1 can display only auxiliary information in the auxiliary display area without displaying video data, which is suitable for small windows such as pop-up videos.

[0134] Application Example 2: Video data can be used as the background and auxiliary information data can be used as the main display data at the auxiliary display area. For example, the video data is multiplied by a coefficient less than 1, such as 10%, and then the auxiliary information data is added to obtain new data. For example, assuming the data width is 8 bits, the data range is 0-255. Assuming the video data is 100, multiplied by 30%, the background video data of 30 is obtained. Assuming the auxiliary information data is 0, this pixel displays the background data of 30. Assuming the auxiliary information data is 240, the new data of 270 is obtained, which is larger than 255, so the data is cut off at 255. Therefore, data of 255 is displayed at this position.

[0135] Application Example 3, the video data can be used as the background at the auxiliary display area, and the auxiliary information data can be used as the main display data. For example, the video data is multiplied by a coefficient less than 1, such as 10%, to obtain new video background data. When the auxiliary information data is 0, the new video background data is displayed. When the auxiliary information data is not 0, the auxiliary information data is displayed. That is, compared with the above-mentioned Application Example 2, no addition is performed, but when the auxiliary information is 0, the reduced video data is displayed, and when the auxiliary information is not 0, the auxiliary information data is displayed, which can be applied to scenes such as translated text display.

[0136] Therefore, image data is sent only by utilizing the MIPI video transmitter, while auxiliary display information is received through other interfaces, and the auxiliary display information is processed by utilizing the data processing capability of the micro display's controller, thereby screening out the target display area, and combining the image data with the auxiliary display information to display them in the target display area, thereby realizing the micro display's function of synchronously displaying video data and other auxiliary information, and reducing the requirements for the MIPI chip, thereby reducing the cost of the entire micro display product and improving the user experience.

[0137] In the embodiment of the present invention, an image display device based on a micro display screen is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0138] This embodiment provides an image display device based on a micro display screen, such as Figure 6 As shown, the device comprises:

[0139] The receiving module 601 is used to receive the image to be displayed sent by the MIPI video transmitter through the MIPI interface, and receive the auxiliary display information through other interfaces;

[0140] A first processing module 602 is used to determine a resolution corresponding to the auxiliary display information, and determine a plurality of candidate display areas of the auxiliary display information in the micro display screen based on the resolution;

[0141] The second processing module 603 is used to count the pixel distribution results of the image to be displayed in each candidate display area;

[0142] A third processing module 604 is used to determine a target display area from multiple candidate display areas based on the pixel distribution result;

[0143] A fourth processing module 605 is used to combine the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image in the target display area;

[0144] The fifth processing module 606 is used to display the image to be displayed outside the target display area and the target image in the target display area on the micro display screen.

[0145] In some optional implementations, the first processing module 602 includes:

[0146] A first processing unit, configured to determine the number of horizontal pixels and the number of vertical pixels corresponding to the auxiliary display information based on the resolution;

[0147] The second processing unit is used to determine a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the size relationship between the number of horizontal pixels and the number of vertical pixels.

[0148] In some optional implementations, the second processing unit includes:

[0149] A judging subunit, used to judge whether the number of horizontal pixels is greater than the number of vertical pixels;

[0150] A first processing subunit, configured to determine a first area from top to bottom starting from an upper edge of the micro display screen based on the number of vertical pixels when the number of horizontal pixels is greater than the number of vertical pixels, and to determine a second area from bottom to top starting from a lower edge of the micro display screen;

[0151] The second processing subunit is used to divide the first area into the second area based on the number of horizontal pixels to obtain a plurality of candidate display areas.

[0152] In some optional implementations, the second processing unit further includes:

[0153] a third processing subunit, configured to determine a third area from left to right starting from a left edge of the micro display screen based on the number of horizontal pixels when the number of horizontal pixels is not greater than the number of vertical pixels, and to determine a fourth area from right to left starting from a right edge of the micro display screen;

[0154] The fourth processing subunit is used to divide the third area and the fourth area based on the number of vertical pixels to obtain a plurality of candidate display areas.

[0155] In some optional implementations, the third processing module 604 includes:

[0156] A third processing unit, used to count the number of pixels in each preset pixel interval in the pixel distribution result corresponding to each candidate display area;

[0157] The fourth processing unit is configured to determine the candidate display area having the largest number of preset pixel intervals with zero pixels in the pixel distribution result as the target display area.

[0158] In some optional implementations, the fourth processing module 605 includes:

[0159] A fifth processing unit, configured to respectively obtain a first pixel and a second pixel at a current pixel position from the image data of the image to be displayed in the target display area and the image data corresponding to the auxiliary display information;

[0160] a sixth processing unit, configured to combine the first pixel with the second pixel to obtain a display pixel at a current pixel position;

[0161] The seventh processing unit is used to generate a target image of the target display area based on display pixels at each pixel position in the target display area.

[0162] In some optional implementations, the receiving module 601 includes:

[0163] an eighth processing unit, configured to determine whether auxiliary display information is received through other interfaces within a preset time after receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface;

[0164] The ninth processing unit is used to display the image to be displayed on the micro display screen if no auxiliary display information is received through other interfaces within a preset time.

[0165] The tenth processing unit is configured to call the first processing module if auxiliary display information is received through other interfaces within a preset time.

[0166] The image display device based on the micro display screen in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0167] The further functional description of each of the above modules and units is the same as that of the above corresponding method embodiments and will not be repeated here.

[0168] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a controller of a micro display screen provided by an optional embodiment of the present invention, such as Figure 7 As shown, the controller includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0169] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0170] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0171] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0173] The controller further comprises a communication interface 30 for the control unit to communicate with other devices or a communication network.

[0174] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0175] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An image display method based on a micro display screen, applied to a controller of a micro display screen, characterized in that: The method comprises: Receive the image to be displayed sent by the MIPI video transmitter through the MIPI interface, and receive auxiliary display information through other interfaces; Determine a resolution corresponding to the auxiliary display information, and determine a plurality of candidate display areas of the auxiliary display information in the micro display screen based on the resolution; Counting pixel distribution results of the image to be displayed in each candidate display area respectively; Determine a target display area from a plurality of candidate display areas based on the pixel distribution result; Combining the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image in the target display area; The image to be displayed outside the target display area and the target image in the target display area are displayed in the micro display screen.

2. The method according to claim 1, characterized in that The determining, based on the resolution, a plurality of candidate display areas of the auxiliary display information in the micro display screen comprises: Determining the number of horizontal pixels and the number of vertical pixels corresponding to the auxiliary display information based on the resolution; Based on the magnitude relationship between the number of horizontal pixels and the number of vertical pixels, a plurality of candidate display areas of the auxiliary display information in the micro display screen are determined.

3. The method according to claim 2, characterized in that The determining, based on the magnitude relationship between the number of horizontal pixels and the number of vertical pixels, a plurality of candidate display areas of the auxiliary display information in the micro display screen comprises: Determining whether the number of horizontal pixels is greater than the number of vertical pixels; When the number of horizontal pixels is greater than the number of vertical pixels, a first area is determined from top to bottom starting from the upper edge of the micro display screen based on the number of vertical pixels, and a second area is determined from bottom to top starting from the lower edge of the micro display screen; The first area and the second area are divided based on the number of horizontal pixels to obtain a plurality of candidate display areas.

4. The method according to claim 3, characterized in that The method further comprises: When the number of horizontal pixels is not greater than the number of vertical pixels, a third area is determined from left to right starting from the left edge of the micro display screen based on the number of horizontal pixels, and a fourth area is determined from right to left starting from the right edge of the micro display screen; The third area and the fourth area are divided based on the number of vertical pixels to obtain a plurality of candidate display areas.

5. The method according to claim 1, characterized in that The determining a target display area from a plurality of candidate display areas based on the pixel distribution result comprises: Respectively counting the number of pixels in each preset pixel interval in the pixel distribution result corresponding to each candidate display area; The candidate display area having the largest number of preset pixel intervals with zero pixels in the pixel distribution result is determined as the target display area.

6. The method according to claim 1, characterized in that The step of combining the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain the target image of the target display area includes: Respectively from the first pixel and the second pixel at the current pixel position of the image data of the image to be displayed in the target display area and the image data corresponding to the auxiliary display information; Combining the first pixel with the second pixel to obtain a display pixel at the current pixel position; A target image of the target display area is generated based on display pixels at each pixel position in the target display area.

7. The method according to any one of claims 1 to 6, characterized in that: The receiving of the image to be displayed sent by the MIPI video transmitter through the MIPI interface and receiving the auxiliary display information through other interfaces include: After receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface, determining whether auxiliary display information is received through other interfaces within a preset time; If no auxiliary display information is received through other interfaces within a preset time, displaying the image to be displayed on the micro display screen; If the auxiliary display information is received through other interfaces within a preset time, the steps of determining a resolution corresponding to the auxiliary display information and determining a plurality of candidate display areas for the auxiliary display information in the micro display screen based on the resolution are performed.

8. An image display device based on a micro display screen, applied to a controller of a micro display screen, characterized in that: The device comprises: A receiving module, used for receiving the image to be displayed sent by the MIPI video transmitter through the MIPI interface, and receiving auxiliary display information through other interfaces; A first processing module, configured to determine a resolution corresponding to the auxiliary display information, and determine a plurality of candidate display areas of the auxiliary display information in the micro display screen based on the resolution; A second processing module, used for respectively counting pixel distribution results of the image to be displayed in each candidate display area; A third processing module, configured to determine a target display area from a plurality of candidate display areas based on the pixel distribution result; A fourth processing module, configured to combine the image data of the image to be displayed in the target display area with the image data corresponding to the auxiliary display information to obtain a target image of the target display area; The fifth processing module is used to display the image to be displayed outside the target display area and the target image in the target display area in the micro display screen.

9. A micro display screen, characterized in that: include: A controller, the controller comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images