Image display processing method and device and storage medium

Abstract

The embodiment of the invention provides an image display processing method and device and a storage medium, and belongs to the technical field of image processing. The method comprises the steps that vertex coordinates of a plurality of target rectangular images are determined in a preset canvas, and layers corresponding to the target rectangular images are different; scanning each target rectangular image, and determining pixel segment information of uncovered pixel segments in each row of each target rectangular image according to a scanning result and each vertex coordinate; sorting the pixel segment information of each pixel segment to obtain target pixel segment information; requesting target image data corresponding to the target pixel segment information from a preset memory; and updating the target rectangular image according to the target image data. The device and the storage medium can realize the method. The embodiment of the invention can give consideration to the image display effect while reducing the hardware resource consumption of the device and the bandwidth consumption of the hardware system of the device.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and storage medium for processing image display. Background Art

[0002] In the field of image display technology, in order to relieve the pressure on the central processing unit (CPU) of the device, a layer processor can be added as a hardware accelerator to process the image separately before displaying it. In the related art, since there are multiple images displayed in the same canvas, there is a possibility of overlapping each other. Therefore, in the related art, it is necessary to use multiple layer processors to extract each image from the memory respectively, and then process the image in the overlapping area through a layer aliasing algorithm to ensure that multiple images can be displayed in the same canvas in the case of overlap. However, although this technical solution can ensure the display effect of the canvas, it will lead to an increase in the consumption of hardware resources of the device and an increase in the bandwidth demand of the hardware system. At the same time, when at least one image displayed at the same time in the canvas corresponds to a frame in the video and the frames of the video are continuously changing for at least a period of time, this technical solution will also cause the consumption of the bandwidth of the device system, thereby increasing the consumption of hardware resources of the entire system. Therefore, how to reduce the consumption of hardware resources of the device and the bandwidth consumption of the hardware system while taking into account the effect of image display is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide an image display processing method, device and storage medium, which can reduce the device hardware resource consumption and hardware system bandwidth consumption while taking into account the image display effect.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for processing image display, which is applied to a layer processor; the method comprises: In response to an image position adjustment request triggered by a position change of at least one target rectangular image in a preset canvas, vertex coordinates of a plurality of target rectangular images displayed in the canvas are determined, wherein each of the target rectangular images corresponds to a different layer; and at least two of the plurality of target rectangular images at least partially overlap; Scan each of the target rectangular images respectively, and determine pixel segment information of uncovered pixel segments in each row of each of the target rectangular images according to the scanning results and the vertex coordinates; Sort the pixel segment information to obtain target pixel segment information; In response to the image update request of the canvas, requesting target image data corresponding to the target pixel segment information from a preset memory; According to the target image data, the uncovered pixels in each target rectangular image in the canvas are updated.

[0005] To achieve the above object, a second aspect of an embodiment of the present application provides a terminal device, wherein the terminal device includes: A layer processor, wherein the layer processor executes the method according to any one of the first aspects; A memory, wherein the memory is connected to the layer processor via a preset bus communication.

[0006] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the image display processing method described in any one of the first aspects when executing the computer program.

[0007] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0008] The image display processing method, device and storage medium proposed in the present application can determine the pixel segment information of the uncovered pixel segments in the display area of ​​the target rectangular image by scanning the vertex coordinates of each target rectangular image and each target rectangular image, so that the target rectangular image at the same position can be extracted from the memory based on each pixel segment information. At this time, since the target rectangular image sent by the memory does not contain covered pixels, the system bandwidth consumption of the entire device can be saved. At the same time, there is no need to perform overlapping processing on the overlapping area in the layer processor. When the display position of each target rectangular image changes, it is only necessary to re-determine the target pixel segment information to update each target rectangular image after the position adjustment in the canvas. Compared with the related art, the embodiment of the present application can reduce the interaction of image data with the memory without adding a layer aliasing algorithm, especially when at least one video is displayed in the canvas, and one frame of each video corresponds to a target rectangular image that is frequently updated, the hardware resource consumption of the entire device can be further reduced. Therefore, the embodiment of the present application can reduce the hardware resource consumption of the device and the bandwidth consumption of the hardware system while taking into account the effect of image display. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of the structure of an embodiment of a terminal device provided in an embodiment of the present application; Figure 2 It is a module diagram of a partial structure of an embodiment of a terminal device provided in an embodiment of the present application; Figure 3It is a schematic diagram of canvas display of an embodiment of the image display processing method provided in the embodiment of the present application; Figure 4 It is a flowchart of a method for processing image display provided in an embodiment of the present application; Figure 5 This is a schematic diagram of pixel segment information storage in one embodiment of the image display processing method provided in the embodiment of the present application; Figure 6 It is a flowchart of sorting pixel segment information in an embodiment of the image display processing method provided in an embodiment of the present application; Figure 7 It is a structural diagram of the hardware structure corresponding to the image display processing method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0011] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0013] In the field of image display technology, in order to relieve the pressure on the central processing unit (CPU) of the device, a layer processor can be added as a hardware accelerator to process the image separately and then display it. In the related art, since there are multiple images displayed in the same canvas, there is a possibility of overlapping each other. Therefore, in the related art, it is necessary to use multiple layer processors to extract each image from the memory respectively, and then use the layer aliasing algorithm to cover the image in the overlapping area to ensure that multiple images can be displayed in the same canvas in the case of overlap. However, although this technical solution can ensure the display effect of the canvas, it will lead to an increase in the consumption of hardware resources of the device and an increase in the bandwidth demand of the hardware system. At the same time, when displaying a video in the canvas, it is necessary to update the displayed image in the canvas according to the content of the video frame, especially when the image of each frame in the video is changing, it is necessary to update the image of each frame in the canvas. At this time, this technical solution will also cause the consumption of the bandwidth of the device system, which will lead to an increase in the consumption of hardware resources of the entire system. Therefore, how to reduce the consumption of hardware resources of the device and the bandwidth demand of the hardware system while taking into account the effect of image display is a technical problem that needs to be solved urgently. Based on this, the embodiments of the present application propose an image display processing method, device and storage medium, which can reduce the hardware resource consumption of the device and the bandwidth consumption of the hardware system while taking into account the effect of image display.

[0014] The image display processing method, device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the image display processing method in the embodiments of the present application is described.

[0015] The processing method of the image display of the present application can be used in many general or special computer system environments or configurations. For example: a hardware accelerator in a personal computer, a server computer, a handheld device or a portable device, a tablet device, a multiprocessor system, a microprocessor-based system, a set-top box, a programmable consumer electronic device, a network PC, a minicomputer, a mainframe computer, a distributed computing environment including any of the above systems or devices, etc., is used for image display processing. The present application can be described in the general context of a computer executable instruction executed by a computer, such as a program module. Generally, a program module includes routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by a remote processing device connected through a communication network. In a distributed computing environment, a program module can be located in a local and remote computer storage medium including a storage device.

[0016] Refer to the following Figure 1As shown, according to the terminal device provided in the embodiment of the present application, the terminal device includes: Layer Processor; The memory is connected to the layer processor through a preset bus communication; Among them, the layer processor responds to an image position adjustment request triggered by a position change of at least one target rectangular image in a preset canvas, determines the vertex coordinates of multiple target rectangular images displayed in the canvas, and each target rectangular image corresponds to a different layer; scans each target rectangular image separately, and determines the pixel segment information of the uncovered pixel segments in each row of each target rectangular image based on the scanning results and the vertex coordinates; sorts the pixel segment information to obtain the target pixel segment information; responds to the image update request of the canvas, requests the target image data corresponding to the target pixel segment information from the preset memory; and updates the uncovered pixels in each target rectangular image in the canvas based on the target image data.

[0017] Each target rectangular image corresponds to a rectangular display area in the canvas. The target rectangular image indicates that the outline of the image is a rectangle, and the target rectangular image in the canvas is the image that has been displayed. The embodiment of the present application does not limit the outline and effect of the target rectangular image finally presented in the canvas, such as adding outlines of different shapes to the boundaries of each target rectangular image in the canvas.

[0018] It is understandable that the memory can be used to cache images displayed in the display area where each target rectangular image is located in the canvas (for example, if a video is displayed in a display area in the canvas, then the memory stores images of each frame of the video displayed as a target rectangular image), and the canvas is used to display the target rectangular image to the user. In the embodiment of the present application, there is no restriction on the position of the target rectangular image that can be displayed in the canvas. The position of the target rectangular image in the canvas is determined by the user, such as being triggered by a real-time user request or being determined by the user configuring the canvas parameters when displaying the canvas. In this regard, the embodiment of the present application does not impose any restrictions.

[0019] It is understandable that, since there are multiple target rectangular images in the canvas, and the positions of the target rectangular images can be randomly generated based on user triggers, there is a certain probability that the target rectangular images will overlap. The embodiment of the present application can be applied to scenarios where the content of at least one target rectangular image in the same canvas changes over time at different times, resulting in frequent changes in the content displayed on the canvas at different times. In this way, the consumption of device hardware resources and hardware system bandwidth can be reduced while taking into account the effect of image display.

[0020] The embodiment of the present application does not limit the type of memory, nor does it limit the type of bus; the embodiment of the present application does not limit the structure of the layer processor, and those skilled in the art can selectively set it according to actual needs.

[0021] It is understandable that each target rectangular image is composed of multiple rows and columns of pixels, and the definition of the rows or columns of the target rectangular image in the embodiment of the present application can be selectively set according to actual needs. For example, the horizontal direction of the canvas represents rows, and the vertical direction represents columns. For example, in other embodiments, the horizontal direction of the canvas represents columns, and the vertical direction represents rows.

[0022] By comparing the coordinates of each vertex with the coordinates of each pixel in each row, it is possible to determine whether the scanned pixel is within or outside the overlapping area, and then determine the coordinate range of the uncovered pixel segment to obtain the pixel segment information of each row.

[0023] The embodiment of the present application does not limit how to store each pixel segment information, and those skilled in the art can selectively set it according to actual needs. The pixel segment information is sorted to obtain the target pixel segment information, and multiple target rectangular images can be used as a complete image to extract the images row by row or column by column, thereby ensuring the display effect.

[0024] For example, refer to Figure 2 As shown, taking the AXI bus as an example, the layer processor includes a processing module and a DMA (Direct Memory Access); the DMA and the memory are connected through the bus, wherein a plurality of target rectangular images to be displayed are stored in the memory, and target rectangular images at different display positions are stored in different storage areas, or the same target rectangular image to be displayed can be stored in different storage areas (wherein the target rectangular images can correspond to images corresponding to each frame in the video); the DMA receives a request message from the processing module, and the DMA converts the target pixel segment information into an image request message in AXI format through the AXI protocol, and then sends the image request message to the memory through the AXI interface; and receives the target rectangular image data returned by the memory.

[0025] The first scanning module scans the overlapping form of each row of images according to the pixel coordinates in the layer where each target rectangular image is located (reflected by the attribution and length information of each pixel segment); The valid segment processing module is used to send corresponding requests in turn according to the information of all valid pixel segments after scanning a line of image segmentation, and save the valid segment information (that is, the pixel segments not covered) to the valid segment storage module; The data preprocessing module will delete invalid data from the requested data and extract valid data according to the pixel segment information of the uncovered pixel segment read from the valid segment storage module.

[0026] The data pool is used to store valid data; The second scanning module is used to scan the coordinates of the target rectangular image in the canvas according to the filling direction of the canvas (such as from left to right, from top to bottom), and instruct the filling module to extract data from the data pool; The filling module is used to fill the invalid position of the target rectangular image in the canvas with background data, and the valid area is filled with data from the data pool.

[0027] Refer to the following Figure 4 As shown, the image display processing method provided in the embodiment of the present application is applied to a layer processor; the image display processing method includes: Step S100, in response to an image position adjustment request triggered by a position change of at least one target rectangular image in a preset canvas, determining vertex coordinates of multiple target rectangular images displayed in the canvas, wherein each target rectangular image corresponds to a different layer; and at least two of the multiple target rectangular images at least partially overlap; Step S200, scanning each target rectangular image respectively, and determining pixel segment information of uncovered pixel segments in each row of each target rectangular image according to the scanning result and each vertex coordinate; Step S300, sorting the information of each pixel segment to obtain the target pixel segment information; Step S400: In response to the image update request of the canvas, request the target image data corresponding to the target pixel segment information from the preset memory; Step S500: Update uncovered pixels in each target rectangular image in the canvas according to the target image data.

[0028] Therefore, by using the vertex coordinates of each target rectangular image and scanning each target rectangular image, the pixel segment information of the uncovered pixel segment in the display area of ​​the target rectangular image can be determined, so that the target rectangular image at the same position can be extracted from the memory based on each pixel segment information. At this time, since the target rectangular image sent by the memory does not contain covered pixels, the system bandwidth consumption of the entire device can be saved. At the same time, there is no need to perform overlapping processing on the overlapping area in the layer processor. When the display position of each target rectangular image changes, it is only necessary to re-determine the target pixel segment information to update each target rectangular image after the position adjustment in the canvas. The embodiment of the present application does not need to add a layer aliasing algorithm and can reduce the interaction of image data with the memory. In particular, when at least one video is displayed in the canvas, and one frame of each video corresponds to a target rectangular image that is frequently updated, the bandwidth consumption of the entire device hardware resources can be further reduced. Therefore, the embodiment of the present application can reduce the device hardware resource consumption and the hardware system bandwidth consumption while taking into account the image display effect.

[0029] The embodiment of the present application does not limit the scanning method of the target rectangular image in step S200. The scanning may be started line by line along the horizontal direction of the canvas, or along the vertical direction of the canvas, or each target rectangular image may be scanned line by line. Those skilled in the art may selectively set it according to actual needs.

[0030] It can be understood that sorting the information of each pixel segment refers to sorting the offset position along a preset direction, such as sorting according to the size of the longitudinal offset of the canvas, or sorting along the size of the lateral offset of the canvas, with the x-coordinate corresponding to the lateral offset and the y-coordinate corresponding to the longitudinal offset. When sorting according to the size of the longitudinal offset of the canvas, the information of each pixel segment is sorted according to the y-coordinate, and when sorting according to the size of the lateral offset of the canvas, the information of each pixel segment is sorted according to the x-coordinate.

[0031] The image update request is used to update the displayed image data in the canvas and can be triggered periodically.

[0032] It is understandable that in step S200, scanning each target rectangular image separately includes: Get the display order of each target rectangular image; According to the display order, the image displayed at the bottom layer of each target rectangular image is used as the target traversal image; Start scanning the first row of the target traversal image row by row along the column direction of the canvas; After the target traversal image is traversed and scanned, the target traversal image is updated to the target rectangular image of the previous layer according to the display order and jumps to the first row of the target traversal image along the column direction of the canvas to start traversal scanning row by row; When all target rectangular images except the target rectangular image displayed on the outermost layer have been traversed and scanned, the scanning stops.

[0033] The rows of the target traversal image are perpendicular to the column direction of the canvas, and the column direction of the canvas can be either the horizontal direction or the vertical direction of the canvas.

[0034] The second layer indicates the next layer immediately following the outermost layer. Figure 3 As shown, five target rectangular images are set in the canvas, which are target rectangular image PIC4, target rectangular image PIC3, target rectangular image PIC2, ​​target rectangular image PIC1 and target rectangular image PIC0 from the outer layer to the inner layer along the depth direction of the canvas; among them, PIC0 is the bottom layer image, PIC4 is the outermost layer image, and PIC3 is the target rectangular image displayed in the second layer.

[0035] Scan in order of display and start from the bottom layer. Then, each time you make a coverage judgment, you only need to judge whether the target rectangular image on the coverage layer is covered. This makes the judgment more efficient. Figure 3 As shown, for PIC0, PIC1 to PIC4 need to be judged, and for PIC1, only PIC2 to PIC4 need to be judged. And because the outermost target rectangular image will definitely not be blocked, reducing the coverage judgment can further improve the processing efficiency.

[0036] The display order can be pre-set without real-time scanning, or it can be determined in real time based on the scan. In this regard, the embodiments of the present application do not limit how the display order is determined. The embodiments of the present application do not limit how the scanning results are stored. In some embodiments, only the endpoint information of the uncovered pixel segments can be saved as pixel segment information and dynamically requested for memory to save. In other embodiments, a fixed storage space is pre-allocated for each target rectangular image of each different layer. The storage space can store information on each pixel segment when each upper image is segmented. For example, Figure 3As shown, taking the canvas distributed with 5 target rectangular images as an example, the 5 target rectangular images are PIC0, PIC1, PIC2, ​​PIC3 and PIC4 from inside to outside along the depth direction of the canvas, then for PIC0, it is divided into at most 5 valid segments (that is, uncovered pixel segments), namely Seg0, Seg1, Seg2, Seg3 and Seg4, wherein Seg0 and Seg1 are formed by segmentation of PIC1 and PIC2, ​​the starting coordinates of Seg0 are the starting coordinates of PIC0, the ending coordinates of Seg0 are the starting coordinates of PIC1 when PIC0 is segmented by PIC1, the starting coordinates of Seg1 are the ending coordinates of PIC1 when PIC0 is segmented by PIC1, the ending coordinates of Seg1 are the starting coordinates of PIC2 when PIC0 is segmented by PIC2, ​​Seg1 and Seg2 are segmented by PIC1, PIC2 and PIC3, and their corresponding coordinates refer to the above-mentioned Seg0 and Seg1, and Seg2 and Seg3 are segmented by PIC2 and PIC3. By analogy, each row of the target rectangular image of each different layer can be allocated storage space according to the situation of being divided into the most valid segments, and when scanning line by line, the starting coordinates and terminal coordinates of each valid segment are filled in respectively to obtain the scanning result. Taking PIC0 as an example, assuming that PIC1 does not divide PIC0, and PIC2 divides PIC0, then x1, x2, and x3 are all filled with the starting coordinates of PIC2 when PIC0 is divided by PIC2. Similarly, other valid segments are set in the same way.

[0037] It is understandable that the scanning result in step S200 is the scanning result of each row of the target traversal image, and the pixel segment information of the uncovered pixel segment in each row of each target rectangular image is determined according to the scanning result and each vertex coordinate, including: According to the display order, the vertex coordinates of the target rectangular image in the layer covering the target traversal image are determined as the target vertex coordinates; In the process of traversing each row of pixels in the target traversal image, the coordinates of the traversed pixels are compared with the coordinates of the target vertices to determine the pixel segment information of the uncovered pixel segment in the target traversal image; Each row in the target rectangular image of the uppermost layer is used as an uncovered pixel segment, and pixel segment information of the corresponding uncovered pixel segment is determined according to the vertex coordinates of the target rectangular image of the uppermost layer.

[0038] It is understandable that by performing a coverage determination each time a row is scanned, processing efficiency can be improved.

[0039] It is understandable that in step S400, requesting the target image data corresponding to the target pixel segment information from the preset memory includes: Obtaining a bus protocol of a bus connected to the memory; According to the bus protocol, each of the target pixel segment information is encapsulated to obtain an image request message corresponding to each of the target pixel segment information; Sending image request messages to the memories respectively through the bus; Receive target image data returned from the memory and corresponding to the image request information one by one.

[0040] By encapsulating the image request information according to the bus protocol, the layer processor can directly read data from the memory, reducing the pressure on the CPU.

[0041] The image request message may be encapsulated one at a time and then sent out, or all the image request messages may be encapsulated and then sent out.

[0042] It is understandable that the bus protocol is the AXI protocol; receiving the target image data returned from the memory corresponding to the image request information one by one, including: Get the field alignment length determined by the AXI protocol; The target image data is read from the memory according to the field alignment length and the image request message.

[0043] For example, if the AXI bus is configured with a data width of 64 bits, a piece of data has 8 bytes, so the data must also be aligned to 8 bytes when it is retrieved. The image request message records its valid field information, so based on the image request information, invalid data can be removed when the returned target image data is not aligned.

[0044] The embodiments of the present application do not limit how to set the extraction indication information. In some embodiments, the extraction indication information includes a data head offset (head_offset), a data tail offset (tail_offset), and a data length (data_len).

[0045] It can be understood that the image request message includes a data header offset and a data tail offset. According to the field alignment length and the image request message, the target image data is read from the memory, including: Receive response data of field-aligned length from memory; Determine the reading start position of the response data according to the data header offset; Starting from the starting position, data is extracted one by one from the response data until the position indicated by the data tail offset is reached to obtain the target image data.

[0046] Take the AXI bus configured with a 64-bit data width as an example. Assuming that a pixel occupies 1 byte, a piece of data on the bus can be represented as: A0A1A2A3A4A5A6A7, and the next piece is represented as A8A9A10A11A12A13A14A15. When the extraction indication information indicates that the valid data starts from A2, the head_offset is 2. The extraction indication information indicates that the valid length of the current segment is 10 points, then the tail_offset indicates that the last piece of data, A12A13A14A5, these 4 bytes of data are invalid data, so the tail_offset is 4.

[0047] It is understandable that the information of each pixel segment is sorted to obtain the target pixel segment information, including: Get the preset sorting method; According to the sorting method, the pixel segment information of each pixel segment located on the same straight line is sorted according to the coordinate size of the pixel segment to obtain the target pixel segment information corresponding to each row in the canvas.

[0048] The sorting method can be from large to small or from small to large, and this embodiment of the present application does not limit this.

[0049] For example, refer to Figure 6 As shown, taking four target rectangular images set in the canvas as an example, PIC0, PIC1 and PIC2 are scanned respectively, and the scanning results of PIC0, PIC1 and PIC2 are sorted together, so as to obtain the pixel segment information of the valid segments of each row on the canvas, and generate the target image data of each row on the canvas based on the pixel segment information.

[0050] For example, refer to Figure 3 , Figure 5 and Figure 6 As shown in the figure, take the case where there are five overlapping target rectangular images in the canvas, the five target rectangular images are PIC0, PIC1, PIC2, ​​PIC3 and PIC4 respectively; the memory stores five target rectangular images to be displayed at time t+1. Assuming that at time t the layer processor receives a request for adjusting the image position of the canvas, wherein the request for adjusting the image position indicates that the display areas of the five target rectangular images have changed, the layer processor performs the following steps: 1) Since PIC0 can be divided into 5 segments at most by PIC1, PIC2, ​​PIC3 and PIC4, there will be less than 5 segments when there is overlap with PIC1, PIC2, ​​PIC3 and PIC4. In order to take all situations into account, 5 segment information spaces are opened for each row of PIC0 to store the horizontal offset of each valid segment (that is, the uncovered pixel segment) relative to PIC0 and the length of the valid segment (wherein, when the valid segment is less than 5 segments, the invalid segment length is represented by 0). At this time, the 5 segment information of each row of PIC0 after being divided can be obtained by scanning PIC0; 2) Since PIC1 can be divided into 4 segments at most by PIC2, ​​PIC3 and PIC4, there will be less than 4 segments when there is mutual overlap with PIC2, ​​PIC3 and PIC4. Therefore, in order to take all situations into consideration, 4 segment information spaces are allocated for each row of PIC1 to store the horizontal offset of each valid segment relative to PIC1 and the length of the valid segment (wherein, when the valid segment is less than 4 segments, the invalid segment length is represented by 0). At this time, the 4 valid segment information (that is, the pixel segment information of the uncovered pixel segment) of each row of PIC1 after being divided can be obtained by scanning PIC1; 3) Since PIC2 can be divided into at most 3 valid segments by PIC3 and PIC4, there will be less than 3 segments when there is mutual overlap with PIC3 and PIC4. Therefore, in order to take all situations into consideration, 3 segment information spaces are allocated for each row of PIC2 to store the horizontal offset of each valid segment relative to PIC2 and the length of the valid segment (wherein, when the valid segment is less than 3 segments, the invalid segment length is represented by 0). At this time, the 3 valid segment information (that is, the pixel segment information of the uncovered pixel segment) of each row of PIC2 after being divided can be obtained by scanning PIC2; 4) Similarly, for PIC3, we can get the two valid segment information after each row is split; for PIC4, since PIC,4 is at the top layer and will not be covered by any PIC, we can directly retain the horizontal offset and length of PIC4 as the last segment information; 5) Repeat steps 1) to 4) above to obtain the pixel segment information of each valid segment in the canvas row. There are at most 15 valid segments in a row. For example, if there are 15 valid segments in a row, the 15 valid segments in the row are sorted according to the size of the horizontal offset to obtain a new sequence seg'

[15] (i.e., the target pixel segment information). 6) At time t+1, refer to Figure 6 Next, scan the valid segments of each row in the canvas in sequence, and send out the data requests of the valid segments in turn (due to the randomness of the coverage, refer to Figure 5When storing valid segments, there may be pixel segments with a length of 0 in the valid segment. At the same time, the requested information needs to be recorded. In view of the characteristics of the AXI bus, when requesting data, due to the specific requirements caused by the non-8 / 16-byte alignment of the address, information including head_offset (data header offset), tail_offset (data tail offset), data_len (data length) is set to assist in recording the data of the valid segment.

[0051] 7) According to the instructions of the previous request information, the invalid part of the first data indicated by head_offset and the invalid part of the last data indicated by tail_offset are deleted, and the remaining data are all stored in the data pool.

[0052] 8) Update the target rectangle image in the canvas, refer to Figure 2 As shown, each target rectangular image in the canvas is scanned from left to right and from top to bottom, the invalid part is filled with background data, and the valid part is extracted from the data pool and filled.

[0053] The embodiment of the present application also provides an electronic device, the electronic device includes a memory, a processor and a layer processor, the memory stores a computer program, and the processor implements the above-mentioned image display processing method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0054] See also Figure 7 , Figure 7 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes: The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement solutions other than the technical solutions provided in the embodiments of the present application; Memory 702, which may be NAND flash; A layer processor 703, in which relevant program codes are stored and the layer processor is used to call and execute the image display processing method of the embodiment of the present application; Input / output interface 704, used to implement information input and output; Communication interface 705, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.); A first bus 706 , which transmits information between various components of the device (e.g., the processor 701 , the memory 702 , the input / output interface 704 , and the communication interface 705 ); A second bus 707, the second bus 707 transmits information between the memory 702 and the layer processor 703; The processor 701 , the memory 702 , the input / output interface 704 and the communication interface 705 are connected to each other in communication within the device via the first bus 706 .

[0055] An embodiment of the present application also provides a computer-readable storage medium, which is a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned image display processing method is implemented.

[0056] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0057] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0058] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0059] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0060] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0061] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0062] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0063] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0064] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0065] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0066] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0067] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for processing image display, characterized in that: Applied to a layer processor; the method comprises: In response to an image position adjustment request triggered by a position change of at least one target rectangular image in a preset canvas, vertex coordinates of a plurality of target rectangular images displayed in the canvas are determined, wherein each of the target rectangular images corresponds to a different layer; and at least two of the plurality of target rectangular images at least partially overlap; Scan each of the target rectangular images respectively, and determine pixel segment information of uncovered pixel segments in each row of each of the target rectangular images according to the scanning results and the vertex coordinates; Sort the pixel segment information to obtain target pixel segment information; In response to the image update request of the canvas, requesting the target image data corresponding to the target pixel segment information from a preset memory; the memory and the layer processor are connected via a bus; According to the target image data, the uncovered pixels in each target rectangular image in the canvas are updated.

2. The image display processing method according to claim 1, characterized in that: The scanning of each of the target rectangular images respectively includes: Obtaining the display order of each of the target rectangular images; According to the display order, the image displayed at the bottom layer of each target rectangular image is used as the target traversal image; Starting from the first row of the target traversal image, traversing and scanning row by row along the column direction of the canvas; After the traversal scan of the target traversal image is completed, according to the display order, the target traversal image is updated to the target rectangular image of the previous layer and jumps to the first row of the target traversal image along the column direction of the canvas to start traversal scan row by row; When all the target rectangular images except the target rectangular image displayed on the outermost layer have been traversed and scanned, the scanning is stopped.

3. The image display processing method according to claim 2, characterized in that: The scanning result is a scanning result of each row of the target traversal image, and the pixel segment information of the uncovered pixel segment in each row of each target rectangular image is determined according to the scanning result and each vertex coordinate, including: According to the display order, the vertex coordinates of the target rectangular image in the layer covering the target traversal image are determined as the target vertex coordinates; In the process of traversing each row of pixels in the target traversal image, the coordinates of the traversed pixels are compared with the target vertex coordinates to determine the pixel segment information of the uncovered pixel segment in the target traversal image; When the target traversal image is a target rectangular image displayed on the second layer, each row in the target rectangular image on the top layer is used as an uncovered pixel segment, and the pixel segment information of the corresponding uncovered pixel segment is determined according to the vertex coordinates of the target rectangular image on the top layer.

4. The image display processing method according to claim 1, characterized in that: The requesting the target image data corresponding to the target pixel segment information from a preset memory includes: Acquire a bus protocol of a bus connected to the memory; According to the bus protocol, each of the target pixel segment information is encapsulated to obtain an image request message corresponding to each of the target pixel segment information; sending the image request messages to the memories respectively through the bus; The target image data corresponding to the image request information in a one-to-one manner and returned from the memory is received.

5. The image display processing method according to claim 4, characterized in that: The bus protocol is an AXI protocol; the receiving the target image data returned from the memory and corresponding to the image request information one by one includes: Obtaining the field alignment length determined by the AXI protocol; The target image data is read from the memory according to the field alignment length and the image request message.

6. The image display processing method according to claim 5, characterized in that: The image request message includes a data header offset and a data tail offset, and the reading of target image data from the memory according to the field alignment length and the image request message includes: receiving response data of the field alignment length from the memory; Determine the reading start position of the response data according to the data header offset; Starting from the starting position, data is extracted one by one from the response data until the position indicated by the data tail offset is reached to obtain the target image data.

7. The image display processing method according to claim 1, characterized in that: The pixel segment information is sorted to obtain target pixel segment information, including: Get the preset sorting method; According to the sorting method, the pixel segment information of each pixel segment located on the same straight line is sorted according to the coordinate size of the pixel segment to obtain the target pixel segment information corresponding to each row in the canvas.

8. A terminal device, characterized in that: The terminal device comprises: A layer processor, wherein the layer processor executes the method according to any one of claims 1 to 7; A memory, wherein the memory is connected to the layer processor via a preset bus communication.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the image display processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image display processing method according to any one of claims 1 to 7 is implemented.

Images