Image processing method, image processing device, and display system

Abstract

The application is suitable for the image technical field, and provides an image processing method, an image processing device and a display system. The image processing method comprises the following steps: storing pixel information of a target pixel point corresponding to a first input image in a plurality of input images, wherein the first input image is at the end of at least one of a first refreshing direction and a second refreshing direction, and a second input image to be spliced and output exists; the target pixel point comprises a boundary pixel point spliced and output with the first input image in at least one of the first refreshing direction and the second refreshing direction; based on the pixel information of the target pixel point, each first input image is spliced and processed with the corresponding target pixel point to obtain an expanded image, and each expanded image and a third input image in the plurality of input images are spliced and scaled to obtain a target image. The embodiment of the application can reduce the occupation of storage resources.

Description

Technical Field

[0001] This application belongs to the field of image technology, and in particular relates to an image processing method, an image processing device, and a display system. Background Technology

[0002] Light-emitting diode (LED) displays are increasingly used in product launches, concerts, and outdoor building advertising due to their ability to be freely assembled and spliced ​​according to usage scenarios and space constraints. To drive wide LED displays to play various produced programs, these applications often employ a screen-within-a-screen solution consisting of a multimedia server, splicing control equipment (image processing equipment), and LED sending cards. The image processing equipment often needs to amplify multiple input video sources to adapt to the large LED display. To avoid splicing gaps on the screen, related technologies store multiple video sources in memory and splice them into a complete image. Then, the complete image is scaled according to display requirements before being spliced ​​and output. However, as LED display areas increase and image resolutions reach 4K or even higher, reading and writing complete images requires significant storage resources, leading to increased memory costs. Summary of the Invention

[0003] This application provides an image processing method, an image processing device, and a display system that can reduce the occupation of storage resources and thus reduce memory costs.

[0004] A first aspect of this application provides an image processing method, comprising: receiving multiple input images, the multiple input images being used for stitching and output; storing pixel information of a target pixel corresponding to a first input image among the multiple input images, wherein the first input image has a second input image to be stitched and output at its end in at least one of a first refresh direction and a second refresh direction, and the target pixel belongs to the second input image; the target pixel includes a boundary pixel that is stitched and output with the first input image in at least one of the first refresh direction and the second refresh direction; stitching each first input image with its corresponding target pixel based on the pixel information of the target pixel to obtain an extended image; stitching and scaling each extended image and a third input image among the multiple input images to obtain a target image, wherein the third input image does not have an image to be stitched and displayed in either the first refresh direction or the second refresh direction.

[0005] In some embodiments of the first aspect, before storing the pixel information of the target pixel corresponding to the first input image among the plurality of input images, the image processing method further includes: allocating a corresponding storage area for each first input image, wherein the storage area corresponding to each first input image is used to store the pixel information of the target pixel corresponding to the first input image.

[0006] In some embodiments of the first aspect, based on the pixel information of the target pixel, each first input image is spliced ​​with the corresponding target pixel to obtain an extended image, including: when a whole row of pixels of the first input image is received in the row direction, splicing the row of pixels with the target pixel in the same row of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixel; and / or, when a whole column of pixels of the first input image is received in the column direction, splicing the column of pixels with the target pixel in the same column of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixel.

[0007] In some embodiments of the first aspect, the first refresh direction is a refresh direction in the row direction, and the second refresh direction is a refresh direction in the column direction; before storing the pixel information of the target pixel corresponding to the first input image among the multiple input images, the image processing method further includes: determining the total number of rows N and / or the total number of columns M of the target pixel in the first input image based on the number of neighboring pixels required for the scaling process in the stitching and scaling process, where N and M are both positive integers; and taking the pixels in the first N rows and / or the first M columns of the second input image corresponding to the first input image as the target pixel corresponding to the first input image.

[0008] In some embodiments of the first aspect, if the first input image has a corresponding second input image in both the first refresh direction and the second refresh direction, then the target pixel corresponding to the first input image includes a first pixel and a second pixel; the first pixel is the pixel in the first N rows and / or the first M columns of the second input image corresponding to the first input image; the second pixel is the pixel region in the first N rows × the first M columns of the input image diagonally adjacent to the first input image.

[0009] In some embodiments of the first aspect, in the extended image, the target pixels stitched to the first input image are used for interpolation to obtain interpolated pixels around the boundary line between the first input image and the corresponding second input image.

[0010] In some embodiments of the first aspect, the step of stitching and scaling each of the extended images and the third input image among the plurality of input images to obtain a target image includes: scaling the extended images and the third input image respectively to obtain scaled images; and stitching the scaled images to obtain the target image.

[0011] In some embodiments of the first aspect, after stitching and scaling the extended images and the third input image among the plurality of input images to obtain the target image, the image processing method further includes: outputting the image to be output from each video output interface of the target image through the corresponding video output interface.

[0012] In some embodiments of the first aspect, the third input image is a second input image that corresponds to at least one of the first input images among a plurality of input images.

[0013] A second aspect of this application provides an image processing apparatus, comprising: a receiving unit for receiving multiple input images, the multiple input images being stitched together for output; a storage unit for storing pixel information of a target pixel corresponding to a first input image among the multiple input images, wherein the first input image has a second input image to be stitched together at its end in at least one of a first refresh direction and a second refresh direction, and the target pixel belongs to the second input image; the target pixel includes a boundary pixel that is stitched together with the first input image in at least one of the first refresh direction and the second refresh direction; an expanding unit for stitching each first input image with a corresponding target pixel based on the pixel information of the target pixel to obtain an expanded image; and an image processing unit for stitching and scaling each of the expanded images and a third input image among the multiple input images to obtain a target image, wherein the third input image does not have an image to be stitched together in either the first refresh direction or the second refresh direction.

[0014] A third aspect of this application provides an image processing apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the image processing method as described in any of the first aspects.

[0015] A fourth aspect of this application provides a display system, comprising: a multimedia server connected to an image processing device for inputting an image to the image processing device; the image processing device connected to a display screen for executing the steps of the image processing method as described in any one of the first aspects and outputting an image to the display screen; and the display screen for displaying the image received from the image processing device.

[0016] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described image processing method.

[0017] A sixth aspect of this application provides a computer program product that, when run, causes the image processing method described above to be executed.

[0018] In the embodiments of this application, multiple input images for splicing output are received. For a first input image among the multiple input images, pixel information of the target pixels corresponding to the first input image is stored. The first input image contains a second input image to be spliced ​​out in at least one of the first and second refresh directions. The target pixels include boundary pixels within the second input image that are spliced ​​out with the first input image in at least one of the first and second refresh directions. Subsequently, based on the pixel information of the target pixels, each first input image is spliced ​​with its corresponding target pixels to obtain an expanded image. The expanded images and a third input image from the multiple input images that does not contain an image to be spliced ​​out in either the first or second refresh direction are then spliced ​​together. By performing stitching and scaling processes, the target image is obtained. Thus, on the one hand, the target pixels on the boundary of the corresponding second input image can be stitched onto the first input image, so that the target pixels can be used to avoid stitching gaps during stitching and scaling. On the other hand, since the stored pixel information is the pixel information of the target pixels corresponding to the first input image, and the first input image is an image of the second input image to be stitched and output from multiple input images in at least one of the first refresh direction and the second refresh direction, it is not necessary to store pixel information for each image, nor is it necessary to cache the entire image, thus reducing storage requirements. Therefore, image processing can be implemented using the internal storage of the image processing device, eliminating the need for external storage, thereby reducing storage costs, hardware complexity, and latency caused by data reading and writing in external storage. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of the image processing method provided in the embodiments of this application;

[0021] Figure 2 This is a first schematic diagram of the extended images provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of image interpolation provided in an embodiment of this application;

[0023] Figure 4 This is a second schematic diagram of the extended image provided in the embodiments of this application;

[0024] Figure 5 This is a third schematic diagram of the extended images provided in the embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the structure of a display system provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0030] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] LED displays are increasingly used in product launches, concerts, and outdoor building advertising due to their ability to be freely assembled and spliced ​​according to usage scenarios and space constraints. To drive wide LED displays to play various produced programs, these applications often employ a screen-within-a-screen solution consisting of a multimedia server, splicing control equipment (image processing equipment), and LED sending cards. The image processing equipment often needs to amplify multiple input video sources to adapt to the large LED display. To avoid splicing gaps on the screen, related technologies store multiple video sources in memory and splice them into a complete image. Then, the complete image is scaled according to display requirements before being spliced ​​and output. However, as LED display areas become larger, image resolutions can reach 4K or even higher, reading and writing complete images requires significant storage resources, leading to increased memory costs.

[0033] In view of this, this application proposes an image processing method that reduces storage requirements while avoiding the appearance of stitching gaps, thus helping to reduce storage costs.

[0034] To illustrate the technical solution of this application, specific embodiments are described below.

[0035] Figure 1 The illustration shows a schematic diagram of the implementation flow of an image processing method provided in an embodiment of this application, which can be applied to an image processing device.

[0036] Specifically, the above image processing method may include the following steps S101 to S104.

[0037] Step S101: Receive multiple input images.

[0038] The multiple input images are used to stitch together the output images, forming a complete image for display on a screen. The image processing device can receive these multiple input images from one or more video input interfaces; this application does not impose any restrictions on this.

[0039] Step S102: For the first input image among multiple input images, store the pixel information corresponding to the target pixel point of the first input image.

[0040] In this design, the first input image has a second input image to be stitched together at the end of at least one of the first and second refresh directions. The first and second refresh directions refer to the refresh directions of the input image in the row and column directions, respectively, and the refresh direction is the processing order of pixels during image processing. In other words, the first input image has another input image to be stitched together at the end of its refresh direction; this second input image is the second input image corresponding to the first input image.

[0041] For example, please refer to Figure 2 If the first refresh direction is from left to right and the second refresh direction is from top to bottom, then... Figure 2 In the given scenario, input image a has input image b to its right and input image c below it, therefore input image a is the first input image, and the corresponding second input images are input image b and input image c; input image b has input image d below it, so input image b is the first input image, and the corresponding second input image is input image d; input image c has input image d to its right, so input image c is the first input image, and the corresponding second input image is input image d; input image d has no input images to its right or below it, so input image d does not belong to the first input image.

[0042] For the first input image, the pixel information of the corresponding target pixels needs to be stored. These target pixels belong to the second input image and include at least the boundary pixels that are stitched together with the first input image in at least one of the first and second refresh directions. Boundary pixels are at least one row and / or at least one column of pixels that connect the second input image to the image boundary. In other words, the target pixels are the pixels within the second input image that need to be stitched together with the corresponding first input image. These target pixels are located at the beginning of either the first or second refresh direction of the second input image.

[0043] For example, Figure 2 The shaded area indicates the image region containing the target pixels that need to be stored. For input image b, at least the pixel information of the first column of pixels needs to be stored; for input image c, at least the pixel information of the first row of pixels needs to be stored; for input image d, at least the pixel information of the first row and the first column of pixels needs to be stored.

[0044] The aforementioned pixel information can refer to pixel values, which reflect the content of the target pixel.

[0045] Step S103: Based on the pixel information of the target pixel, each first input image is concatenated with the corresponding target pixel to obtain an extended image.

[0046] Since the second input image is located at the end of the refresh direction of the first input image, it needs to be concatenated with the first input image for output. For each first input image, the pixel information of the target pixels stored in the corresponding second input image can be used for concatenation with the first input image. By concatenating each first input image with its corresponding target pixels, an expanded image can be obtained.

[0047] It should be noted that if a second input image exists at the end of both the first and second refresh directions, the first input image can be stitched together with the target pixels in each refresh direction to obtain an extended image.

[0048] For example Figure 2 In the input images b, c, and d, the target pixels are stitched together along the arrow direction: the first column of target pixels in input image b and the first row of target pixels in input image c can be stitched together to the last column and last row of pixels in input image a, respectively, to form an extended image a'; the first row of target pixels in input image d can be stitched together to the last row of pixels in input image b, to form an extended image b'; and the first column of target pixels in input image d can be stitched together to the last column of pixels in input image c, to form an extended image c'.

[0049] The target pixels in the extended image can be used to interpolate pixels around the boundaries between the input images, thereby avoiding stitching gaps.

[0050] Step S104: The extended images and the third input image from the multiple input images are stitched together and scaled to obtain the target image.

[0051] In this case, the third input image does not contain any images to be stitched together in either the first or second refresh direction. For example... Figure 2 The input image d in the image.

[0052] Among the multiple input images, the third input image does not have any images to be stitched at the ends of either the first or second refresh direction. Therefore, the third input image does not undergo the stitching process in step S103. The remaining images, however, have images to be stitched at the ends of at least one of the first and second refresh directions, and thus all obtain extended images based on the stitching process in step S103. By stitching and scaling each extended image and the third input image, a target image can be obtained, which can be scaled based on the image content of the extended images and the third input image. The target image is the complete image that needs to be output to the display screen.

[0053] In the embodiments of this application, multiple input images for splicing output are received. For a first input image among the multiple input images, pixel information of the target pixels corresponding to the first input image is stored. The first input image contains a second input image to be spliced ​​out in at least one of the first and second refresh directions. The target pixels include boundary pixels within the second input image that are spliced ​​out with the first input image in at least one of the first and second refresh directions. Subsequently, based on the pixel information of the target pixels, each first input image is spliced ​​with its corresponding target pixels to obtain an expanded image. The expanded images and a third input image from the multiple input images that does not contain an image to be spliced ​​out in either the first or second refresh direction are then spliced ​​together. By performing stitching and scaling processes, the target image is obtained. Thus, on the one hand, the target pixels on the boundary of the corresponding second input image can be stitched onto the first input image, so that the target pixels can be used to avoid stitching gaps during stitching and scaling. On the other hand, since the stored pixel information is the pixel information of the target pixels corresponding to the first input image, and the first input image is an image of the second input image to be stitched and output from multiple input images in at least one of the first refresh direction and the second refresh direction, it is not necessary to store pixel information for each image, nor is it necessary to cache the entire image, thus reducing storage requirements. Therefore, image processing can be implemented using the internal storage of the image processing device, eliminating the need for external storage, thereby reducing storage costs, hardware complexity, and latency caused by data reading and writing in external storage.

[0054] Furthermore, compared to storing the pixel information of adjacent input images for all input images, since the target pixel is located at the beginning of the first refresh direction and / or the second refresh direction, it is not necessary to store the pixel information of the pixels at the end of the refresh direction (e.g., the last row or the last column). Therefore, it is not necessary to wait for the entire image to be completely refreshed, which helps to improve image processing efficiency and reduce display latency.

[0055] In some embodiments of this application, receiving multiple input images may include receiving multiple input images from a multimedia server.

[0056] Specifically, the multimedia server can obtain the complete image required for display on the screen from the video source, or it can generate the complete image required for display on the screen independently. Then, it segments the complete image required for display on the screen into multiple input images, which are then input to the image processing device via the input interface. In this way, image segmentation avoids the situation where the resolution of the complete image is too large to be transmitted via the input interface.

[0057] It should be noted that the multimedia server can be connected to one or more input interfaces of the image processing device. The splicing device can receive multiple input images from multiple input interfaces separately, or it can receive multiple input images from a single input interface; this application does not impose any restrictions on this.

[0058] In some embodiments of this application, before storing the pixel information of the corresponding target pixels of a first input image among multiple input images, the image processing method may further include: allocating a corresponding storage area for each first input image, wherein the storage area corresponding to each first input image is used to store the pixel information of the target pixels corresponding to the first input image. Thus, when performing stitching and scaling processing on each first input image, the pixel information can be read from the corresponding storage area.

[0059] It should be noted that for a single first input image, the number of allocated storage regions matches the number of directions of the target pixels to be stitched together, and different storage regions correspond to different input images. For example, Figure 2 In this context, corresponding storage areas can be allocated for input images a, b, and c respectively. Input image a can be allocated two storage areas, one of which is used to store the pixel information of the target pixel from input image b, and the other of which is used to store the pixel information of the target pixel from input image c.

[0060] Given that the method provided in this application can reduce the occupation of storage resources, the image processing device does not need to be mounted with external storage, but implements the image processing method of this application through memory. Therefore, the above-mentioned storage area can be the storage area of ​​the memory of the image processing device.

[0061] Thus, when storing the pixel information of the target pixel corresponding to the first input image, the pixel information can be stored in the storage area corresponding to the first input image, so that the pixel information of the target pixel can be obtained in the corresponding storage area when the first input image is pixel stitched.

[0062] In some embodiments of this application, the first refresh direction is the refresh direction in the row direction, and the second refresh direction is the refresh direction in the column direction. Before storing the pixel information of the target pixels corresponding to the first input image among multiple input images, the image processing method may further include: determining the total number of rows N and / or the total number of columns M of the target pixels in the second input image based on the number of neighboring pixels required for the scaling process in the stitching and scaling process; and taking the pixels of the first N rows and / or the first M columns in the second input image corresponding to the first input image as the target pixels corresponding to the first input image.

[0063] Where N and M are both positive integers.

[0064] Specifically, the scaling process typically requires interpolation algorithms. Since the positions of pixels in the scaled image do not completely overlap with those in the original image, it's necessary to estimate the pixel information of the new position based on the known pixel information of the original image. The pixel information of each new position can be obtained based on the pixel information of several surrounding pixels in the original image. The number of these surrounding pixels represents the number of neighboring pixels required for the scaling process.

[0065] For example Figure 3 In this process, two horizontally aligned input images with a resolution of 1024 are enlarged to a resolution of 3840. Each pixel requires four neighboring pixels for scaling. Specifically, to interpolate pixels 1919 and 1920 (represented by circles), four pixels (1022 to 1025) from the square in the input image are needed. The number of neighboring pixels can be determined using the interpolation algorithm.

[0066] Based on the number of neighboring pixels, the total number of rows N and / or the total number of columns M of the target pixels in the second input image corresponding to the first input image can be determined.

[0067] The total number of rows N and / or the total number of columns M can be greater than or equal to half the number of neighboring pixels. In other words, the total number of rows N and / or the total number of columns M is greater than or equal to the interpolation kernel radius, which refers to the radius of the pixel range sampled from the input image when the interpolation algorithm calculates the interpolated pixels. Specifically, when a second input image to be stitched exists at the end of the first input image in the first refresh direction, the total number of columns M of the target pixels in the second input image can be determined based on the number of neighboring pixels required for the scaling process in the stitching and scaling process; when a second input image to be stitched exists at the end of the first input image in the second refresh direction, the total number of rows N of the target pixels in the second input image can be determined based on the number of neighboring pixels required for the scaling process in the stitching and scaling process.

[0068] In some embodiments of this application, the values ​​of N and M can be 0-1, where 0 is the number of neighboring pixels.

[0069] by Figure 3 For example, if the number of neighboring pixels is 4, then pixels 1024 to 1026 shown in the square can be used as target pixels. Thus, in the subsequent step S103, pixels 1024 to 1026 shown in the square can be concatenated after pixel 1023 shown in the square. At this time, pixels 1022 to 1026 can be scaled to obtain pixels 1919 to 1922 shown in the circle.

[0070] Therefore, during the scaling process, the first N rows and / or the first M columns of pixels of the second input image to be stitched can be stored in the storage area corresponding to the first input image as target pixels. These target pixels can be used in conjunction with the first input image during the scaling process to interpolate the pixels in the boundary area between the second input image and the first input image, thereby avoiding the problem of stitching gaps that occur when the boundary area only refers to a single input image during scaling.

[0071] In other embodiments of this application, the total number of rows N and / or the total number of columns M can be determined based on scene information, between the interpolation kernel radius and the number of neighboring pixels.

[0072] The scene information may include, but is not limited to, the display resolution of the target image on the screen, the content type in the input image, and the motion state. In some implementations, the total number of rows N and / or the total number of columns M can be obtained by weighting different scene information.

[0073] Specifically, the higher the display resolution of the target image, the more obvious the seams at the edges of the stitching. In this case, the values ​​of the total number of rows N and / or the total number of columns M should be closer to the number of neighboring pixels to avoid poor display quality. When the content type of the input image is static foreground, the values ​​of the total number of rows N and / or the total number of columns M should be closer to the interpolation kernel radius. When the content type of the input image is dynamic foreground, and the dynamic foreground is in motion, the values ​​of the total number of rows N and / or the total number of columns M should be closer to the number of neighboring pixels to avoid increasing artifacts in the moving foreground.

[0074] In some embodiments of this application, each first input image is spliced ​​with the corresponding target pixel based on the pixel information of the target pixel to obtain an extended image. This may include: when a whole row of pixels of the first input image is received in the row direction, the row of pixels is spliced ​​with the target pixels in the same row of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixel.

[0075] Specifically, the first refresh direction is the refresh direction in the row direction. When there is a second input image to be stitched and output at the end of the first input image in the first refresh direction, the image processing device can receive the pixels of the first input image in the first refresh direction. When the entire row of pixels of the first input image is received, the device can stitch the row of pixels with the target pixels in the corresponding row of the second input image based on the pixel information of the corresponding target pixels to obtain the entire row of pixels of the extended image.

[0076] For example, Figure 2 When the first row of pixels of input image a is received, the target pixels (the first M pixels) of the first row of input image b can be concatenated to the end of the first row of input image a to obtain the first row of pixels of input image a'.

[0077] When stitching together a row of pixels from the first input image, the pixel information of the target pixels in that row can be cleared from the storage area corresponding to the first input image to free up storage space and further reduce storage resource usage. Alternatively, the pixel information can be left uncleared, and when the next frame of the input image arrives, the pixel information of the target pixels in the same row can overwrite the already stored pixel information in the storage area.

[0078] In particular, each input image can be refreshed synchronously. When the entire row of pixels of the first input image is received, the first row of pixels of the corresponding second input image has already been received and stored in the storage area corresponding to the first input image. Therefore, the stitching process does not need to wait.

[0079] Similarly, in some embodiments of this application, the process of stitching each first input image with the corresponding target pixel based on the pixel information of the target pixel to obtain an extended image may include: when receiving a whole column of pixels of the first input image in the column direction, stitching the column of pixels with the target pixels in the same column of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixel.

[0080] Specifically, the second refresh direction is the refresh direction in the column direction. When there is a second input image to be stitched and output at the end of the first input image in the second refresh direction, the image processing device can receive the pixels of the first input image in accordance with the second refresh direction. When the entire column of pixels of the first input image is received, the device can stitch the row of pixels with the corresponding target pixels in the same column of the second input image based on the pixel information of the target pixels to obtain the entire row of pixels of the extended image.

[0081] For example, Figure 2When the first column of pixels of input image a is received, the target pixels (the first N pixels) of the first column of input image c can be concatenated to the end of the first column of input image a to obtain the first column of pixels of input image a'.

[0082] Similarly, each input image can be refreshed synchronously. When the entire column of pixels of the first input image is received, the first column of pixels of the corresponding second input image has already been received and stored in the storage area corresponding to the first input image. Therefore, the stitching process does not need to wait.

[0083] It should be noted that if a corresponding second input image exists in both the first and second refresh directions for the first input image, the target pixel point corresponding to the first input image can include both the first pixel point and the second pixel point. The first pixel point is the pixel point in the first N rows and / or the first M columns of the second input image corresponding to the first input image. The second pixel point is the pixel region in the first N rows × the first M columns of the input image diagonally adjacent to the first input image.

[0084] Specifically, the first pixel includes the first M columns of pixels in the second input image to be stitched together at the end of the first input image in the first refresh direction, and the first N rows of pixels in the second input image to be stitched together at the end of the first input image in the second refresh direction. The second pixel is the pixel region of the first N rows × the first M columns located at the intersection of the ends of the first and second refresh directions of the first input image.

[0085] Here, the second pixel can be obtained from the storage area corresponding to the second input image. For example, please refer to [reference needed]. Figure 2 and Figure 4 Since input image a has corresponding first input images in both the first and second refresh directions, input image a needs to be stitched together with the first M columns of pixels from input image b and the first N rows of pixels from input image c, and also needs to be stitched together with the pixels from (0,0) to (M-1, N-1) of the diagonally adjacent input image d (shown in the black area). The pixels from (0,0) to (M-1, N-1) of input image d can be obtained from the storage area corresponding to input image c (i.e., the second input image corresponding to input image a) or the storage area corresponding to input image b (i.e., the second input image corresponding to input image a). Along... Figure 4 As indicated by the arrow, pixels from (0, 0) to (M-1, N-1) in the input image d can be concatenated below the first M columns of pixels in the input image b and to the right of the first N rows of pixels in the input image c. Therefore, based on the pixel information of the target pixels, concatenating the first input image with the corresponding target pixels yields extended images a' to c'.

[0086] In some embodiments of this application, the target image is obtained by stitching and scaling the extended images and the third input image among the multiple input images. This may include: scaling the extended images and the third input image respectively to obtain scaled images, and stitching the scaled images together to obtain the target image.

[0087] Specifically, in the extended image, the target pixels stitched to the first input image can be used for interpolation to obtain interpolated pixels around the boundary line between the first input image and the corresponding second input image.

[0088] The regions of the first input image and the third input image in the expanded image can be used for interpolation to obtain the pixels inside the first input image and the third input image, respectively, and then the scaled image is obtained.

[0089] For example, Figure 3 In the diagram, pixels 1 to 1023, shown in the square, are pixels in the extended image a' that originally belonged to the input image a. These pixels can be interpolated to obtain pixels 0 to 1913, shown in the circle. Pixels 1024 to 1026, shown in the square, are pixels in the extended image a' that originally belonged to the input image b. These pixels can be interpolated with pixels 1021 to 1023, shown in the square, to obtain pixels 1914 to 1924, shown in the circle. Pixels 1024 to 2047, shown in the square, are pixels in the extended image b' that originally belonged to the input image b. These pixels can be interpolated to obtain pixels 1925 to 3839, shown in the circle.

[0090] Subsequently, the scaled images are stitched together according to their positions to obtain the target image.

[0091] Therefore, instead of scaling the entire stitched image, the extended image and the third input image are processed separately, improving scaling efficiency. Furthermore, when stitching a row / column of pixels from the first input image, interpolation can be directly applied to that row / column in the extended image to achieve scaling, thereby improving image processing efficiency.

[0092] In some embodiments of this application, after stitching and scaling the extended images and the third input image from the multiple input images to obtain the target image, the image processing method may further include: outputting the image to be output from each video output interface of the target image through the corresponding video output interface.

[0093] Specifically, the display screen can be composed of multiple cabinets / display modules spliced ​​together. These cabinets / display modules can be connected to multiple video output interfaces of the image processing equipment. The target image can contain the image to be output from each video output interface. The image to be output from each video output interface is output through the corresponding video output interface, so that the image to be output from each video output interface can be used for display in the cabinet / display module connected to the corresponding video output interface, thereby enabling the various cabinets / display modules to splice together and display the target image. The correspondence between the image to be output and the video output interface can be obtained through pre-configuration. In particular, when the resolution of the image to be output is smaller than that of the cabinet / display module connected to the corresponding video output interface, the image to be output can be arranged and displayed on the cabinet / display module according to the display position within the cabinet / display module.

[0094] Furthermore, after interpolating a row / column of pixels in the extended image or the third input image as described above, the same row / column of pixels in the extended image or the third input image can be directly stitched together to obtain a row / column of pixels in the target image. Then, based on the video output interface to which that row / column of pixels belongs, it can be output through the corresponding video output interface. In this way, there is no need to wait for the entire image stitching process to be completed before outputting; output can be done row by row or column by column, improving display refresh efficiency.

[0095] It should be noted that among the above multiple input images, there is a second input image corresponding to at least one first input image that is also a third input image.

[0096] Figure 4 This illustrates the case where there are four input images. In this case, input images b and c are the second input images corresponding to input image a, but not the third input image; while input image d is the second input image corresponding to input images b and c, and is also the third input image.

[0097] Figure 5 The example illustrates a scenario where there are two input images. In this case, input image a has an input image b to be stitched together at its end along the first refresh direction. Input image a is the first input image, and input image b is the second input image corresponding to the first input image. However, input image b has no images to be stitched together in either the first or second refresh direction; therefore, input image b is the third input image. In other words, input image b is both the second and third input image corresponding to input image a.

[0098] Therefore, it can be seen that among multiple input images, there exists at least one first input image, and the corresponding second input image is the third input image.

[0099] like Figure 6The diagram shown is a structural schematic of an image processing device 600 provided in an embodiment of this application. The image processing device 600 is disposed on an image processing device.

[0100] Specifically, the image processing device 600 may include:

[0101] The receiving unit 601 is used to receive multiple input images, which are then stitched together for output.

[0102] Storage unit 602 is used to store pixel information of target pixels corresponding to a first input image among the plurality of input images, wherein the first input image has a second input image to be stitched and output at the end of at least one of a first refresh direction and a second refresh direction, and the target pixels belong to the second input image; the target pixels include boundary pixels that are stitched and output with the first input image in at least one of the first refresh direction and the second refresh direction.

[0103] The extension unit 603 is used to concatenate each of the first input images with the corresponding target pixels based on the pixel information of the target pixels to obtain an extended image;

[0104] Image processing unit 604 is used to stitch and scale the extended images and the third input image among the multiple input images to obtain a target image, wherein the third input image does not contain any images to be stitched and displayed in either the first refresh direction or the second refresh direction.

[0105] In some embodiments of this application, the image processing apparatus 600 may further include a storage allocation unit, configured to: allocate a corresponding storage area for each of the first input images before storing the pixel information of the target pixel corresponding to the first input image among the plurality of input images, wherein the storage area corresponding to each of the first input images is used to store the pixel information of the target pixel corresponding to the first input image.

[0106] In some embodiments of this application, the extension unit 603 is specifically used to: when receiving a whole row of pixels of the first input image in the row direction, splice the row of pixels with the target pixels in the same row of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixels.

[0107] In some embodiments of this application, the extension unit 603 is specifically used to: when receiving a whole column of pixels of the first input image in the column direction, splice the column of pixels and the target pixels in the same column of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixels.

[0108] In some embodiments of this application, the first refresh direction is a refresh direction in the row direction, and the second refresh direction is a refresh direction in the column direction; the image processing apparatus 600 may further include a pixel point determination unit, configured to: before storing the pixel information of the target pixel point corresponding to the first input image for the first input image among the plurality of input images, determine the total number of rows N and / or the total number of columns M of the target pixel point in the first input image based on the number of neighboring pixels required for the scaling process in the stitching and scaling process, where N and M are both positive integers; and take the pixels of the first N rows and / or the first M columns in the second input image corresponding to the first input image as the target pixel point corresponding to the first input image.

[0109] In some embodiments of this application, if the first input image has a corresponding second input image in both the first refresh direction and the second refresh direction, then the target pixel corresponding to the first input image includes a first pixel and a second pixel; the first pixel is the pixel in the first N rows and / or the first M columns of the second input image corresponding to the first input image; the second pixel is the pixel region in the first N rows × the first M columns of the input image diagonally adjacent to the first input image.

[0110] In some embodiments of this application, in the extended image, the target pixel points spliced ​​to the first input image are used for interpolation to obtain interpolated pixel points around the boundary line between the first input image and the corresponding second input image.

[0111] In some embodiments of this application, the image processing unit 604 is specifically used to: scale the extended image and the third input image respectively to obtain scaled images; and to stitch the scaled images together to obtain the target image.

[0112] In some embodiments of this application, the image processing apparatus 600 may further include an output unit for: after stitching and scaling each of the extended images and the third input image among the plurality of input images to obtain a target image, outputting the image to be output from each video output interface of the target image through the corresponding video output interface.

[0113] In some embodiments of this application, among a plurality of input images, there exists a second input image corresponding to at least one of the first input images that is the third input image.

[0114] It should be noted that, for the sake of convenience and brevity, the above... Figure 6 For the specific working process of the device shown, please refer to... Figures 1 to 5 The corresponding process of the system will not be described in detail here.

[0115] like Figure 7 The diagram shown is a schematic representation of an image processing device according to an embodiment of this application. Specifically, the image processing device 7 may include: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as an image processing program. When the processor 70 executes the computer program 72, it implements the steps described in the various image processing method embodiments above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of the receiving unit 601, storage unit 602, expansion unit 603, and image processing unit 604 shown are illustrated.

[0116] The computer program can be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the image processing device.

[0117] The image processing device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of an image processing device and does not constitute a limitation on the image processing device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the image processing device may also include input / output devices, network access devices, buses, etc.

[0118] The processor 70 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0119] The memory 71 can be an internal storage unit of the image processing device, such as a hard drive or RAM. The memory 71 can also be an external storage device of the image processing device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 71 can include both internal and external storage units of the image processing device. The memory 71 is used to store the computer program and other programs and data required by the image processing device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0120] It should be noted that, for the sake of convenience and brevity, the structure of the above-mentioned image processing device can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0121] like Figure 8 The diagram shown is a schematic of a display system provided in an embodiment of this application.

[0122] The display system 8 may include:

[0123] Multimedia server 81 is connected to image processing device 82 and is used to input images to image processing device 82;

[0124] Image processing device 82, connected to display screen 83, is used to perform tasks such as... Figures 1 to 5 The image processing method includes the steps of outputting an image to the display screen 83.

[0125] Display screen 83 is used to display images received from the image processing device.

[0126] The multimedia server 81 can be specifically used for creating and generating input images, and can be a computer, smartphone, or other device. The image processing device 82 can be specifically used for switching and processing the images provided by the multimedia server 81, and can be a switcher, splicing control device, or other device. Switching refers to processing an input image from a specific video input interface and then outputting it from a specific video output interface. A display control device may also be included between the image processing device 82 and the display screen 83. The display control device may include, but is not limited to, a sending card, a receiving card, or a scanning card, used to convert the images output by the image processing device 82 and drive the display screen 83 for display.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0128] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0129] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0130] In the embodiments provided in this application, it should be understood that the disclosed apparatus / system can be implemented in other ways. For example, the apparatus / system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between apparatuses or units may be electrical, mechanical, or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0133] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: receiving a plurality of input images, the plurality of input images being used for splicing output; storing pixel information of a target pixel point corresponding to a first input image in the plurality of input images, wherein the first input image has a second input image to be spliced and output at the end of at least one of a first refresh direction and a second refresh direction, and the target pixel point belongs to the second input image; the target pixel point includes a boundary pixel point spliced and output with the first input image in at least one of the first refresh direction and the second refresh direction; based on the pixel information of the target pixel point, splicing and processing each first input image and the corresponding target pixel point to obtain an expanded image; splicing and scaling each expanded image and a third input image in the plurality of input images to obtain a target image, wherein the third input image does not have an image to be spliced and displayed in the first refresh direction and the second refresh direction.

2. The image processing method of claim 1, wherein, Before the step of storing the pixel information of the target pixel point corresponding to the first input image in the plurality of input images, the image processing method further comprises: allocating a corresponding storage area for each first input image, and the storage area corresponding to each first input image is used to store the pixel information of the target pixel point corresponding to the first input image.

3. The image processing method of claim 2, wherein, The step of splicing and processing each first input image and the corresponding target pixel point based on the pixel information of the target pixel point to obtain an expanded image comprises: when receiving an entire row of pixel points of the first input image in a row direction, splicing and processing the row of pixel points and the target pixel point in the same row of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixel point; and / or, when receiving an entire column of pixel points of the first input image in a column direction, splicing and processing the column of pixel points and the target pixel point in the same column of the second input image corresponding to the first input image based on the pixel information of the corresponding target pixel point.

4. The image processing method of claim 1, wherein, The first refresh direction is a refresh direction in a row direction, and the second refresh direction is a refresh direction in a column direction. Before the step of storing the pixel information of the target pixel point corresponding to the first input image in the plurality of input images, the image processing method further comprises: determining the total number of rows N and / or the total number of columns M of the target pixel point in the second input image based on the number of neighborhood pixel points required for scaling in the splicing and scaling processing, wherein N and M are positive integers; regarding the first N rows and / or the first M columns of pixel points in the second input image corresponding to the first input image as the target pixel point corresponding to the first input image.

5. The image processing method of claim 4, wherein, If the first input image has a corresponding second input image in the first refresh direction and the second refresh direction, the target pixel point corresponding to the first input image includes a first pixel point and a second pixel point. The first pixel points are pixel points in the first N rows and / or the first M columns of the second input image corresponding to the first input image; and the second pixel points are pixel points in a pixel region of the first N rows and the first M columns in an input image diagonally adjacent to the first input image.

6. The image processing method of any one of claims 1-5, wherein, In the extended image, the target pixel points spliced with the first input image are used to interpolate to obtain interpolation pixel points around a boundary line between the first input image and the corresponding second input image.

7. The image processing method of any one of claims 1-5, wherein, The method further comprises: scaling the extended image and the third input image to obtain scaled images; splicing the scaled images to obtain the target image.

8. The image processing method of any one of claims 1-5, wherein, After the splicing and scaling of the extended image and the third input image, the image processing method further comprises: outputting the image to be output by each video output interface in the target image through the corresponding video output interface.

9. The image processing method of any one of claims 1-5, wherein, The second input image corresponding to the at least one first input image in the plurality of input images is the third input image.

10. An image processing apparatus characterized by comprising: The computer program is stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the image processing method according to any one of claims 1 to 9.

11. A display system characterized by, The method further comprises: a multimedia server connected with the image processing device, configured to input images to the image processing device; the image processing device connected with a display screen, configured to execute the steps of the image processing method according to any one of claims 1 to 9, and output images to the display screen; the display screen configured to display the images received from the image processing device.

Images