Image processing method and device, head-up display and storage medium

Abstract

The application discloses an image processing method, and relates to the fields of auxiliary driving and image processing. The method comprises the following steps: for each sub-block image, acquiring the reference feature pixel point quantity, the first feature pixel point and the second feature pixel point of the to-be-fused boundary contour of the sub-block image; acquiring the coordinate data of the reference feature pixel point of the to-be-fused boundary contour according to the above parameters; determining the fusion boundary of the sub-block image according to the coordinate data corresponding to the first feature pixel point, the second feature pixel point and the reference feature pixel point; and performing image fusion on each sub-block image according to the position relationship between each fusion boundary and each sub-block image. The technical scheme of the embodiment of the application not only realizes the accurate construction of the fusion boundary conforming to the actual contour of the sub-block image, but also avoids the problems of obvious splitting, ghosting or display fault at the splicing position of the sub-block image, thereby improving the content continuity, integrity and clarity of the fused image.

Description

Technical Field

[0001] This invention relates to the fields of driver assistance technology and image processing, and more particularly to an image processing method, apparatus, head-up display, and storage medium. Background Technology

[0002] With the continuous development of intelligent assisted driving technology, head-up displays (HUDs) have become an indispensable component of intelligent driving technology, and the naked-eye 3D function of HUDs has become a focus of attention in the industry.

[0003] Since naked-eye 3D images are formed based on the parallax of the images seen by the left and right eyes, the displayed images seen by the left and right eyes need to be fused in advance. Specifically, the displayed views seen by the left and right eyes are divided into different partitions, each partition corresponding to a block image. Each block image is displayed with a matching sub-pixel distribution state, and finally the block images are fused into a complete image. In the existing technology, straight lines are usually used as the fusion boundary of the block images, and the fusion between block images is completed based on the boundary of each straight line.

[0004] However, due to the irregularity of the boundaries of the segmented images, the fusion boundary defined by the above method often fails to display the full content of the segmented image, resulting in content loss. Furthermore, when fusion two adjacent segmented images based on this fusion boundary, the fusion result often has a large error and cannot truly reflect the actual displayed content. Summary of the Invention

[0005] This invention provides an image processing method, apparatus, head-up display, storage medium, and computer program product to solve the problem of content loss in fused images, which fails to accurately reflect the actual displayed content.

[0006] According to one aspect of the present invention, an image processing method is provided, comprising: For each image block, obtain the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that image block; based on the coordinate data of the first feature pixel, the coordinate data of the second feature pixel, and the number of reference feature pixels, obtain the coordinate data of the reference feature pixels of the boundary contour to be fused; based on the coordinate data corresponding to the first feature pixel, the second feature pixel, and the reference feature pixels, determine the fusion boundary of the image block. Based on the fusion boundaries and the positional relationships between the segmented images, image fusion is performed on the segmented images.

[0007] According to another aspect of the present invention, an image display method is provided, comprising: Obtain a fused image; wherein the fused image includes multiple segmented images, and at least one of the multiple segmented images has a boundary that is a polyline; Output the fused image.

[0008] According to another aspect of the present invention, an image display method is provided, comprising: Obtain a fused image; wherein the fused image includes multiple segmented images, and the boundary of at least one of the multiple segmented images includes at least three feature pixels; Output the fused image.

[0009] According to another aspect of the present invention, an image display method is provided, comprising: Obtain a fused image; wherein the fused image comprises multiple segmented images, and the boundary of at least one of the multiple segmented images is a curve; Output the fused image.

[0010] According to another aspect of the present invention, an image processing apparatus is provided, comprising: The fusion boundary acquisition module is used to, for each segmented image, acquire the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that segmented image; acquire the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixels, the coordinate data of the second feature pixels, and the number of reference feature pixels; and determine the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixels, the second feature pixels, and the reference feature pixels, respectively. The image fusion execution module is used to perform image fusion on the image blocks according to the fusion boundaries and the positional relationship between the image blocks.

[0011] According to another aspect of the present invention, a head-up display is provided, the head-up display comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform an image processing method according to any embodiment of the present invention, or to perform an image display method according to any embodiment of the present invention.

[0012] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute, implement the image processing method described in any embodiment of the present invention, or execute the image display method described in any embodiment of the present invention.

[0013] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the image processing method described in any embodiment of the present invention, or performs the image display method described in any embodiment of the present invention.

[0014] The technical solution of this invention involves, for each segmented image, obtaining the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that segmented image; obtaining the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixels, the coordinate data of the second feature pixels, and the number of reference feature pixels; determining the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixels, the second feature pixels, and the reference feature pixels; and performing image fusion on each segmented image based on the fusion boundaries and the positional relationship between each segmented image. This not only achieves accurate construction of fusion boundaries that fit the actual contour of the segmented images, making the transition between segmented images smoother and more natural, avoiding obvious breaks, ghosting, or display discontinuities at the splicing points of segmented images, thus improving the continuity and integrity of the fused image content, but also improves the clarity of the fused image based on the image fusion result of this accurate fusion boundary, realistically restoring the actual displayed content.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of an image processing method provided according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the fusion process of four segmented images provided in Embodiment 1 of the present invention; Figure 3This is a schematic diagram of a fusion boundary structure provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of another fusion boundary structure provided by Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of a sub-pixel distribution state provided in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of the fusion process of multiple segmented images according to Embodiment 1 of the present invention; Figure 7 This is a flowchart of another image processing method provided according to Embodiment 2 of the present invention; Figure 8 This is a flowchart of another image processing method provided according to Embodiment 3 of the present invention; Figure 9 This is a flowchart of an image display method provided according to Embodiment 4 of the present invention; Figure 10 This is a flowchart of another image display method provided in Embodiment 5 of the present invention; Figure 11 This is a flowchart of yet another image display method provided according to Embodiment Six of the present invention; Figure 12 This is a schematic diagram of the structure of an image processing apparatus according to Embodiment Seven of the present invention; Figure 13 This is a schematic diagram of the structure of a head-up display that implements the image processing method of this invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Example 1 Figure 1 This is a flowchart of an image processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation where segmented images are merged based on the fusion boundary determined by the fusion boundary contours of the segmented images to be merged. This method can be executed by an image processing device, which can be implemented in hardware and / or software. The image processing device can be configured in a smart device, such as a head-up display device or a computer, or it can be configured in a server, such as a business server or an application server. Figure 1 As shown, the method includes: S101. For each segmented image, obtain the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in the segmented image; obtain the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixel, the coordinate data of the second feature pixel, and the number of reference feature pixels; determine the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixel, the second feature pixel, and the reference feature pixels.

[0021] Because naked-eye 3D relies on the left and right eyes receiving different parallax images to create a stereoscopic effect, and the parallax angle, 3D depth of field, and in-screen / out-of-screen positions vary in different areas, it is necessary to partition the image. Specifically, the boundaries of the feature structures within the image are determined, and based on these boundaries, image blocks are defined. Each image block is then individually matched with its corresponding sub-pixel arrangement, illumination method, and spatial distribution to ensure precise separation of light rays between the left and right eyes, prevent crosstalk, and achieve a correct 3D effect. For example, using... Figure 2 For example, based on the boundaries, the number of image blocks can be divided into 4, namely... Figure 2In the image, there are 1 (block image 1), 2 (block image 2), 3 (block image 3), and 4 (block image 4). The boundary is the contour to be fused, which refers to the actual boundary line of the block images. The actual boundary line can be calculated using boundary detection algorithms, contrast enhancement algorithms, etc. Figure 3 For example, the actual boundary of the current segmented image is marked with a dashed line, which is the boundary contour to be fused. The boundary contour to be fused is usually an irregular curve.

[0022] The first feature pixel is a starting feature point selected on the boundary contour to be fused in the current segmented image. It can be an endpoint, inflection point, or extreme point on the boundary contour and is used to determine the starting position and reference direction of the boundary. The second feature pixel is another key feature point different from the first feature pixel. It can also be an endpoint, inflection point, or extreme point on the boundary contour and is used together with the first feature pixel to determine the overall direction, length, and orientation of the boundary.

[0023] For example, the first feature pixel can be the largest pixel on the boundary contour to be fused, and the second feature pixel can be the smallest pixel on the boundary contour to be fused. Furthermore, the first feature pixel can also be the non-edge pixel closest to the largest pixel (i.e., the pixel with the closest value to the largest pixel), and the second feature pixel can also be the non-edge pixel closest to the smallest pixel (i.e., the pixel with the closest value to the smallest pixel), to avoid noise issues associated with edge pixels. Additionally, the first feature pixel can be a peak inflection point from an upward trend to a downward trend, and the second feature pixel can be a trough inflection point from a downward trend to an upward trend. The first feature pixel can also be an endpoint on the vertical axis, and the second feature pixel can be an endpoint on the horizontal axis.

[0024] For example, the x-coordinate can be used as the selection criterion for the first and second feature pixels, that is, the pixel with the largest x-coordinate is selected as the first feature pixel, and the pixel with the smallest x-coordinate is selected as the second feature pixel; alternatively, the y-coordinate can be used as the selection criterion, that is, the pixel with the largest y-coordinate is selected as the first feature pixel, and the pixel with the smallest y-coordinate is selected as the second feature pixel. Furthermore, both the x-coordinate and y-coordinate can be used simultaneously as the selection criterion, that is, the pixel with the largest sum of its x-coordinate and y-coordinate is selected as the first feature pixel, and the pixel with the smallest sum of its x-coordinate and y-coordinate is selected as the second feature pixel. Specifically, the first and second feature pixels can be obtained by calculating the coordinates of each pixel, or they can be calculated using other devices.

[0025] In relevant image fusion schemes, the line connecting the largest and smallest pixels can be used as the fusion boundary of the image block; for example... Figure 3 As shown, the region within the fusion boundary is the optimal region used during fusion, while the region within the outline of the boundary to be fused but outside the fusion boundary is considered a non-optimal region and is not included in the fusion scope. However, this fusion method results in a large area of ​​the current block image being missing during the fusion process. Since the fusion boundary also serves as the boundary of another block image adjacent to the current block image, the adjacent block image needs to fill in the missing area of ​​the current block image to ensure the information integrity of the fused image. However, the adjacent block image often cannot completely cover the large area missing from the current block image, which may lead to information loss in the fusion result.

[0026] To avoid the aforementioned information loss, in this embodiment of the invention, in addition to obtaining the first and second feature pixels on the boundary contour to be fused, it is also necessary to obtain a corresponding number of reference feature pixels on the boundary contour to be fused based on the number of reference feature pixels. For different image blocks, the number of reference feature pixels can be the same or different. Furthermore, the number of reference feature pixels can be pre-configured as needed. For example, to improve the accuracy of the fusion boundary position of the current image block, the number of reference feature pixels can be configured to a larger value; to improve the efficiency of obtaining the fusion boundary of the current image block and avoid excessive computation, the number of reference feature pixels can be configured to a smaller value.

[0027] Reference feature pixels can be equally divided pixels, meaning that the spacing between adjacent reference feature pixels is equal, or the spacing between adjacent reference feature pixels along the horizontal or vertical axis is equal; reference feature pixels can also be non-equally divided pixels, for example, the spacing between adjacent reference feature pixels closer to the first and second feature pixels is larger, and the spacing between adjacent reference feature pixels farther from the first and second feature pixels is smaller; depending on the curvature of the boundary contour to be fused, when the curvature is large, the spacing between adjacent reference feature pixels is small, and when the curvature is small, the spacing between adjacent reference feature pixels is large.

[0028] Optionally, in this embodiment of the invention, for each segmented image, obtaining the number of reference feature pixels of the boundary contour to be fused of the segmented image includes: obtaining the number of reference feature pixels according to the shape parameters of the boundary contour to be fused of the segmented image; wherein, the shape parameters include, but are not limited to, one or more of the following: curvature, perimeter.

[0029] Specifically, curvature represents the degree of bending of the boundary contour to be fused. It can be obtained by calculating the rate of change of the tangent direction of the curve with respect to the arc length. The greater the curvature, the greater the degree of bending, and the more reference feature pixels are needed to fully describe the curve pattern of the current boundary contour to be fused. For example, based on the curvature calculation result, the number of matching reference feature pixels can be obtained by querying based on the current curvature through the correspondence between the number of curvatures. Similarly, the longer the perimeter of the boundary contour to be fused, the more reference feature pixels are needed to fully describe its curve range. For example, based on the perimeter calculation result, the number of matching reference feature pixels can be obtained by querying based on the current perimeter through the correspondence between the number of perimeters.

[0030] Furthermore, different weights can be assigned to curvature and perimeter to obtain the product of the current curvature and its corresponding weight, as well as the product of the current perimeter and its corresponding weight. The sum of these two products is used as the shape parameter score. This shape parameter score is also positively correlated with the number of reference feature pixels; that is, the larger the shape parameter score, the larger the number of reference feature pixels. By using the correspondence between the score and the number of reference feature pixels, the matching number of reference feature pixels can be obtained based on the current shape parameter score. Accordingly, the boundary contours to be fused in different image blocks can correspond to different numbers of reference feature pixels. Based on the perimeter and / or curvature of each boundary contour to be fused, the number of reference feature pixels for each boundary contour to be fused is determined. This ensures that the fusion boundary of each boundary contour closely resembles its actual curve shape, reduces computational complexity, and improves the efficiency of obtaining the fused image.

[0031] Based on the number of reference feature pixels, one or more reference feature pixels are selected on the boundary contour to be fused, so that the distance between the reference feature pixels and the first and second feature pixels, as well as the distance between each reference feature pixel, meets the configuration requirements described in the above technical solution, and the coordinate data of each reference feature pixel is obtained accordingly.

[0032] Taking the first feature pixel as the maximum pixel, the second feature pixel as the minimum pixel, and the reference feature pixel as the equally divided pixel, and using the ordinate as the selection criterion for the maximum and minimum pixel, as an example... Figure 4 In this example, taking three equally divided pixels as an example, the coordinate difference between the largest and smallest pixels is divided into three equal parts, and the ordinate of each equal part is recorded. Then, the ordinate is substituted into the boundary contour to be merged, the equally divided pixel on the boundary contour to be merged is found, and the abscissa of the equally divided pixel is obtained. Based on this, the abscissa and ordinate of each equally divided pixel are known.

[0033] Specifically, based on the maximum pixel, the minimum pixel, and the number of equally divided pixels, the ordinate of each equally divided pixel can be obtained using the following equation: ; in, Indicates the number of equally divided pixels; This represents the y-coordinate of the smallest pixel. This represents the y-coordinate of the largest pixel. Indicates the first 1 equally divided pixel point, 1 ; Indicates the first The ordinates of the pixels are divided into equal parts.

[0034] Based on this, the ordinate of each equally divided pixel can be obtained through the above equation. After substituting the ordinate of each equally divided pixel into the boundary contour to be fused, the equally divided pixel corresponding to each ordinate and the x-coordinate of each equally divided pixel can be determined in the boundary contour to be fused. Based on this, the coordinates of each equally divided pixel point are formed. That is, to obtain the coordinate data of each reference feature pixel on the boundary contour to be fused.

[0035] Based on the coordinate data corresponding to the first feature pixel, the second feature pixel, and each reference feature pixel, a boundary curve can be obtained by fitting coordinate values. This boundary curve can be used as the fusion boundary of the current segmented image. Alternatively, adjacent feature pixels (including the first feature pixel, the second feature pixel, and each reference feature pixel) can be connected by straight lines to generate a polyline equation, which can then be used as the fusion boundary of the current segmented image.

[0036] Taking the example of connecting adjacent feature pixels with straight lines to obtain the polyline equation, based on the first feature pixel... and the first reference feature pixel (That is, the reference feature pixel that is closest to the first feature pixel in the vertical direction), the equation of the straight line between the first feature pixel and the first reference feature pixel can be calculated; similarly, based on the first reference feature pixel... and the second reference feature pixel (That is, the reference feature pixel that is closest to the first reference feature pixel in the vertical direction and whose vertical coordinate is smaller than that of the first reference feature pixel), and the equation of the straight line between the first reference feature pixel and the second reference feature pixel can be calculated.

[0037] Based on this, the straight line equations between each adjacent reference feature pixel are obtained. This process continues until all straight line equations between adjacent reference feature pixels are obtained, at which point the terminal pixel is obtained. Reference feature pixels With the second feature pixel The equation of the straight line between them, based on which and Within the area between, the above The polyline equation composed of straight line equations is the polyline equation corresponding to the boundary contour to be fused. This polyline equation actually defines the fusion boundary of the current block image, that is, the boundary position of the fusion region when the current block image is fused with other block images.

[0038] Specifically, step S10 describes how to obtain the fusion boundary of a segmented image based on the number of reference feature pixels, the first feature pixels, and the second feature pixels of the boundary contour to be fused of the segmented image. After each segmented image has obtained its respective fusion boundary based on the above step S101, step S102 can be executed.

[0039] S102. Based on the fusion boundaries and the positional relationship between the image blocks, perform image fusion on each image block.

[0040] like Figure 5 As shown, in a 3D HUD, taking a block image containing 8 sub-pixels as an example; S1 to S8 represent the arrangement sequence numbers of each sub-pixel in a block image, L represents the sub-pixel seen by the left eye, and R represents the sub-pixel seen by the right eye. By periodically moving the starting sub-pixel of the block image (i.e., L1) and continuously changing the sub-pixel positions sequentially, a total of 8 sub-pixel distribution states can be obtained for this block image, namely states 1 to 8. For any block image, under the above 8 sub-pixel distribution states, the display image under each sub-pixel distribution state is obtained, i.e., the sub-pixel distribution state image, and the distribution state with the lowest crosstalk between the left-eye and right-eye images is taken as the sub-pixel distribution state corresponding to block image 1. Based on this method, the sub-pixel distribution state corresponding to each block image can be obtained. For example... Figure 6 As shown, block image 1 corresponds to sub-pixel distribution state 1 (i.e., state 1). Assume that block image 2, block image 3, and block image 4 correspond to sub-pixel distribution state 2 (i.e., state 2), sub-pixel distribution state 3 (i.e., state 3), and sub-pixel distribution state 4 (i.e., state 4), respectively.

[0041] After obtaining the fusion boundaries corresponding to each image block based on the above embodiments, image fusion can be performed on each image block based on the fusion boundaries and the positional relationship between each image block. For example, adjacent image blocks with assigned sub-pixel distribution states can be stitched together sequentially according to their respective fusion boundaries, following a preset stitching order, such as from edge to center, from top to bottom, etc. Figure 6 For example, based on the position information of segmented image 1 and segmented image 2, it can be determined that segmented image 1 and segmented image 2 are adjacent and segmented image 1 is located on the left. The right fusion boundary of segmented image 1 and the left fusion boundary of segmented image 2 are the same common boundary. Therefore, image stitching and fusion are performed based on the right fusion boundary of segmented image 1 and the left fusion boundary of segmented image 2.

[0042] Similarly, based on the positional information of segmented images 2 and 3, it can be determined that segmented image 2 is adjacent to segmented image 3 and segmented image 2 is located on the left. The right fusion boundary of segmented image 2 and the left fusion boundary of segmented image 3 are the same common boundary. Therefore, image stitching and fusion are performed based on the right fusion boundary of segmented image 2 and the left fusion boundary of segmented image 3. Based on the positional information of segmented images 3 and 4, it can be determined that segmented image 3 is adjacent to segmented image 4 and segmented image 3 is located on the left. The right fusion boundary of segmented image 3 and the left fusion boundary of segmented image 4 are the same common boundary. Therefore, image stitching and fusion are performed based on the right fusion boundary of segmented image 3 and the left fusion boundary of segmented image 4.

[0043] Optionally, in this embodiment of the invention, image fusion is performed on each of the fusion boundaries and the positional relationship between the segmented images, including: adjusting the fusion boundary between the first segmented image and the second segmented image according to the overlapping area between the first segmented image and the second segmented image; wherein the first segmented image and the second segmented image are two adjacent segmented images at any position among the segmented images; and performing image fusion on the first segmented image and the second segmented image according to the adjusted fusion boundary.

[0044] Specifically, for adjacent first and second block images, the calculated fusion boundary may still result in overlapping areas between the first and second block images. In this case, the position of the fusion boundary can be adjusted based on the center line of the overlapping area (e.g., a center straight line, a center broken line, or a center curve). For example, the coordinate range of the center line can be determined based on the start and end points of the horizontal coordinate of the center line of the overlapping area, and the line segments in the fusion boundary whose horizontal coordinates are also within the coordinate range can be replaced with the aforementioned center line. In other words, the overlapping area is evenly distributed to the first and second block images, thereby completing the position update of the fusion boundary.

[0045] Alternatively, the overlapping area can be completely preserved in either the first or second image block. Taking the preservation of the overlapping area in the first image block as an example, the boundary coordinates of the overlapping area near the second image block are obtained, and the line segment in the fusion boundary corresponding to the boundary coordinates is replaced with the boundary coordinates. In addition, the overlapping area can be distributed to the first and second image blocks respectively according to other proportions. By updating the position of the fusion boundary, overlapping areas between the image blocks are avoided, further improving the image fusion effect.

[0046] The technical solution of this invention involves, for each segmented image, obtaining the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that segmented image; obtaining the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixels, the coordinate data of the second feature pixels, and the number of reference feature pixels; determining the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixels, the second feature pixels, and the reference feature pixels; and performing image fusion on each segmented image based on the fusion boundaries and the positional relationship between each segmented image. This not only achieves accurate construction of fusion boundaries that fit the actual contour of the segmented images, making the transition between segmented images smoother and more natural, avoiding obvious breaks, ghosting, or display discontinuities at the splicing points of segmented images, thus improving the continuity and integrity of the fused image content, but also improves the clarity of the fused image based on the image fusion result of this accurate fusion boundary, realistically restoring the actual displayed content.

[0047] Example 2 Figure 7 This is a flowchart of an image processing method provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is that the number of reference feature pixels of the boundary contour to be fused is iteratively calculated, such as... Figure 7 As shown, the method specifically includes: For each image block, execute S201~S203: S201. Obtain the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in the segmented image.

[0048] S202. Based on the coordinate data of the first feature pixel, the coordinate data of the second feature pixel, and the number of reference feature pixels, obtain the coordinate data of the reference feature pixels of the boundary contour to be fused.

[0049] S203. Determine the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixel, the second feature pixel, and the reference feature pixel.

[0050] S204. Based on the fusion boundaries and the positional relationship between the image blocks, perform image fusion on each image block.

[0051] S205. If it is determined that the image fusion result does not meet the pre-configured crosstalk constraint conditions, the number of reference feature pixels is increased, and the fusion boundary of each image block is re-determined based on the updated number of reference feature pixels, until the image fusion result obtained according to the re-determined fusion boundary meets the crosstalk constraint conditions.

[0052] Since naked-eye 3D images are formed from the parallax images of the left and right eyes, after obtaining the image fusion result, i.e., the fused image, the crosstalk value between the left and right eye images is calculated. Crosstalk refers to the phenomenon where one eye sees the other eye's image when the left and right eye images are not completely separated, resulting in ghosting, visual fatigue, and decreased stereoscopic effect, thus reducing image quality. The crosstalk value can be quantified by the brightness ratio, which is the ratio of the brightness of the image portion that should not be seen to the brightness of the image portion that should be seen. It can also be calculated using methods such as full-screen measurement based on camera shooting or stepwise measurement based on photometer. It can also be calculated using methods such as grayscale difference, pixel overlap, or parallax error. The image fusion result mentioned above can refer to the fusion result between two adjacent image blocks, the fusion result of multiple image blocks (e.g., the fusion result between three adjacent image blocks), or a complete image formed by fusing all image blocks.

[0053] The crosstalk constraint records the crosstalk threshold between the left-eye and right-eye images, such as the brightness ratio threshold, grayscale difference threshold, pixel overlap threshold, and disparity error threshold mentioned above, to quantify the crosstalk constraint. When the crosstalk value between the left-eye fused image and the right-eye fused image is greater than or equal to the crosstalk threshold, it indicates that the image fusion result does not meet the crosstalk constraint, that is, the image quality of the current fused image is poor. At this time, it is necessary to iteratively increment the number of reference feature pixels (for example, the iteration step size is 1, that is, to increment the number of reference feature pixels by 1), and re-execute S201~S204 with the updated number of reference feature pixels. Then, the newly obtained image fusion result is evaluated again according to the crosstalk constraint until the current image fusion result meets the crosstalk constraint.

[0054] In one possible implementation, when the image fusion result that meets the crosstalk constraint condition is obtained based on the above embodiment, the number of reference feature pixels used in the current iteration is recorded. This number can be used as the optimal parameter in the current display scene so that it can be directly called when processing similar images in the future, reducing repeated iterative calculations and improving fusion efficiency. At the same time, it can avoid computational redundancy caused by too many feature points, and optimize the system's computational overhead while ensuring low crosstalk and high imaging quality.

[0055] Specifically, if the image fusion result corresponding to some image blocks does not meet the crosstalk constraint, the number of reference feature pixels in that image block can be incrementally iterated. If the image fusion result corresponding to all image blocks does not meet the crosstalk constraint, incremental iteration can be performed on the boundary contours to be fused of each image block simultaneously. Alternatively, incremental iteration can be performed first on the boundary contours to be fused with fewer reference feature pixels until the number of reference feature pixels on the boundary contours to be fused of each image block is the same. Then, incremental iteration can be performed on the boundary contours to be fused of each image block simultaneously to achieve targeted adjustment of the number of reference feature pixels.

[0056] S206. If the image fusion result meets the pre-configured crosstalk constraint conditions, reduce the number of reference feature pixels. Based on the updated number of reference feature pixels, redetermine the fusion boundary of each image block until the image fusion result obtained according to the redetermined fusion boundary does not meet the crosstalk constraint conditions. Then, determine the image fusion result determined in the previous iteration as the final image fusion result.

[0057] When it is determined that the image fusion result meets the pre-configured crosstalk constraint, that is, the crosstalk value between the left-eye fused image and the right-eye fused image is less than the crosstalk threshold, the number of reference feature pixels obtained this time may not be the minimum number of reference feature pixels. There may be redundancy in the number of reference feature pixels, which prolongs the calculation time. At this time, the current number of reference feature pixels is iteratively decreased (for example, the iteration step size is 1, that is, the number of reference feature pixels is reduced by 1), and S201~S204 is re-executed with the updated number of reference feature pixels. Then, the newly obtained image fusion result is evaluated again according to the crosstalk constraint until the current image fusion result does not meet the crosstalk constraint.

[0058] In one possible implementation, when the image fusion result obtained based on the above embodiment does not meet the crosstalk constraint, the number of reference feature pixels used in the previous iteration is recorded. This number can be used as the optimal parameter in the current display scene so that it can be directly called when processing similar images in the future, reducing repeated iterative calculations and improving fusion efficiency. At the same time, it can avoid computational redundancy caused by too many feature points, and optimize the system's computational overhead while ensuring low crosstalk and high imaging quality.

[0059] Specifically, if the image fusion result corresponding to a portion of the image blocks meets the crosstalk constraint, then the number of reference feature pixels in that portion of the image blocks can be iteratively decreased. If the image fusion result corresponding to all the image blocks meets the crosstalk constraint, then the decreasing iteration can be performed simultaneously on the boundary contours to be fused in each image block. Alternatively, the decreasing iteration can be performed first on the boundary contours to be fused with a larger number of reference feature pixels until the number of reference feature pixels on the boundary contours to be fused in each image block is the same. Then, the decreasing iteration can be performed simultaneously on the boundary contours to be fused in each image block, thereby achieving targeted adjustment of the number of reference feature pixels.

[0060] The technical solution of this invention, when it is determined that the image fusion result does not meet the pre-configured crosstalk constraints, records the number of reference feature pixels used in the current iteration under incremental iteration and uses this number as the optimal parameter for the current display scene. When the image fusion result does not meet the pre-configured crosstalk constraints, it records the number of reference feature pixels used in the previous iteration under decremental iteration and uses this number as the optimal parameter for the current display scene, so that it can be directly called when processing similar images in the future, reducing repeated iterative calculations and improving fusion efficiency; at the same time, it can avoid computational redundancy caused by too many feature points, and optimize the system's computational overhead while ensuring low crosstalk and high imaging quality.

[0061] Example 3 Figure 8 This is a flowchart of an image processing method provided in Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that, based on the grayscale values ​​of each pixel in the image region to be processed, the boundary contours to be fused for each segment of the image are obtained, such as... Figure 8 As shown, the method specifically includes: S301. Based on the grayscale values ​​of each pixel in the image region to be processed, perform boundary extraction on the image region to be processed and obtain the coordinates of each boundary pixel.

[0062] Specifically, the image region to be processed can refer to a specific image range selected from the virtual image projected onto the windshield by the HUD, which requires image feature extraction and calibration analysis. Grayscale value refers to the brightness value of a pixel in the image, and is a basic quantitative indicator for image feature analysis. The difference in grayscale values ​​of different pixels constitutes the brightness and boundary features of the image. Since the grayscale values ​​of pixels within the image region to be processed vary, boundary pixels are the dividing points between the image region to be processed and the background region, and their grayscale values ​​exhibit obvious abrupt changes. Therefore, boundary pixels can be used as the basis for boundary extraction. In practical applications, boundaries can be extracted based on grayscale values ​​in various ways, such as gradient analysis and index distribution analysis. Using these methods, pixels with abrupt changes in grayscale values ​​within the image region to be processed, i.e., boundary pixels, can be accurately identified. Then, the two-dimensional coordinates of each boundary pixel are collected and recorded one by one to obtain the set of boundary pixel coordinates corresponding to each image region to be processed.

[0063] For example, based on the grayscale values ​​of each pixel in each image region to be processed, the boundary of each image region to be processed is extracted, and the coordinates of each boundary pixel can be determined in the following way: First, the grayscale gradient of each pixel in the image region to be processed is calculated, then the first pixel with a gradient greater than or equal to the threshold is selected as the boundary candidate pixel according to the preset gradient threshold, and the second pixel with a gradient less than the threshold is the non-boundary pixel. Finally, the intersection is determined according to the distribution of the two types of pixels, and the pixel coordinates at the intersection are collected as the boundary pixel coordinates.

[0064] Optionally, in this embodiment of the invention, the step of extracting the boundaries of the image region to be processed based on the gray values ​​of each pixel in the image region to be processed and obtaining the coordinates of each boundary pixel includes: determining the gradient of the gray values ​​of each pixel in the image region to be processed; and extracting the boundaries of the image region to be processed based on each gradient to obtain the coordinates of each boundary pixel.

[0065] Specifically, the image calibration device calculates the grayscale gradient for each pixel within each region of the image to be processed. The grayscale gradient refers to the rate of change of grayscale value between a pixel and its neighboring pixels; its magnitude directly reflects the degree of abrupt change in the grayscale value of that pixel. Boundary pixels, located at region boundaries, have significantly different grayscale values ​​from their neighboring pixels, thus their grayscale gradient values ​​are usually much larger than those of non-boundary pixels. Non-boundary pixels, situated within the region, exhibit gentler grayscale changes, resulting in smaller grayscale gradient values. In practical calculations, mature gradient calculation algorithms such as the Sobel algorithm, Prewitt algorithm, and Canny algorithm can be used. By calculating the grayscale changes of a pixel in the horizontal, vertical, or diagonal directions, the gradient value of that pixel is accurately obtained, providing a basis for subsequent boundary selection.

[0066] After obtaining the gradient values ​​of all pixels within the image region to be processed, boundary pixels are selected from all pixels using preset filtering rules. These filtering rules can be set according to the actual image features and calibration accuracy requirements. Pixels whose grayscale gradient values ​​meet preset conditions are selected; for example, pixels with gradient values ​​greater than a preset threshold are selected. These pixels are considered boundary pixels with significant grayscale value abrupt changes. Finally, the coordinates of the selected boundary pixels are collected and organized to obtain the boundary pixel coordinates of the image region to be processed. For example, preliminary edge filtering is performed using a preset gradient magnitude threshold, removing background and smooth region pixels with gradient magnitudes below the threshold, and retaining candidate edge pixels with high gradient magnitudes. Then, non-maximum suppression is performed along the gradient direction of each pixel, retaining only pixels with the largest local magnitude in the gradient direction and removing redundant pseudo-edges. Finally, through dual-threshold detection and edge connectivity tracking, pixels that meet the strong edge threshold are used as reference edge points, and pixels within the weak edge threshold and connected to the reference edge are added to form a continuous and complete boundary. The horizontal and vertical coordinates of all pixels on each boundary are recorded to obtain the set of boundary pixel coordinates.

[0067] Optionally, in this embodiment of the invention, the step of extracting the boundary of the image region to be processed based on each gradient to obtain the coordinates of each boundary pixel includes: determining a gradient threshold; obtaining each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; and extracting the boundary of the image region to be processed based on each first pixel and each second pixel to obtain the coordinates of each boundary pixel.

[0068] Specifically, the gradient threshold is used to distinguish between boundary pixels and non-boundary pixels. In practical applications, the gradient threshold can be flexibly selected according to the actual application scenario. It supports manually fixed threshold settings, such as directly taking fixed parameters like 50 or 100; it can also adopt adaptive threshold calculation methods, such as automatically deriving the optimal threshold based on the grayscale distribution characteristics of the image region; and it can also adapt to the differentiated setting requirements of global threshold and local threshold. When presetting the gradient threshold, if it is necessary to highlight image edges and retain more fine boundary features, the gradient threshold can be set smaller, while if it is necessary to reduce noise interference and eliminate weak false boundaries, the gradient threshold can be set larger. After obtaining the gradient values ​​of all pixels in the image region to be processed, all pixels are divided into two categories according to the set gradient threshold. One category is pixels with gradient values ​​greater than or equal to the gradient threshold, defined as the first pixel. The grayscale value of the first pixel changes significantly, which meets the characteristics of a boundary pixel and is likely to be a boundary pixel. The other category is pixels with gradient values ​​less than the gradient threshold, defined as the second pixel. The grayscale value of the second pixel changes gradually and belongs to non-boundary pixels within the image region to be processed.

[0069] Furthermore, the image calibration device analyzes the distribution of the first and second pixels to determine the boundary between the two types of pixels. The pixels at the boundary are the actual boundary pixels of the image region to be processed. By collecting and organizing the coordinates of the pixels at the boundary, the set of boundary pixel coordinates of the image region to be processed can be obtained, thus completing the boundary extraction process.

[0070] S302. Based on the coordinates of each boundary pixel, determine the boundary contours to be fused for each image block.

[0071] Based on the coordinates of each boundary pixel, each boundary pixel is fitted to a smooth curve, and this fitted curve is the boundary contour to be fused in the current segmented image.

[0072] S303. For each block image, obtain the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that block image.

[0073] S304. Based on the coordinate data of the first feature pixel, the coordinate data of the second feature pixel, and the number of reference feature pixels, obtain the coordinate data of the reference feature pixels of the boundary contour to be fused.

[0074] S305. Determine the fusion boundary of the image block based on the coordinate data corresponding to the first feature pixel, the second feature pixel, and the reference feature pixel.

[0075] S306. Based on each fusion boundary and the positional relationship between each block image, perform image fusion on each block image.

[0076] The technical solution of this invention extracts the boundaries of the image region to be processed based on the grayscale values ​​of each pixel, obtaining the coordinates of each boundary pixel. Based on the coordinates of each boundary pixel, the boundary contours to be fused for each image block are determined. Thus, the grayscale values ​​of each pixel in the image region to be processed, obtained through calculation, reflect the abrupt change features in the image region, and effectively distinguish between boundary pixels and non-boundary pixels based on these abrupt change features, ensuring the accuracy of the boundary pixel extraction results, and consequently ensuring the accurate location of the obtained boundary contours to be fused.

[0077] Example 4 Figure 9 This is a flowchart of an image display method provided in Embodiment 4 of the present invention. This method can be executed by an image processing device, which can be implemented in hardware and / or software, and can be configured in a head-up display. Figure 9 As shown, the method includes S401. Obtain the fused image; wherein the fused image includes multiple block images, and the boundary of at least one of the block images is a polyline.

[0078] The acquisition of the fused image can be performed after receiving the image display instruction, or it can be performed according to a predetermined image acquisition cycle. The fusion process of the fused image can be executed in the current image display device or in other devices (e.g., the image processing device in Embodiment 7). The boundaries of one or more block images in each block image can be fitted into a polyline by fitting multiple feature pixels, or a polyline represented in the form of a polyline equation can be generated by connecting them sequentially. The specific execution process can be referred to Embodiments 1-3 above, and repeated parts will not be described again.

[0079] S402, Output the fused image.

[0080] The merged image can be output to other devices or apparatuses for display. For example, a 3D HUD can project the merged image onto the windshield to form a virtual image, or it can be output to the display screen of the current image display device itself for display.

[0081] The technical solution of this invention, after acquiring a fused image comprising multiple segmented images, wherein the boundary of at least one of the segmented images is a polygonal line, outputs the fused image. The output fused image not only achieves the accurate construction of a fusion boundary that fits the actual contour of the segmented images, making the splicing transition between the segmented images smoother and more natural, avoiding problems such as obvious discontinuity, ghosting, or display breaks at the splicing points of the segmented images, thus improving the continuity and integrity of the content of the fused image, but also improves the clarity of the fused image based on the image fusion result of the accurate fusion boundary, truly restoring the actual displayed content.

[0082] Example 5 Figure 10 This is a flowchart of an image display method provided in Embodiment 5 of the present invention. This method can be executed by an image processing device, which can be implemented in hardware and / or software, and can be configured in a head-up display. Figure 10 As shown, the method includes S501. Obtain the fused image; wherein the fused image includes multiple block images, and the boundary of at least one block image in the multiple block images includes at least three feature pixels.

[0083] The boundary of one or more blocks in each block image includes at least three feature pixels, namely, a first feature pixel, a second feature pixel, and at least one reference feature pixel. Based on the above at least three feature pixels, a polyline boundary, a curve boundary, or a straight line boundary can be fitted. The specific execution process can be referred to in Examples 1-3 above, and repeated parts will not be described again.

[0084] S502, Output the fused image.

[0085] The technical solution of this invention, after acquiring a fused image comprising multiple segmented images, wherein the boundary of at least one of the segmented images comprises at least three feature pixels, outputs the fused image. The output fused image not only achieves the accurate construction of a fusion boundary that fits the actual contour of the segmented images, making the splicing transition between the segmented images smoother and more natural, avoiding problems such as obvious discontinuity, ghosting, or display breaks at the splicing points of the segmented images, thus improving the continuity and integrity of the content of the fused image, but also improves the clarity of the fused image based on the image fusion result of the accurate fusion boundary, truly restoring the actual displayed content.

[0086] Example 6 Figure 11 This is a flowchart of an image display method provided in Embodiment Six of the present invention. This method can be executed by an image processing device, which can be implemented in hardware and / or software, and can be configured in a head-up display. Figure 11 As shown, the method includes S601. Obtain the fused image; wherein the fused image includes multiple block images, and the boundary of at least one of the block images is a curve.

[0087] The boundaries of one or more blocks in each block image can be fitted into curves by fitting multiple feature pixels. The specific execution process can be referred to in Examples 1-3 above, and repeated parts will not be described again.

[0088] S602, Output the fused image.

[0089] The technical solution of this invention, after acquiring a fused image comprising multiple segmented images, wherein the boundary of at least one of the segmented images is a curve, outputs the fused image. The output fused image not only achieves the accurate construction of a fusion boundary that fits the actual contour of the segmented images, making the splicing transition between the segmented images smoother and more natural, avoiding problems such as obvious discontinuity, ghosting, or display breaks at the splicing points of the segmented images, thus improving the continuity and integrity of the content of the fused image, but also improves the clarity of the fused image based on the image fusion result of the accurate fusion boundary, truly restoring the actual displayed content.

[0090] Example 7 Figure 12 This is a structural block diagram of an image processing device provided in Embodiment 7 of the present invention. The device specifically includes: The fusion boundary acquisition module 701 is used to acquire, for each segmented image, the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that segmented image; acquire the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixels, the coordinate data of the second feature pixels, and the number of reference feature pixels; and determine the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixels, the second feature pixels, and the reference feature pixels, respectively. The image fusion execution module 702 is used to perform image fusion on each of the fusion boundaries and the positional relationship between each of the segmented images.

[0091] The technical solution of this invention involves, for each segmented image, obtaining the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that segmented image; obtaining the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixels, the coordinate data of the second feature pixels, and the number of reference feature pixels; determining the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixels, the second feature pixels, and the reference feature pixels; and performing image fusion on each segmented image based on the fusion boundaries and the positional relationship between each segmented image. This not only achieves accurate construction of fusion boundaries that fit the actual contour of the segmented images, making the transition between segmented images smoother and more natural, avoiding obvious breaks, ghosting, or display discontinuities at the splicing points of segmented images, thus improving the continuity and integrity of the fused image content, but also improves the clarity of the fused image based on the image fusion result of this accurate fusion boundary, realistically restoring the actual displayed content.

[0092] Optionally, the image processing apparatus is further configured to, when determining that the image fusion result does not meet the pre-configured crosstalk constraint conditions, increase the number of reference feature pixels, and redetermine the fusion boundary of each image block based on the updated number of reference feature pixels, until the image fusion result obtained according to the redetermined fusion boundaries meets the crosstalk constraint conditions.

[0093] Optionally, the image processing device is further configured to, when determining that the image fusion result meets the pre-configured crosstalk constraint conditions, reduce the number of reference feature pixels, redetermine the fusion boundary of each image block based on the updated number of reference feature pixels, until the image fusion result obtained according to the redetermined fusion boundaries does not meet the crosstalk constraint conditions, then determine the image fusion result determined in the previous iteration as the final image fusion result.

[0094] Optionally, the image processing device is further configured to obtain the number of reference feature pixels based on the shape parameters of the boundary contour to be fused of the segmented image; wherein the shape parameters include, but are not limited to, one or more of the following: curvature, perimeter.

[0095] Optionally, the image processing device is further configured to adjust the fusion boundary between the first block image and the second block image based on the overlapping area between the first block image and the second block image; wherein the first block image and the second block image are two block images that are adjacent at any position in each block image; and to perform image fusion on the first block image and the second block image based on the adjusted fusion boundary.

[0096] Optionally, the image processing device is further configured to extract the boundaries of the image region to be processed based on the grayscale values ​​of each pixel in the image region to be processed, and obtain the coordinates of each boundary pixel; and determine the boundary contours to be fused of each segmented image based on the coordinates of each boundary pixel.

[0097] Optionally, the image processing device is further configured to determine the gradient of the grayscale value of each pixel in the image region to be processed; and based on each gradient, to perform boundary extraction on the image region to be processed and obtain the coordinates of each boundary pixel.

[0098] Optionally, the image processing apparatus is further configured to determine a gradient threshold; acquire each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; and based on the first pixel and the second pixel, perform boundary extraction on the image region to be processed to acquire the coordinates of each boundary pixel.

[0099] The above-described apparatus can execute the image processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the image processing method provided in any embodiment of the present invention.

[0100] Example 8 Figure 13A schematic diagram of a head-up display 10, which can be used to implement embodiments of the present invention, is shown. The head-up display is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade head-up displays, mainframe computers, and other suitable computers. The head-up display can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0101] like Figure 13 As shown, the head-up display 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the head-up display 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] Multiple components in the head-up display 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the head-up display 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0103] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image processing methods.

[0104] In some embodiments, the image processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0108] To provide interaction with a user terminal, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user terminal; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user terminal provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide interaction with the user terminal; for example, the feedback provided to the user terminal can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user terminal can be received in any form (including sound input, voice input, or haptic input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., client computers with graphical user interfaces or web browsers through which client computers can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0110] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0111] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0112] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image processing method, characterized in that, include: For each image block, obtain the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that image block; based on the coordinate data of the first feature pixel, the coordinate data of the second feature pixel, and the number of reference feature pixels, obtain the coordinate data of the reference feature pixels of the boundary contour to be fused; based on the coordinate data corresponding to the first feature pixel, the second feature pixel, and the reference feature pixels, determine the fusion boundary of the image block. Based on the fusion boundaries and the positional relationships between the segmented images, image fusion is performed on the segmented images.

2. The image processing method according to claim 1, characterized in that, The method further includes: If the image fusion result does not meet the pre-configured crosstalk constraint, the number of reference feature pixels is increased. Based on the updated number of reference feature pixels, the fusion boundary of each image block is redefined until the image fusion result obtained according to the redefined fusion boundary meets the crosstalk constraint.

3. The image processing method according to claim 1, characterized in that, The method further includes: If the image fusion result is determined to meet the pre-configured crosstalk constraint, the number of reference feature pixels is reduced. Based on the updated number of reference feature pixels, the fusion boundary of each image block is redefined until the image fusion result obtained according to the redefined fusion boundary does not meet the crosstalk constraint. Then, the image fusion result determined in the previous iteration is determined as the final image fusion result.

4. The image processing method according to claim 1, characterized in that, For each image block, the number of reference feature pixels for obtaining the boundary contour to be fused in that image block includes: The number of reference feature pixels is obtained based on the shape parameters of the boundary contour to be fused in the segmented image; wherein, the shape parameters include, but are not limited to, one or more of the following: curvature, perimeter.

5. The image processing method according to claim 1, characterized in that, Based on the fusion boundaries and the positional relationships between the image blocks, image fusion is performed on the image blocks, including: Based on the overlapping area between the first block image and the second block image, the fusion boundary between the first block image and the second block image is adjusted; wherein, the first block image and the second block image are two block images that are adjacent at any position in each block image; Based on the adjusted fusion boundary, the first block image and the second block image are fused.

6. The image processing method according to claim 1, characterized in that, Obtaining the boundary contours to be fused for each image segment includes: Based on the grayscale values ​​of each pixel in the image region to be processed, the boundary of the image region to be processed is extracted to obtain the coordinates of each boundary pixel; Based on the coordinates of each boundary pixel, the boundary contours to be fused for each image block are determined.

7. The method according to claim 6, characterized in that, The step of extracting the boundaries of the image region to be processed based on the grayscale values ​​of each pixel in the image region to be processed, and obtaining the coordinates of each boundary pixel, includes: Determine the gradient of the grayscale value of each pixel in the image region to be processed; Based on the gradients, the boundaries of the image region to be processed are extracted to obtain the coordinates of each boundary pixel.

8. The method according to claim 7, characterized in that, The step of extracting the boundaries of the image region to be processed based on each gradient and obtaining the coordinates of each boundary pixel includes: Determine the gradient threshold; Obtain each first pixel in the image region to be processed whose gradient is greater than or equal to the gradient threshold, and each second pixel in the image region to be processed whose gradient is less than the gradient threshold; Based on the first pixel and the second pixel, the boundary of the image region to be processed is extracted to obtain the coordinates of each boundary pixel.

9. An image display method, characterized in that, include: Obtain a fused image; wherein the fused image includes multiple segmented images, and at least one of the multiple segmented images has a boundary that is a polyline; Output the fused image.

10. An image display method, characterized in that, include: Obtain a fused image; wherein the fused image includes multiple segmented images, and the boundary of at least one of the multiple segmented images includes at least three feature pixels; Output the fused image.

11. An image display method, characterized in that, include: Obtain a fused image; wherein the fused image comprises multiple segmented images, and the boundary of at least one of the multiple segmented images is a curve; Output the fused image.

12. An image processing apparatus, characterized in that, include: The fusion boundary acquisition module is used to, for each segmented image, acquire the number of reference feature pixels, the first feature pixel, and the second feature pixel of the boundary contour to be fused in that segmented image; acquire the coordinate data of the reference feature pixels of the boundary contour to be fused based on the coordinate data of the first feature pixels, the coordinate data of the second feature pixels, and the number of reference feature pixels; and determine the fusion boundary of the segmented image based on the coordinate data corresponding to the first feature pixels, the second feature pixels, and the reference feature pixels, respectively. The image fusion execution module is used to perform image fusion on the image blocks according to the fusion boundaries and the positional relationship between the image blocks.

13. A heads-up display, characterized in that, The head-up display includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image processing method of any one of claims 1-8, or the image display method of claim 9, or the image display method of claim 10, or the image display method of claim 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image processing method of any one of claims 1-8, or the image display method of claim 9, or the image display method of claim 10, or the image display method of claim 11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the image processing method of any one of claims 1-8, or the image display method of claim 9, or the image display method of claim 10, or the image display method of claim 11.

Images