A method, device and system for determining HUD image size

Abstract

Embodiments of the present invention relate to the field of head-up display technology, and in particular, to a method, device, and system for determining the size of a HUD image. In the method for determining the size of a HUD image, an image to be matched and a matching image of the HUD image are obtained; a first characteristic element of the HUD image is determined in the image to be matched; a first pixel position of the first characteristic element in the image to be matched is extracted; a second pixel position of the first characteristic element in the matching image is determined; the actual space coordinates of the first characteristic element are calculated based on the first pixel position and the second pixel position; and the size of the HUD image is calculated based on the actual space coordinates. In the above manner, the present invention can improve the efficiency of measuring the size of the HUD image.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of head-up display technology, and in particular to a method, device and system for determining the size of a HUD image. Background Art

[0002] Head Up Display (HUD) is a technology that projects important driving information such as speed and navigation onto the windshield in front of the driver so that the driver can see the important driving information such as speed and navigation without lowering or turning his head.

[0003] Among them, the HUD image projected by the head-up display is a virtual image generated by mirror reflection. The size of the HUD image is usually measured by an actual ruler. However, the inventors found in the process of implementing the present invention that when measuring the size of the HUD image by an actual ruler, manual measurement is required, and manual measurement of the size of the HUD image will cost more manpower and has low efficiency. Summary of the invention

[0004] Embodiments of the present invention provide a method, device, and system for determining the size of a HUD image, which can improve the efficiency of measuring the size of a HUD image.

[0005] In order to solve the above technical problems, a technical solution adopted in an embodiment of the present invention is to provide a method for determining the size of a HUD image, which is applied to a binocular camera device, and the method includes:

[0006] Acquire the image to be matched and the matching image of the HUD image;

[0007] Determining a first characteristic element of the HUD image in the image to be matched;

[0008] Extracting a first pixel position of the first feature element in the image to be matched;

[0009] Determining a second pixel position of the first feature element in the matching image;

[0010] Calculating the actual space coordinates of the first feature element according to the first pixel position and the second pixel position;

[0011] The size of the HUD image is calculated according to the actual space coordinates.

[0012] Optionally, determining the first characteristic element of the HUD image in the image to be matched specifically includes:

[0013] Acquire a binary image of the image to be matched;

[0014] Determining a second characteristic element of the HUD image in the binary image;

[0015] Based on the second feature element, the first feature element corresponding to the second feature element is determined in the image to be matched.

[0016] Optionally, the binary image includes a first grayscale value and a second grayscale value, and the first grayscale value is used to indicate the HUD image; then,

[0017] The determining the second characteristic element of the HUD image in the binary image specifically includes:

[0018] Determine the area in the binary image where the first grayscale value is most continuous as a visible area;

[0019] Extracting edge pixel points of the visible area;

[0020] A second characteristic element of the HUD image is determined according to the edge pixel point.

[0021] Optionally, determining the second characteristic element of the HUD image according to the edge pixel point specifically includes:

[0022] Determine the upper edge, lower edge, left edge and right edge of the visible area according to the edge pixel points;

[0023] Determine the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point of the visible area according to the upper edge, the lower edge, the left edge and the right edge;

[0024] Calculating the credibility of the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point;

[0025] Based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point, a second characteristic element of the HUD image is determined.

[0026] Optionally, determining the upper edge, the lower edge, the left edge and the right edge of the visible area according to the edge pixel points specifically includes:

[0027] Determine a third pixel position of a central pixel of the visible area;

[0028] Dividing the edge pixel points into upper edge pixel points, lower edge pixel points, left edge pixel points and right edge pixel points according to the third pixel position;

[0029] Fitting the upper edge according to the upper edge pixel points;

[0030] Fitting the lower edge according to the lower edge pixel points;

[0031] Fitting the left edge according to the left edge pixel points;

[0032] The right edge is fitted according to the right edge pixels.

[0033] Optionally, the calculating the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point specifically includes:

[0034] Calculating the credibility of the upper edge, the lower edge, the left edge and the right edge;

[0035] Determine the credibility of the upper left corner point by multiplying the credibility of the upper edge and the left edge;

[0036] Determine the credibility of the upper right corner point by multiplying the credibility of the upper edge and the right edge;

[0037] Determine the credibility of the lower left corner point by multiplying the credibility of the lower edge and the left edge;

[0038] The credibility of the lower right corner point is determined by multiplying the credibility of the lower edge and the right edge.

[0039] Optionally, determining the second characteristic element of the HUD image based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point specifically includes:

[0040] Determine the offset upper left corner point based on the credibility of the upper left corner point;

[0041] Determine the offset upper right corner point based on the credibility of the upper right corner point;

[0042] Determine the offset lower left corner point based on the credibility of the lower left corner point;

[0043] Determine the offset lower right corner point based on the credibility of the lower right corner point;

[0044] A second characteristic element of the HUD image is determined according to the offset upper left corner point, the offset upper right corner point, the offset lower left corner point, and the offset lower right corner point.

[0045] In order to solve the above technical problems, another technical solution adopted by the embodiment of the present invention is: providing a device for determining the size of a HUD image, which is applied to a binocular camera device, and the device comprises:

[0046] An acquisition module, used for acquiring an image to be matched and a matching image of the HUD image;

[0047] A first determining module, configured to determine a first characteristic element of the HUD image in the image to be matched;

[0048] An extraction module, used for extracting a first pixel position of the first feature element in the image to be matched;

[0049] A second determining module, used to determine a second pixel position of the first feature element in the matching image;

[0050] A first calculation module, used for calculating the actual space coordinates of the first feature element according to the first pixel position and the second pixel position;

[0051] The second calculation module is used to calculate the size of the HUD image according to the actual space coordinates.

[0052] In order to solve the above technical problem, another technical solution adopted by the embodiment of the present invention is: providing a system for determining the size of a HUD image, comprising:

[0053] An in-vehicle head-up display, wherein the in-vehicle head-up display is used to project the HUD image;

[0054] A binocular camera device, the binocular camera device comprising a first lens, a second lens, at least one processor and a memory connected to the at least one processor in communication,

[0055] The first lens is used to capture the image to be matched with the HUD image.

[0056] The second lens is used to capture a matching image of the HUD image.

[0057] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can be used to perform the above method.

[0058] To solve the above technical problems, another technical solution adopted by an embodiment of the present invention is: providing a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the binocular camera device to execute the method described above.

[0059] The beneficial effect of the embodiments of the present invention is as follows: Different from the prior art, the embodiments of the present invention provide a method, device and system for determining the size of a HUD image. The method for determining the size of a HUD image is applied to a binocular camera device. After obtaining the to-be-matched image and the matching image of the HUD image, the first characteristic element of the HUD image is determined in the to-be-matched image and the first pixel position of the first characteristic element in the to-be-matched image is extracted. At the same time, the second pixel position of the first characteristic element in the matching image is determined, and then the actual space coordinates of the first characteristic element are calculated by the first pixel position and the second pixel position, and the size of the HUD image is calculated according to the actual space coordinates. That is, the present invention can calculate the size of the HUD image by using the to-be-matched image and the matching image of the HUD image obtained by the binocular camera device, without manual measurement, with high efficiency and capable of reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0061] Figure 1 is a structural schematic diagram of a system for determining a HUD image size provided by an embodiment of the present invention;

[0062] Figure 2 Schematic diagram of the hardware structure of a binocular camera device provided by an embodiment of the present invention;

[0063] Figure 3 is a flow chart of a method for determining the size of a HUD image provided by an embodiment of the present invention;

[0064] Figure 4 is a schematic diagram of a binary image;

[0065] Figure 5 It is a structural schematic diagram of a device for determining the size of a HUD image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the technical solutions of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] It should be noted that when an element is described as being "connected" to another element, it may be directly connected to the other element, or one or more intermediate elements may exist therebetween. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0068] The present invention provides a method and device for determining the size of a HUD image. The method and device are applied to a binocular camera device, so that the binocular camera device can calculate the size of the HUD image by acquiring an image to be matched and a matching image of the HUD image. Based on this, the size of the HUD image can be determined without manually measuring the HUD image, thereby improving the efficiency of measuring the size of the HUD image.

[0069] The present invention will be described below by means of specific embodiments.

[0070] See also Figure 1 , is a structural schematic diagram of a system for determining the size of a HUD image provided by an embodiment of the present invention, the system for determining the size of a HUD image includes: a vehicle-mounted head-up display 100 and a binocular camera device 200, the vehicle-mounted head-up display 100 is used to project the HUD image, and the binocular camera device 200 is used to collect the image to be matched and the matching image of the HUD image projected by the vehicle-mounted head-up display 100 and execute the method for determining the size of the HUD image.

[0071] Among them, when the vehicle-mounted head-up display 100 projects the HUD image, the HUD image is projected onto a black background to form a strong contrast between the HUD image and the background, so that when the binocular camera device 200 collects the to-be-matched image and the matching image of the HUD image projected by the vehicle-mounted head-up display 100, there is an obvious difference in the brightness of the HUD image and the background image in the to-be-matched image and the matching image, so as to facilitate the binocular camera device 200 to distinguish the HUD image and the background image in the to-be-matched image and the matching image.

[0072] like Figure 2 As shown, the binocular camera device 200 includes: a first lens 210, a second lens 220, a memory 230 and a processor 240, and the processor 240 is connected to the first lens 210, the second lens 220 and the memory 230 respectively. It should be understood by those skilled in the art that Figure 2 The structure shown does not constitute a limitation on the binocular camera device 200. The binocular camera device 200 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or adopt a different component arrangement.

[0073] Among them, the first lens 210 and the second lens 220 are respectively the left lens and the right lens of the binocular camera device 200, the left lens is used to capture the left image, and the right lens is used to capture the right image, and the left image and the right image are binocular images of the same scene at the same time under different viewing angles. Based on this, when the first lens 210 is a left lens, the first lens 210 is used to capture the left image, at this time, the second lens 220 is a right lens, and the second lens 220 is used to capture the right image; when the first lens 210 is a right lens, the first lens 210 is used to capture the right image, at this time, the second lens 220 is a left lens, and the second lens 220 is used to capture the left image.

[0074] The first lens 210 is used to capture the image to be matched of the HUD image, and the second lens 220 is used to capture the matching image of the HUD image, and the image to be matched and the matching image are the left image and the right image, respectively. When the first lens 210 is a left lens and the second lens 220 is a right lens, the image to be matched is the left image and the matching image is the right image; when the first lens 210 is a right lens and the second lens 220 is a left lens, the image to be matched is the right image and the matching image is the left image.

[0075] Since the image to be matched and the matching image are binocular images of the same scene at the same time in different perspectives, there are corresponding matching points between the image to be matched and the matching image, and there is parallax between the matching points of the image to be matched and the matching image. The binocular camera device 200 can determine the depth of the matching point based on the parallax of the matching point, and then use the depth of the matching point to calculate the actual spatial coordinates of the matching point.

[0076] Since the image to be matched and the matching image are binocular images of the same scene at the same time under different viewing angles, the size of the image to be matched and the matching image are the same, both M*N. Based on this, the image to be matched and the matching image are both composed of a number of pixels arranged in M ​​rows and N columns.

[0077] Wherein, the image to be matched and the matching image are RGB images, and each pixel in the image to be matched and the matching image is represented by an RGB color value.

[0078] The memory 230 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, instructions, and modules.

[0079] The memory 230 may include a program storage area and a data storage area. The program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the binocular camera device 200, etc. The data storage area may also store preset data.

[0080] In addition, the memory 230 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0081] In some embodiments, the memory 230 may optionally include a memory remotely disposed relative to the processor 240, and the remotely disposed memory may be connected to the binocular camera device 200 via a network. The above network includes but is not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0082] The processor 240 is the control center of the binocular camera device 200, and can use various interfaces and lines to connect various parts of the entire binocular camera device 200, and execute various functions of the binocular camera device 200 and process data by running or executing non-volatile software programs, instructions and modules stored in the memory 230, and calling data stored in the memory 230, thereby controlling the binocular camera device 200 as a whole. For example, the method for determining the HUD image size described in any embodiment of the present invention is implemented.

[0083] The processor 240 may be one or more, Figure 2 A processor 240 is taken as an example.

[0084] The processor 240 may be connected to the first lens 210, the second lens 220, and the memory 230 via a bus or other means. Figure 2 The example of connecting through bus is taken in the following.

[0085] The processor 240 may include a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA), etc. The processor 240 may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0086] It can be understood that when the processor 240 executes the method for determining the size of the HUD image, the processor 240 obtains the image to be matched of the HUD image from the first lens 210 and obtains the matching image of the HUD image from the second lens 220; then, the processor 240 determines the first characteristic element of the HUD image in the image to be matched and extracts the first pixel position of the first characteristic element in the image to be matched, and at the same time, the processor 240 determines the second pixel position of the first characteristic element in the matching image; then, the processor 240 calculates the actual space coordinates of the first characteristic element according to the first pixel position and the second pixel position, and calculates the size of the HUD image according to the actual space coordinates, so that the binocular camera device 200 can calculate the size of the HUD image by obtaining the image to be matched and the matching image of the HUD image, and can determine the size of the HUD image without manually measuring the HUD image, thereby improving the efficiency of measuring the size of the HUD image.

[0087] In some embodiments, the binocular camera device 200 further includes a display module, which is used to display information provided to the user, and / or display various menus of the binocular camera device, and / or accept user input. In an embodiment of the present invention, the display module can display the determined HUD image size to the user.

[0088] Of course, in some alternative embodiments, the binocular camera device 200 can realize the function of the display module by connecting to the terminal device, such as displaying information provided to the user, and / or displaying various menus of the binocular camera device, and / or accepting user input. In an embodiment of the present invention, the binocular camera device 200 can display the determined HUD image size to the user through the terminal device.

[0089] For further information, see Figure 3 , is a flow chart of a method for determining the size of a HUD image provided by an embodiment of the present invention. The method for determining the size of a HUD image can be applied to the above-mentioned binocular camera device 200, and as one of the specific implementation methods, the method for determining the size of a HUD image is executed by the above-mentioned processor 240, so that the binocular camera device 200 can calculate the size of the HUD image, thereby improving the efficiency of measuring the size of the HUD image.

[0090] Specifically, the method for determining the HUD image size includes:

[0091] S100: Acquire an image to be matched and a matching image of a HUD image.

[0092] In the embodiment of the present invention, the image to be matched and the matching image of the HUD image are binocular images of the HUD image at different viewing angles at the same moment, and both the image to be matched and the matching image include the HUD image and the background image, and there is an obvious difference in brightness between the HUD image and the background image in the image to be matched and the matching image, so that the binocular camera device 200 can easily distinguish the HUD image and the background image in the image to be matched and the matching image. Among them, since the HUD image is projected onto a black background, the brightness of the HUD image in the image to be matched and the matching image is greater than the brightness of the background image.

[0093] Since the image to be matched and the matching image are binocular images of the HUD image at the same moment under different viewing angles, there are corresponding matching points between the image to be matched and the matching image, and there is parallax between the matching points of the image to be matched and the matching image.

[0094] Since the image to be matched and the matching image are binocular images of the HUD image at the same moment under different viewing angles, the size of the image to be matched and the matching image are the same, both M*N. Based on this, the image to be matched and the matching image are both composed of a number of pixels arranged in M ​​rows and N columns.

[0095] The image to be matched and the matching image are both RGB images, so each pixel in the image to be matched and the matching image is represented by an RGB color value.

[0096] In the embodiment of the present invention, the image to be matched of the HUD image is acquired through the first lens 210 , and the matching image of the HUD image is acquired through the second lens 220 .

[0097] Before acquiring the image to be matched and the matching image of the HUD image, the binocular camera device 200 will be calibrated to put the binocular camera device 200 in an ideal state. When the binocular camera device 200 is in an ideal state, the parallax of the matching point between the image to be matched and the matching image captured by the binocular camera device 200 is only related to the horizontal position difference between the first lens 210 and the second lens 220 of the binocular camera device 200. At the same time, by calibrating the binocular camera device 200, the physical size of the unit pixel in the binocular image captured by the binocular camera device 200 is determined.

[0098] S200: Determine a first characteristic element of the HUD image in the image to be matched.

[0099] In an embodiment of the present invention, the first characteristic element of the HUD image includes a characteristic corner point or a boundary characteristic point of the HUD image, and the characteristic corner point or the boundary characteristic point is a feature of the HUD image in the image to be matched. Based on this, the first characteristic element of the HUD image is determined in the image to be matched, that is, the characteristic corner point of the HUD image is determined in the image to be matched, or the boundary characteristic point of the HUD image is determined in the image to be matched.

[0100] Specifically, when determining the first characteristic element of the HUD image in the image to be matched, first, obtain a binary image of the image to be matched; then, determine the second characteristic element of the HUD image in the binary image; finally, based on the second characteristic element, determine the first characteristic element corresponding to the second characteristic element in the image to be matched.

[0101] Since the image to be matched is an RGB image, when obtaining the binary image of the image to be matched, first, the image to be matched is gray-processed to obtain a gray-scale image; then, the gray-scale image is binarized to obtain a binary image.

[0102] Among them, by converting the RGB color value of each pixel in the image to be matched into a grayscale value, a grayscale image of the image to be matched is obtained, and the grayscale image can reflect the brightness and darkness changes in the image to be matched. In the grayscale image of the image to be matched, the grayscale value of the pixel of the HUD image with higher brightness is greater than the grayscale value of the pixel of the background image with lower brightness.

[0103] Based on this, a grayscale threshold for distinguishing pixels of the HUD image from pixels of the background image can be set. When the grayscale value of a pixel in the grayscale image is greater than the grayscale threshold, the grayscale value of the pixel is converted to a first grayscale value. When the grayscale value of a pixel in the grayscale image is less than the grayscale threshold, the grayscale value of the pixel is converted to a second grayscale value to obtain a binary image consisting of the first grayscale value and the second grayscale value. Preferably, the first grayscale value is 255 and the second grayscale value is 0.

[0104] Since the grayscale value of the pixel of the HUD image is greater than the grayscale value of the pixel of the background image, in the binary image, the first grayscale value is used to indicate the HUD image, and the second grayscale value is used to indicate the background image.

[0105] In an embodiment of the present invention, the second feature element of the HUD image includes a reference feature corner point or a reference boundary feature point of the HUD image, where the reference feature corner point or the reference boundary feature point is a feature of the HUD image in the binary image.

[0106] Among them, the second feature element corresponds to the first feature element. When the first feature element is a feature corner point, the reference feature corner point of the HUD image is determined in the binary image. At this time, based on the reference feature corner point, the feature corner point corresponding to the reference feature corner point is determined in the image to be matched; when the first feature element is a boundary feature point, the reference boundary feature point of the HUD image is determined in the binary image. At this time, based on the reference boundary feature point, the boundary feature point corresponding to the reference boundary feature point is determined in the image to be matched, and the boundary feature point includes an upper boundary feature point, a lower boundary feature point, a left boundary feature point and a right boundary feature point.

[0107] Since there is a one-to-one correspondence between each pixel point in the binary image of the image to be matched and each pixel point in the image to be matched, the first feature element is composed of the pixel points in the image to be matched that correspond to each pixel point of the second feature element.

[0108] Based on this, based on the reference feature corner points, when determining the feature corner points corresponding to the reference feature corner points in the image to be matched, the features composed of the pixel points corresponding to the respective pixel points of the reference feature corner points in the image to be matched are determined as the feature corner points; based on the reference boundary feature points, when determining the boundary feature points corresponding to the reference boundary feature points in the image to be matched, the features composed of the pixel points corresponding to the respective pixel points of the reference boundary feature points in the image to be matched are determined as the boundary feature points.

[0109] Furthermore, since the HUD image is indicated by the first grayscale value in the binary image, when determining the second characteristic element of the HUD image in the binary image, first, the area in the binary image where the most first grayscale values ​​exist continuously is determined as the visible area; then, the edge pixels of the visible area are extracted; finally, the second characteristic element of the HUD image is determined based on the edge pixels.

[0110] Since ambient light will generate noise on the background during the HUD image projection process, affecting the distinction between the HUD image and the background image, in order to eliminate the influence of noise on the distinction between the HUD image and the background image, the area with the most continuous first grayscale values ​​in the binary image is determined as the visible area, and other areas with fewer continuous first grayscale values ​​are considered to be caused by noise. Figure 4 In the binary image shown, the number of first gray values ​​that continuously exist in region A is greater than the number of first gray values ​​that continuously exist in region B, so region A is determined to be a visible region.

[0111] After the visible area is determined in the binary image, the edge pixels of the visible area are extracted using a boundary extraction algorithm.

[0112] At this time, the second characteristic element of the HUD image is determined according to the edge pixel points, specifically including:

[0113] S1. Determine the upper edge, lower edge, left edge and right edge of the visible area according to the edge pixels.

[0114] Specifically, when determining the upper edge, lower edge, left edge and right edge of the visible area according to the edge pixel points, determine the third pixel position of the central pixel point of the visible area, and divide the edge pixel points into upper edge pixel points, lower edge pixel points, left edge pixel points and right edge pixel points according to the third pixel position; fit the upper edge of the visible area according to the upper edge pixel points, fit the lower edge of the visible area according to the lower edge pixel points, fit the left edge of the visible area according to the left edge pixel points, and fit the right edge of the visible area according to the right edge pixel points.

[0115] Among them, the central pixel point of the visible area is the pixel point located at the midpoint position of the visible area; the third pixel position of the central pixel point is the row position and column position of the central pixel point in the binary image, that is, the third pixel position includes the central row position and the central column position.

[0116] Based on this, when the edge pixels are divided into upper edge pixels, lower edge pixels, left edge pixels and right edge pixels according to the third pixel position, the pixel points whose row positions are smaller than the center row positions are determined as upper edge pixels, the pixel points whose row positions are larger than the center row positions are determined as lower edge pixels, the pixel points whose column positions are smaller than the center column positions are determined as left edge pixels, and the pixel points whose column positions are larger than the center column positions are determined as right edge pixels.

[0117] S2. Determine the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point of the visible area according to the upper edge, the lower edge, the left edge, and the right edge of the visible area.

[0118] Specifically, when determining the upper left corner point, upper right corner point, lower left corner point and lower right corner point of the visible area based on the upper edge, lower edge, left edge and right edge of the visible area, the intersection point of the upper edge and the left edge of the visible area is determined as the upper left corner point of the visible area, the intersection point of the upper edge and the right edge of the visible area is determined as the upper right corner point of the visible area, the intersection point of the lower edge and the left edge of the visible area is determined as the lower left corner point of the visible area, and the intersection point of the lower edge and the right edge of the visible area is determined as the lower right corner point of the visible area.

[0119] Among them, since there is a fitting error in the fitting process when fitting the upper edge of the visible area according to the upper edge pixel points, fitting the lower edge of the visible area according to the lower edge pixel points, fitting the left edge of the visible area according to the left edge pixel points, and fitting the right edge of the visible area according to the right edge pixel points, there is at least one line segment in the fitted upper edge, lower edge, left edge and right edge of the visible area respectively. Based on this, when determining the upper left corner point, upper right corner point, lower left corner point and lower right corner point of the visible area according to the upper edge, lower edge, left edge and right edge of the visible area, the number of upper left corner points can be determined according to the product of the number of line segments of the upper edge of the visible area and the number of line segments of the left edge, the number of upper right corner points can be determined according to the product of the number of line segments of the upper edge of the visible area and the number of line segments of the right edge, the number of lower left corner points can be determined according to the product of the number of line segments of the lower edge of the visible area and the number of line segments of the left edge, and the number of lower right corner points can be determined according to the product of the number of line segments of the lower edge of the visible area and the number of line segments of the right edge.

[0120] For example, when the number of line segments of the upper edge, the number of line segments of the lower edge, the number of line segments of the left edge, and the number of line segments of the right edge are all 1, then the number of upper left corner points, the number of upper right corner points, the number of lower left corner points, and the number of lower right corner points are all 1.

[0121] For another example, when the number of line segments at the upper edge is 2, the number of line segments at the lower edge is 1, the number of line segments at the left edge is 3, and the number of line segments at the right edge is 2, the number of upper left corner points is 6, the number of upper right corner points is 4, the number of lower left corner points is 3, and the number of lower right corner points is 2.

[0122] S3. Calculate the credibility of the upper left corner, upper right corner, lower left corner and lower right corner of the visible area.

[0123] Specifically, when calculating the credibility of the upper left corner, upper right corner, lower left corner and lower right corner of the visible area, first, calculate the credibility of the upper edge, lower edge, left edge and right edge of the visible area respectively; then, the credibility of the upper edge of the visible area and the credibility of the left edge are multiplied to determine the credibility of the upper left corner of the visible area, the credibility of the upper edge of the visible area and the credibility of the right edge are multiplied to determine the credibility of the upper right corner of the visible area, the credibility of the lower edge of the visible area and the credibility of the left edge are multiplied to determine the credibility of the lower left corner of the visible area, and the credibility of the lower edge of the visible area and the credibility of the right edge are multiplied to determine the credibility of the lower right corner of the visible area.

[0124] Among them, since the upper left corner point of the visible area is determined according to the intersection of the upper edge and the left edge of the visible area, the upper right corner point of the visible area is determined according to the intersection of the upper edge and the right edge of the visible area, the lower left corner point of the visible area is determined according to the intersection of the lower edge and the left edge of the visible area, and the lower right corner point of the visible area is determined according to the intersection of the lower edge and the right edge of the visible area, the credibility of the upper left corner point, upper right corner point, lower left corner point and lower right corner point of the visible area can be determined according to the credibility of the upper edge, lower edge, left edge and right edge of the visible area.

[0125] Since there is at least one line segment at each of the fitted upper edge, lower edge, left edge and right edge of the visible area, the credibility of the upper edge, lower edge, left edge and right edge of the visible area is calculated respectively, that is, the credibility of each line segment at the upper edge of the visible area, the credibility of each line segment at the lower edge of the visible area, the credibility of each line segment at the left edge of the visible area, and the credibility of each line segment at the right edge of the visible area are calculated.

[0126] Among them, when calculating the credibility of each line segment of the upper edge of the visible area, the first length of each line segment of the upper edge is determined, and the sum of the first lengths of all the line segments of the upper edge is determined; the ratio of the first length of each line segment of the upper edge to the sum of the first lengths is determined as the credibility of each line segment of the upper edge. For example, when the upper edge includes line segment 1 and line segment 2, the first length of line segment 1 is determined to be A, the first length of line segment 2 is determined to be B, and the sum of the first length A of line segment 1 and the first length B of line segment 2 is determined as the sum of the first lengths of all the line segments of the upper edge. At this time, the ratio of the first length A of line segment 1 to the sum of the first lengths is the credibility of line segment 1, and the ratio of the first length B of line segment 2 to the sum of the first lengths is the credibility of line segment 2.

[0127] When calculating the credibility of each line segment of the lower edge of the visible area, the second length of each line segment of the lower edge is determined, and the sum of the second lengths of all line segments of the lower edge is determined; the ratio of the second length of each line segment of the lower edge to the sum of the second lengths is determined as the credibility of each line segment of the lower edge. For example, when the lower edge includes line segment 3, the second length of line segment 3 is determined to be C, and the second length C of line segment 3 is determined as the sum of the second lengths of all line segments of the lower edge. At this time, the ratio of the second length C of line segment 3 to the sum of the second lengths is the credibility of line segment 3.

[0128] When calculating the credibility of each line segment on the left edge of the visible area, the third length of each line segment on the left edge is determined, and the sum of the third lengths of all line segments on the left edge is determined; the ratio of the third length of each line segment on the left edge to the sum of the third lengths is determined as the credibility of each line segment on the left edge. For example, when the left edge includes line segment 4, line segment 5, and line segment 6, the third length of line segment 4 is determined to be D, the third length of line segment 5 is determined to be E, and the third length of line segment 6 is determined to be F, and the sum of the third length D of line segment 4, the third length E of line segment 5, and the third length F of line segment 6 is determined as the sum of the third lengths of all line segments on the left edge. At this time, the ratio of the third length D of line segment 4 to the sum of the third lengths is the credibility of line segment 4, the ratio of the third length E of line segment 5 to the sum of the third lengths is the credibility of line segment 5, and the ratio of the third length F of line segment 6 to the sum of the third lengths is the credibility of line segment 6.

[0129] When calculating the credibility of each line segment on the right edge of the visible area, the fourth length of each line segment on the right edge is determined, and the sum of the fourth lengths of all line segments on the right edge is determined; the ratio of the fourth length of each line segment on the right edge to the sum of the fourth lengths is determined as the credibility of each line segment on the right edge. For example, when the right edge includes line segment 7 and line segment 8, the fourth length of line segment 7 is determined to be G, the fourth length of line segment 8 is determined to be H, and the fourth length G of line segment 7 and the fourth length H of line segment 8 are determined as the sum of the fourth lengths of all line segments on the right edge. At this time, the ratio of the fourth length G of line segment 7 to the sum of the fourth lengths is the credibility of line segment 7, and the ratio of the fourth length H of line segment 8 to the sum of the fourth lengths is the credibility of line segment 8.

[0130] At this time, the product of the credibility of each line segment at the upper edge of the visible area and the credibility of each line segment at the left edge of the visible area is the credibility of each upper left corner point of the visible area; the product of the credibility of each line segment at the upper edge of the visible area and the credibility of each line segment at the right edge of the visible area is the credibility of each upper right corner point of the visible area; the product of the credibility of each line segment at the lower edge of the visible area and the credibility of each line segment at the left edge of the visible area is the credibility of each lower left corner point of the visible area; the product of the credibility of each line segment at the lower edge of the visible area and the credibility of each line segment at the right edge of the visible area is the credibility of each lower right corner point of the visible area.

[0131] For example, when the upper edge includes line segment 1 and line segment 2, and the left edge includes line segment 4, line segment 5, and line segment 6, the upper left corner point includes upper left corner point AF, wherein upper left corner point A is the intersection point of line segment 1 and line segment 4, upper left corner point B is the intersection point of line segment 1 and line segment 5, upper left corner point C is the intersection point of line segment 1 and line segment 6, upper left corner point D is the intersection point of line segment 2 and line segment 4, upper left corner point E is the intersection point of line segment 2 and line segment 5, and upper left corner point F is the intersection point of line segment 2 and line segment 6. Then, the product of the credibility of line segment 1 and the credibility of line segment 4 can be obtained. The product of the credibility of line segment 1 and the credibility of line segment 5 is determined as the credibility of the upper left corner point A, the product of the credibility of line segment 1 and the credibility of line segment 5 is determined as the credibility of the upper left corner point B, the product of the credibility of line segment 1 and the credibility of line segment 6 is determined as the credibility of the upper left corner point C, the product of the credibility of line segment 2 and the credibility of line segment 4 is determined as the credibility of the upper left corner point D, the product of the credibility of line segment 2 and the credibility of line segment 5 is determined as the credibility of the upper left corner point E, and the product of the credibility of line segment 2 and the credibility of line segment 6 is determined as the credibility of the upper left corner point F.

[0132] S4. Determine a second characteristic element of the HUD image based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point of the visible area.

[0133] Specifically, when determining the second characteristic element of the HUD image based on the credibility of the upper left corner, upper right corner, lower left corner and lower right corner points of the visible area, the offset upper left corner point is determined based on the credibility of the upper left corner point of the visible area; the offset upper right corner point is determined based on the credibility of the upper right corner point of the visible area; the offset lower left corner point is determined based on the credibility of the lower left corner point of the visible area; the offset lower right corner point is determined based on the credibility of the lower right corner point of the visible area; the second characteristic element of the HUD image is determined based on the offset upper left corner point, the offset upper right corner point, the offset lower left corner point and the offset lower right corner point.

[0134] Among them, based on the credibility of the upper left corner point of the visible area, when determining the offset upper left corner point, the number of upper left corner points of the visible area is counted and the pixel position of each upper left corner point of the visible area is obtained; based on the credibility of each upper left corner point, the pixel position of each upper left corner point is weightedly calculated to obtain the first weighted pixel position of each upper left corner point; the ratio of the sum of the first weighted pixel positions of all upper left corner points to the number of upper left corner points is determined as the first offset pixel position; and the offset upper left corner point is determined according to the first offset pixel position.

[0135] For example, when the upper left corner point of the visible area includes the upper left corner point AF, the number of upper left corner points is 6, the pixel position of the upper left corner point A is (x1, y1), the pixel position of the upper left corner point B is (x2, y2), the pixel position of the upper left corner point C is (x3, y3), the pixel position of the upper left corner point D is (x4, y4), the pixel position of the upper left corner point E is (x5, y5), and the pixel position of the upper left corner point F is (x6, y6). If the credibility of the upper left corner point A is k1, the credibility of the upper left corner point B is k2, the credibility of the upper left corner point C is k3, the credibility of the upper left corner point D is k4, the credibility of the upper left corner point E is k5, and the credibility of the upper left corner point F is k6, then (k1x1, k1y1) is determined as the first weighted pixel position of the upper left corner point A, and (k2x2, k2y2) is determined as the first weighted pixel position of the upper left corner point B, (k3x3, k3y3) is determined as the first weighted pixel position of the upper left corner point C, (k4x4, k4y4) is determined as the first weighted pixel position of the upper left corner point D, (k5x5, k5y5) is determined as the first weighted pixel position of the upper left corner point E, and (k6x6, k6y6) is determined as the first weighted pixel position of the upper left corner point F. Then, ((k1x1+k2x2+k3x3+k4x4+k5x5+k6x6) / 6, (k1y1+k2y2+k3y3+k4y4+k5y5+k6y6) / 6) is determined as the first offset pixel position, and the pixel point corresponding to the first offset pixel position is the offset upper left corner point.

[0136] Based on the credibility of the upper right corner point of the visible area, when determining the offset upper right corner point, the number of the upper right corner points of the visible area is counted and the pixel position of each upper right corner point of the visible area is obtained; based on the credibility of each upper right corner point, the pixel position of each upper right corner point is weightedly calculated to obtain the second weighted pixel position of each upper right corner point; the ratio of the sum of the second weighted pixel positions of all the upper right corner points to the number of the upper right corner points is determined as the second offset pixel position; and the offset upper right corner point is determined according to the second offset pixel position.

[0137] Based on the credibility of the lower left corner point of the visible area, when determining the offset lower left corner point, the number of the lower left corner points of the visible area is counted and the pixel position of each lower left corner point of the visible area is obtained; based on the credibility of each lower left corner point, the pixel position of each lower left corner point is weightedly calculated to obtain the third weighted pixel position of each lower left corner point; the ratio of the sum of the third weighted pixel positions of all lower left corner points to the number of lower left corner points is determined as the third offset pixel position; and the offset lower left corner point is determined according to the third offset pixel position.

[0138] Based on the credibility of the lower right corner point of the visible area, when determining the offset lower right corner point, the number of the lower right corner points of the visible area is counted and the pixel position of each lower right corner point of the visible area is obtained; based on the credibility of each lower right corner point, the pixel position of each lower right corner point is weightedly calculated to obtain the fourth weighted pixel position of each lower right corner point; the ratio of the sum of the fourth weighted pixel positions of all lower right corner points to the number of lower right corner points is determined as the fourth offset pixel position; and the offset lower right corner point is determined according to the fourth offset pixel position.

[0139] When determining the second characteristic element of the HUD image according to the offset upper left corner point, the offset upper right corner point, the offset lower left corner point and the offset lower right corner point, if the second characteristic element is a reference characteristic corner point, the offset upper left corner point, the offset upper right corner point, the offset lower left corner point and the offset lower right corner point are determined as the reference characteristic corner points of the HUD image; if the second characteristic element is a reference boundary characteristic point, the offset upper left corner point and the offset upper right corner point are connected to obtain the upper boundary, the offset lower left corner point and the offset lower right corner point are connected to obtain the lower boundary, the offset upper left corner point and the offset lower left corner point are connected to obtain the left boundary, and the offset upper right corner point is connected to obtain the reference boundary. The right boundary is obtained by offsetting the lower right corner point, and the upper boundary, lower boundary, left boundary and right boundary are expanded. The reference upper boundary feature points of the HUD image are extracted from the upper boundary after the expansion process, the reference lower boundary feature points of the HUD image are extracted from the lower boundary after the expansion process, the reference left boundary feature points of the HUD image are extracted from the left boundary after the expansion process, and the reference right boundary feature points of the HUD image are extracted from the right boundary after the expansion process. The reference upper boundary feature points, the reference lower boundary feature points, the reference left boundary feature points and the reference right boundary feature points are determined as reference boundary feature points.

[0140] S300: extracting a first pixel position of a first feature element in the image to be matched;

[0141] S400: Determine a second pixel position of the first feature element in the matching image.

[0142] The first pixel position is the row and column position of the pixel point in the image to be matched, and the second pixel position is the row and column position of the pixel point in the matching image.

[0143] Based on this, the first pixel position of the first feature element in the image to be matched is extracted, that is, the first pixel position of each pixel point in the first feature element in the image to be matched is extracted.

[0144] When determining the second pixel position of the first feature element in the matching image, matching points are searched in the matching image according to the respective pixel points of the first feature element in the image to be matched, and the pixel position of the matching point in the matching image that matches the respective pixel points of the first feature element is determined as the second pixel position.

[0145] S500: Calculating the actual space coordinates of the first feature element according to the first pixel position and the second pixel position;

[0146] S600: Calculate the size of the HUD image according to the actual space coordinates.

[0147] In the embodiment of the present invention, the spatial coordinates of the points in the actual HUD image corresponding to each pixel point in the first feature element in the camera coordinate system are referred to as the actual spatial coordinates of the first feature element. For example, when the first feature element includes the upper left corner point, the spatial coordinates of the upper left corner point in the actual HUD image in the camera coordinate system are the actual spatial coordinates of the upper left corner point.

[0148] When the first characteristic element is a characteristic corner point, after determining the first pixel position and the second pixel position of each characteristic corner point, the actual space coordinates of each characteristic corner point are determined in turn according to the first pixel position and the second pixel position of each characteristic corner point; the size of the HUD image is calculated according to the actual space coordinates of each characteristic corner point. For example, the actual space coordinates of the upper left corner point are determined according to the first pixel position and the second pixel position of the upper left corner point, the actual space coordinates of the upper right corner point are determined according to the first pixel position and the second pixel position of the upper right corner point, the actual space coordinates of the lower left corner point are determined according to the first pixel position and the second pixel position of the lower left corner point, and the actual space coordinates of the lower right corner point are determined according to the first pixel position and the second pixel position of the lower right corner point; then, the size of the HUD image is calculated according to the actual space coordinates of the upper left corner point, the actual space coordinates of the upper right corner point, the actual space coordinates of the lower left corner point, and the actual space coordinates of the lower right corner point.

[0149] Among them, the actual space coordinates of each feature corner point are determined in turn according to the first pixel position and the second pixel position of each feature corner point, specifically including: determining the offset pixel of each feature corner point according to the first pixel position and the second pixel position of each feature corner point; determining the parallax of each feature corner point according to the offset pixel of each feature corner point and the physical size of the unit pixel; calculating the depth of each feature corner point according to the horizontal distance between the first lens and the second lens, the focal length of the binocular camera device and the parallax of each feature corner point; determining the coordinates of each feature corner point in the image coordinate system; calculating the actual space coordinates of each feature corner point according to the focal length of the binocular camera device, the depth of each feature corner point and the coordinates of each feature corner point in the image coordinate system.

[0150] Specifically, in an embodiment of the present invention, a camera coordinate system is constructed with the center of the first lens as the origin, the horizontal direction of the first lens and the second lens as the x-axis direction, the direction perpendicular to the x-axis direction in the imaging plane as the y-axis direction, and the direction perpendicular to the x-axis direction in the imaging plane as the z-axis direction. An image coordinate system is constructed with the intersection of the optical axis of the first lens and the imaging plane as the origin, the horizontal direction of the first lens and the second lens as the x-axis direction, and the direction perpendicular to the x-axis direction in the imaging plane as the y-axis direction. That is, the image coordinate system is parallel to the camera coordinate system, and the image coordinate system is translated by a preset distance in the z-axis direction relative to the camera coordinate system, and the preset distance is the focal length of the camera. Then, according to the focal length of the binocular camera device, the depth of each feature corner point, and the x-coordinate of each feature corner point in the image coordinate system, the x-coordinate of the actual space coordinate of each feature corner point can be calculated; according to the focal length of the binocular camera device, the depth of each feature corner point, and the y-coordinate of each feature corner point in the image coordinate system, the y-coordinate of the actual space coordinate of each feature corner point can be calculated; the depth of each feature corner point is determined as the z-coordinate of the actual space coordinate of each feature corner point. The trigonometric proportionality principle is used to calculate the x-coordinate and y-coordinate of the actual space coordinates of each feature corner point.

[0151] The size of the HUD image is calculated according to the actual space coordinates of each feature corner point, specifically including: extracting the first y coordinate of the actual space coordinates of any one of the upper left corner point or the upper right corner point and the second y coordinate of the actual space coordinates of any one of the lower left corner point or the lower right corner point from the actual space coordinates of the upper left corner point, the actual space coordinates of the upper right corner point, the actual space coordinates of the lower left corner point, and the actual space coordinates of the lower right corner point, and determining the difference between the first y coordinate and the second y coordinate as the width of the HUD image; extracting the first x coordinate of the actual space coordinates of any one of the upper left corner point or the lower left corner point and the second x coordinate of the actual space coordinates of any one of the upper right corner point or the lower right corner point, and determining the difference between the first x coordinate and the second x coordinate as the width and length of the HUD image, to obtain the size of the HUD image.

[0152] When the first characteristic element is a boundary characteristic point, after determining the first pixel position and the second pixel position of each characteristic point in each boundary, the actual space coordinates of each boundary are determined in turn according to the first pixel position and the second pixel position of each characteristic point in each boundary; the size of the HUD image is calculated according to the actual space coordinates of each boundary. For example, the actual space coordinates of the upper boundary are determined according to the first pixel position and the second pixel position of each characteristic point in the upper boundary, the actual space coordinates of the lower boundary are determined according to the first pixel position and the second pixel position of each characteristic point in the lower boundary, the actual space coordinates of the left boundary are determined according to the first pixel position and the second pixel position of each characteristic point in the left boundary, and the actual space coordinates of the right boundary are determined according to the first pixel position and the second pixel position of each characteristic point in the right boundary; then, the size of the HUD image is calculated according to the actual space coordinates of the upper boundary, the actual space coordinates of the lower boundary, the actual space coordinates of the left boundary, and the actual space coordinates of the right boundary.

[0153] Among them, the actual spatial coordinates of each boundary are determined in turn according to the first pixel position and the second pixel position of each feature point in each boundary, specifically including: determining the offset pixel mean of each boundary according to the first pixel position and the second pixel position of each feature point in each boundary; determining the disparity mean of each boundary according to the offset pixel mean of each boundary and the physical size of the unit pixel; calculating the depth of each boundary according to the horizontal distance between the first lens and the second lens, the focal length of the binocular camera device and the disparity mean of each boundary; determining the coordinates of each feature point in each boundary in the image coordinate system; calculating the spatial coordinates of each feature point in each boundary according to the focal length of the binocular camera device, the depth of each boundary and the coordinates of each feature point in each boundary in the image coordinate system; calculating the actual spatial coordinates of each boundary according to the spatial coordinates of each feature point in each boundary.

[0154] Specifically, in the embodiment of the present invention, a camera coordinate system is constructed with the center of the first lens as the origin, the horizontal direction of the first lens and the second lens as the x-axis direction, the direction perpendicular to the x-axis direction in the imaging plane as the y-axis direction, and the direction perpendicular to the imaging plane of the first lens as the z-axis direction. An image coordinate system is constructed with the intersection of the optical axis of the first lens and the imaging plane as the origin, the horizontal direction of the first lens and the second lens as the x-axis direction, and the direction perpendicular to the x-axis direction in the imaging plane as the y-axis direction. That is, the image coordinate system is parallel to the camera coordinate system, and the image coordinate system is relative to the camera coordinate system. Only a preset distance is translated in the z-axis direction, and the preset distance is the focal length of the binocular camera device. Then, according to the focal length of the binocular camera device, the depth of each boundary, and the x coordinate of each feature point in each boundary in the image coordinate system, the x coordinate of the spatial coordinate of each feature point in each boundary can be calculated; according to the focal length of the binocular camera device, the depth of each boundary, and the y coordinate of each feature point in each boundary in the image coordinate system, the y coordinate of the spatial coordinate of each feature point in each boundary can be calculated; the depth of each boundary is determined as the z coordinate of the spatial coordinate of each feature point in each boundary. When calculating the x coordinate and y coordinate of the spatial coordinate of each feature point in each boundary, the trigonometric proportional principle is used.

[0155] Furthermore, the mean of the spatial coordinates of each feature point on each boundary is determined as the actual spatial coordinates of each boundary.

[0156] Based on this, the size of the HUD image is calculated according to the actual space coordinates of each boundary, specifically including: extracting the third y coordinate of the actual space coordinate of the upper boundary and the fourth y coordinate of the actual space coordinate of the lower boundary, and determining the difference between the third y coordinate and the fourth y coordinate as the width of the HUD image; extracting the third x coordinate of the actual space coordinate of the left boundary and the fourth x coordinate of the actual space coordinate of the right boundary, and determining the difference between the third x coordinate and the fourth x coordinate as the length of the HUD image.

[0157] It can be understood that in the embodiment of the present invention, the size of the HUD image can be calculated by acquiring the image to be matched and the matching image of the HUD image through a binocular camera device, and the size of the HUD image can be determined without manually measuring the HUD image, thereby improving the efficiency of measuring the size of the HUD image.

[0158] For further information, see Figure 5, is a structural schematic diagram of a device for determining the size of a HUD image provided by an embodiment of the present invention. The device for determining the size of a HUD image can be applied to the above-mentioned binocular camera device 200, and as one of the specific implementation methods, the functions of each module of the device for determining the size of a HUD image are executed by the above-mentioned processor 240, so that the binocular camera device 200 can calculate the size of the HUD image and improve the efficiency of measuring the size of the HUD image.

[0159] The term "module" used below refers to a combination of software and / or hardware that can implement a predetermined function. Although the devices described in the following embodiments can be implemented in software, implementation in hardware, or a combination of software and hardware, is also conceivable.

[0160] Specifically, the device for determining the HUD image size includes:

[0161] An acquisition module 10 is used to acquire the image to be matched and the matching image of the HUD image;

[0162] A first determining module 20, configured to determine a first characteristic element of the HUD image in the image to be matched;

[0163] An extraction module 30, configured to extract a first pixel position of the first feature element in the image to be matched;

[0164] A second determination module 40, configured to determine a second pixel position of the first feature element in the matching image;

[0165] A first calculation module 50, configured to calculate the actual spatial coordinates of the first feature element according to the first pixel position and the second pixel position;

[0166] The second calculation module 60 is used to calculate the size of the HUD image according to the actual space coordinates.

[0167] In some embodiments, the first determining module 20 is specifically configured to:

[0168] Acquire a binary image of the image to be matched;

[0169] Determining a second characteristic element of the HUD image in the binary image;

[0170] Based on the second feature element, the first feature element corresponding to the second feature element is determined in the image to be matched.

[0171] In some embodiments, the binary image includes a first grayscale value and a second grayscale value, and the first grayscale value is used to indicate the HUD image; then,

[0172] The first determining module 20 is specifically used for:

[0173] When determining the second characteristic element of the HUD image in the binary image, determining the area in the binary image where the first grayscale value exists most continuously as a visible area;

[0174] Extracting edge pixel points of the visible area;

[0175] A second characteristic element of the HUD image is determined according to the edge pixel point.

[0176] In some embodiments, the first determining module 20 is specifically configured to:

[0177] When determining the second characteristic element of the HUD image according to the edge pixel points, determining the upper edge, the lower edge, the left edge and the right edge of the visible area according to the edge pixel points;

[0178] Determine the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point of the visible area according to the upper edge, the lower edge, the left edge and the right edge;

[0179] Calculating the credibility of the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point;

[0180] Based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point, a second characteristic element of the HUD image is determined.

[0181] In some embodiments, the first determining module 20 is specifically configured to:

[0182] When determining the upper edge, the lower edge, the left edge and the right edge of the visible area according to the edge pixel points, determining a third pixel position of the central pixel point of the visible area;

[0183] Dividing the edge pixel points into upper edge pixel points, lower edge pixel points, left edge pixel points and right edge pixel points according to the third pixel position;

[0184] Fitting the upper edge according to the upper edge pixel points;

[0185] Fitting the lower edge according to the lower edge pixel points;

[0186] Fitting the left edge according to the left edge pixel points;

[0187] The right edge is fitted according to the right edge pixels.

[0188] In some embodiments, the first determining module 20 is specifically configured to:

[0189] When calculating the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point, the credibility of the upper edge, the lower edge, the left edge, and the right edge is calculated;

[0190] Determine the credibility of the upper left corner point by multiplying the credibility of the upper edge and the left edge;

[0191] Determine the credibility of the upper right corner point by multiplying the credibility of the upper edge and the right edge;

[0192] Determine the credibility of the lower left corner point by multiplying the credibility of the lower edge and the left edge;

[0193] The credibility of the lower right corner point is determined by multiplying the credibility of the lower edge and the right edge.

[0194] In some embodiments, the first determining module 20 is specifically configured to:

[0195] When determining the second characteristic element of the HUD image based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point, determining the corner point offset distance and the corner point offset direction based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point;

[0196] Determine the offset upper left corner point, the offset upper right corner point, the offset lower left corner point and the offset lower right corner point according to the pixel positions of the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point, as well as the corner point offset distance and the corner point offset direction;

[0197] A second characteristic element of the HUD image is determined based on the offset upper left corner point, the offset upper right corner point, the offset lower left corner point, and the offset lower right corner point.

[0198] In some embodiments, the first determining module 20 is specifically configured to:

[0199] When determining the corner point offset distance and the corner point offset direction based on the credibility of the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point, determine the corner point with the greatest credibility among the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point;

[0200] Calculating the credibility weights of the upper left corner point, the upper right corner point, the lower left corner point, and the lower right corner point;

[0201] Determine the corner point offset distance according to the credibility weight;

[0202] The corner point offset direction is determined according to the corner point with the greatest credibility.

[0203] Since the device embodiment and the method embodiment are based on the same concept, the content of the device embodiment may refer to the method embodiment under the premise that the contents do not conflict with each other, and they will not be described one by one here.

[0204] It can be understood that in the embodiment of the present invention, the size of the HUD image can be calculated by acquiring the image to be matched and the matching image of the HUD image through a binocular camera device, and the size of the HUD image can be determined without manually measuring the HUD image, thereby improving the efficiency of measuring the size of the HUD image.

[0205] The embodiment of the present invention further provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, such as Figure 2 A processor 240 in the embodiment may enable the computer to execute the steps of a method for determining the size of a HUD image in any of the above method embodiments, or implement the functions of various modules of a device for determining the size of a HUD image in any of the above device embodiments.

[0206] An embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-volatile computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by one or more processors, for example Figure 2 A processor 240 in the embodiment may enable the computer to execute the steps of a method for determining the size of a HUD image in any of the above method embodiments, or implement the functions of various modules of a device for determining the size of a HUD image in any of the above device embodiments.

[0207] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0208] Through the description of the above embodiments, a person of ordinary skill in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. A person of ordinary skill in the art can understand that all or part of the processes in the above embodiment methods can be completed by hardware related to computer program instructions, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the implementation methods of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0209] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining the size of a HUD image, applied to a binocular camera device, characterized in that: The method comprises: Acquire the image to be matched and the matching image of the HUD image; Acquire a binary image of the image to be matched, the binary image comprising a first grayscale value and a second grayscale value, the first grayscale value being used to indicate the HUD image; determine an area in the binary image where the first grayscale value exists most continuously as a visible area, extract edge pixel points of the visible area; determine an edge of the visible area according to the edge pixel points, determine a corner point of the visible area according to the edge, and calculate the credibility of the corner point; determine an offset corner point based on the credibility of the corner point, and determine a second characteristic element of the HUD image based on the offset corner point; Based on the second feature element, determining a first feature element corresponding to the second feature element in the image to be matched; Extracting a first pixel position of the first feature element in the image to be matched; Determining a second pixel position of the first feature element in the matching image; Calculating the actual space coordinates of the first feature element according to the first pixel position and the second pixel position; The size of the HUD image is calculated according to the actual space coordinates.

2. The method according to claim 1, characterized in that The determining the edge of the visible area according to the edge pixel points, determining the corner points of the visible area according to the edge, and calculating the credibility of the corner points specifically includes: Determine the upper edge, lower edge, left edge and right edge of the visible area according to the edge pixel points; Determine the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point of the visible area according to the upper edge, the lower edge, the left edge and the right edge; The credibility of the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point is calculated.

3. The method according to claim 2, characterized in that The determining the upper edge, the lower edge, the left edge and the right edge of the visible area according to the edge pixel points specifically includes: Determine a third pixel position of a central pixel of the visible area; Dividing the edge pixel points into upper edge pixel points, lower edge pixel points, left edge pixel points and right edge pixel points according to the third pixel position; Fitting the upper edge according to the upper edge pixel points; Fitting the lower edge according to the lower edge pixel points; Fitting the left edge according to the left edge pixel points; The right edge is fitted according to the right edge pixels.

4. The method according to claim 2 or 3, characterized in that: The calculating the credibility of the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point specifically includes: Calculating the credibility of the upper edge, the lower edge, the left edge and the right edge; Determine the credibility of the upper left corner point by multiplying the credibility of the upper edge and the left edge; Determine the credibility of the upper right corner point by multiplying the credibility of the upper edge and the right edge; Determine the credibility of the lower left corner point by multiplying the credibility of the lower edge and the left edge; The credibility of the lower right corner point is determined by multiplying the credibility of the lower edge and the right edge.

5. The method according to claim 4, characterized in that The step of determining the offset corner point based on the credibility of the corner point, and determining the second characteristic element of the HUD image based on the offset corner point specifically includes: Determine the offset upper left corner point based on the credibility of the upper left corner point; Determine the offset upper right corner point based on the credibility of the upper right corner point; Determine the offset lower left corner point based on the credibility of the lower left corner point; Determine the offset lower right corner point based on the credibility of the lower right corner point; A second characteristic element of the HUD image is determined according to the offset upper left corner point, the offset upper right corner point, the offset lower left corner point, and the offset lower right corner point.

6. A device for determining the size of a HUD image, applied to a binocular camera device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be matched and a matching image of the HUD image; A first determination module is used to obtain a binary image of the image to be matched, wherein the binary image includes a first grayscale value and a second grayscale value, and the first grayscale value is used to indicate the HUD image; determine the area in the binary image where the first grayscale value exists most continuously as a visible area, and extract edge pixel points of the visible area; determine the edge of the visible area according to the edge pixel points, determine the corner point of the visible area according to the edge, and calculate the credibility of the corner point; determine the offset corner point based on the credibility of the corner point, and determine the second characteristic element of the HUD image based on the offset corner point; based on the second characteristic element, determine the first characteristic element corresponding to the second characteristic element in the image to be matched; An extraction module, used for extracting a first pixel position of the first feature element in the image to be matched; A second determining module, used to determine a second pixel position of the first feature element in the matching image; A first calculation module, used for calculating the actual space coordinates of the first feature element according to the first pixel position and the second pixel position; The second calculation module is used to calculate the size of the HUD image according to the actual space coordinates.

7. A system for determining the size of a HUD image, characterized in that: include: An in-vehicle head-up display, wherein the in-vehicle head-up display is used to project the HUD image; A binocular camera device, the binocular camera device comprising a first lens, a second lens, at least one processor and a memory connected to the at least one processor in communication, The first lens is used to capture the image to be matched with the HUD image. The second lens is used to capture a matching image of the HUD image. The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to perform the method according to any one of claims 1 to 6.

8. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the binocular camera device to execute the method according to any one of claims 1 to 6.

Images