Apparatus for adjusting surround view image

Abstract

The present invention generates, by using K target field-of-view distances and K target field-of-view heights, K three-dimensional target field-of-view coordinates having three-dimensional coordinates, calculates K cost values by obtaining a cost value of each of the three-dimensional target field-of-view coordinates, expands a two-dimensional area in at least one of a first direction and a second direction by using the K cost values, changes, into a set value, field-of-view departure camera identification information corresponding to a field-of-view departure determination point, which is out of a field-of-view, among K pieces of field-of-view departure camera identification information, and if an i-th piece of field-of-view departure camera identification information among the K pieces of field-of-view departure camera identification information has a set value and an i-th cost value among the K cost values is equal to or less than an inner set value, reduces at least one of an i-th target field-of-view distance and an i-th target field-of-view height and generates a new i-th target field-of-view distance and i-th target field-of-view height.

Description

Surround view video adjustment device

[0001] The present invention relates to a surround view image adjustment device, and more specifically, to a surround view image adjustment device that automatically adjusts the output range of a surround view image.

[0002] Vehicle manufacturers install rear cameras to promote driver convenience and safe driving, enabling drivers to check the rear of the vehicle when reversing.

[0003] Recently, a Surround View System has been developed and is in use, equipped with cameras not only at the rear but also at the front, left, and right sides of the vehicle, allowing the driver to check the entire surroundings—front, back, left, and right—not only when parking but also while driving, thereby enabling safe and convenient driving operations.

[0004] To obtain such surround view images, multiple cameras are mounted at designated locations on the vehicle, calibration is performed to correct the installation positions of each mounted camera, and images acquired from each camera are combined to generate a surround view image that appears as if it were captured by a single virtual camera.

[0005] This generated surround view video is provided to the vehicle driver, helping to ensure the driver's visibility while driving.

[0006] However, when providing such surround view video to the driver, the resolution of the video output device varies depending on the type of vehicle, and the range of the field of view that each driver desires to have displayed in the desired video also differs.

[0007] Therefore, it becomes necessary to adjust the output state of the surround view video to suit these differences in resolution or the driver's requirements.

[0008] To adjust these surround view images, personnel at repair shops or manufacturers manually adjust the mesh slope. Consequently, this adjustment process is time-consuming, and the inconsistency in adjustment settings among different personnel leads to the problem of not providing uniform surround view images.

[0009] The problem that the present invention aims to solve is to enable the adjustment of surround view images to be performed automatically according to predetermined conditions.

[0010] A surround view image adjustment device for generating according to one feature of the present invention comprises a storage unit storing a two-dimensional area, k target viewing distances, k target viewing heights, and r angle of view deviation determination points, and an image adjustment unit connected to the storage unit. The image adjustment unit performs a three-dimensional coordinate transformation using the k target viewing distances and k target viewing heights, respectively, to generate k three-dimensional target viewing coordinates having three-dimensional coordinates, calculates k cost values ​​by obtaining cost values ​​for each three-dimensional target viewing coordinate, expands the two-dimensional area to at least one of a first direction and a second direction using the calculated k cost values, determines whether each angle of view deviation determination point exists within the actual camera's angle of view, changes the value of the corresponding angle of view deviation camera identification information among the k angle of view deviation camera identification information corresponding to the angle of view deviation determination point that is out of the angle of view to a set value, and among the k angle of view deviation camera identification information, the i-th angle of view deviation camera identification information has a set value, and among the k cost values, the value of the i-th cost is inner If it is equal to or smaller than the set value, at least one of the i-th target field of view distance and the i-th target field of view height is reduced by the adjustment size to create a new i-th target field of view distance and i-th target field of view height.

[0011] The above two-dimensional area may include a minimum two-dimensional area and a maximum two-dimensional area, and the image adjustment unit may extend the minimum two-dimensional area among the minimum two-dimensional area and the maximum two-dimensional area to at least one of the first direction and the second direction.

[0012] The expansion range of the above minimum 2-dimensional area may not be larger than the above maximum 2-dimensional area.

[0013] The image adjustment unit can convert each 3D target view coordinate into k 2D target view coordinates having 2D coordinates, and the value of the cost may be the length inward or outward deviation of the 2D target view coordinate from the edge of the display screen.

[0014] The image adjustment unit expands the two-dimensional area in the first direction such that, if at least one cost value related to the first direction among k cost values ​​is greater than the inner setting value, the cost having a smaller value among the costs related to the first direction is located on the inner setting range line determined to be the inner side of the display screen or is located closest to the inner setting range line, and if at least one cost value related to the second direction among k cost values ​​is greater than the inner setting value, the two-dimensional area expands the two-dimensional area in the second direction such that the cost having a smaller value among the costs related to the second direction is located on the inner setting range line determined to be the inner side of the display screen or is located closest to the inner setting range line, and the inner setting value may be the cost value of the inner setting range.

[0015] Among the i-th target viewing distance and the i-th target viewing height, if the i-th target viewing height is greater than the set size, the i-th target viewing height can be reduced by the adjustment size, and if the i-th target viewing height is less than or equal to the set size, the i-th target viewing distance can be reduced by the adjustment size.

[0016] The adjustment size for adjusting the i-th target field of view height and the adjustment size for adjusting the i-th target field of view length may be the same or different.

[0017] The above r angle of view deviation judgment points may include the four corners of the display screen, the center of each side of the display screen, and each point where the straight line in the first direction and the straight line in the second direction of the vehicle displayed on the display screen each intersect each side of the display screen.

[0018] The above image adjustment unit calculates a minimum cost value having the smallest value among the k cost values, and if the i-th angle of view deviation camera identification information does not have a set value and the i-th cost value exceeds an inner set value, the slope of the mesh to which a 2D area extended to at least one of the first direction and the second direction is applied can be increased or decreased using the minimum cost value.

[0019] The above image adjustment unit can reduce the slope of the mesh to which the 2D area is applied if the above minimum cost value is greater than the inner setting value.

[0020] The above image adjustment unit can increase the slope of the mesh to which a two-dimensional area extended to at least one of the first direction and the second direction is applied if the minimum cost value is not greater than the inner setting value and is smaller than the outer setting value.

[0021] The above outer setting value may be the cost value of the outer setting range, and the above outer setting range may be a range enclosed by the edges of the display screen.

[0022] According to these features, since the adjustment of the surround view video is performed automatically to suit the desired conditions, a significant amount of time is saved, and the uniformity of the quality of the adjusted and output surround view video can be improved.

[0023] In addition, since the surround view image can be easily, quickly, and accurately adjusted to match the driver's desired field of view, driver satisfaction and convenience can be improved.

[0024] FIG. 1 is a schematic block diagram of a surround view image adjustment system according to one embodiment of the present invention.

[0025] Figure 2 is a diagram illustrating an example of a mesh for a surround view image.

[0026] FIG. 3 is a diagram illustrating a minimum two-dimensional area and a maximum two-dimensional area in a surround view image adjustment system according to an embodiment of the present invention.

[0027] FIG. 4 is a drawing for explaining the first to fourth target viewing distances and the first to fourth target viewing heights in a surround view image adjustment system according to an embodiment of the present invention.

[0028] FIG. 5 is a drawing illustrating an example of first to fourth three-dimensional target view coordinates generated using the first to fourth target view distances and first to fourth target view heights of FIG. 4.

[0029] FIG. 6 is a diagram illustrating an example of a viewing angle deviation determination point in a surround view image adjustment system according to one embodiment of the present invention.

[0030] FIG. 7 is a diagram illustrating an example in which first to fourth three-dimensional target field of view coordinates are converted into first to fourth two-dimensional target field of view coordinates in a surround view image adjustment system according to an embodiment of the present invention.

[0031] FIG. 8 is a diagram illustrating the cost and cost values ​​of the first to fourth two-dimensional target field of view coordinates in a surround view image adjustment system according to an embodiment of the present invention.

[0032] FIGS. 9 to 11 are flowcharts of the operation sequence of a surround view image adjustment system according to one embodiment of the present invention.

[0033] FIG. 12 is an example of a screen output to an output unit to input information for minimum and maximum two-dimensional areas, and a target viewing distance and a target viewing height in a surround view image adjustment system according to an embodiment of the present invention.

[0034] FIG. 13 is an example of a screen illustrating a case where a subject located at a first target viewing distance and a first target viewing height is located within a display screen according to the operation of a surround view image adjustment system according to an embodiment of the present invention.

[0035] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols are assigned the same reference number, and redundant descriptions thereof will be omitted. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted.

[0036] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0037] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0038] In this application, each step described may be performed regardless of the order listed, except where it must be performed in the order listed by a particular causal relationship.

[0039] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] Hereinafter, a surround view image adjustment device according to an embodiment of the present invention will be described with reference to the attached drawings.

[0041] In the present example, the vehicle may be equipped with a plurality of actual cameras, e.g., first to fourth actual cameras, each attached to designated locations, e.g., front, rear, left, and right. Accordingly, each actual camera can generate corresponding shooting data through a shooting operation.

[0042] A virtual camera is a virtual camera having the same structure as a camera, and in this example, it can be assumed that one virtual camera exists.

[0043] Additionally, in this example, the virtual data may be image data based on a virtual camera, serving as basic image data for generating the surround view image data of this example. Such virtual data may be generated using the corresponding shooting data generated from each actual camera.

[0044] Accordingly, in the case of the present example, since there are first to fourth actual cameras mounted in the front, rear, left, and right directions, respectively, there may also be first to fourth virtual data corresponding to the shooting data captured by the first to fourth actual cameras, for example, front shooting data, rear shooting data, left shooting data, and right shooting data, for example, front virtual data, rear virtual data, left virtual data, and right virtual data.

[0045] Therefore, the surround view video data of the present example may be the entire surround view video data generated using these front virtual data, rear virtual data, left side virtual data, and right side virtual data.

[0046] It is assumed that a virtual camera is mounted at a designated location on the upper part of the vehicle (e.g., the sky). In this case, the surround view image, which is the image acquired when viewing the vehicle from the virtual camera at the designated location, may be the entire surround view image, and the image data corresponding to this entire surround view image may be the entire image data.

[0047] Therefore, multiple actual cameras attached to the vehicle are intended to capture video to generate a full surround view image.

[0048] Each virtual data, captured data, and entire image data may all be equipped with multiple pixels arranged in a matrix structure.

[0049] Next, with reference to FIGS. 1 to 4, a surround view image adjustment system of the present example is described for adjusting the surround view image of the present example (i.e., the entire surround view image) provided to a vehicle (100) so that a driver can view it, and which is equipped with a surround view image adjustment device (20) of the present example.

[0050] As illustrated in FIG. 1, the surround view video adjustment system of the present example may include a user input unit (10), a video adjustment device (20) connected to the user input unit (10), and an output unit (30) connected to the video adjustment device (20).

[0051] The user input section (10) is for inputting input information to adjust the output range of the surround view video.

[0052] These user input units (10) can be electrically and physically connected to the image adjustment device (20) using a connecting cable, etc., and can also be disconnected if necessary.

[0053] Accordingly, various types of connection ports (not shown) for connecting a connection cable may be located in the user input unit (10) and the video adjustment device (20).

[0054] In this way, when a user input unit (10) is connected to a video adjustment device (20), a user (e.g., a driver or mechanic, etc.) can input input information to adjust the output range of a surround view video using the user input unit (10).

[0055] Accordingly, the user input unit (10) can generate an electrical signal corresponding to the input information and output it to the image adjustment device (20).

[0056] For example, input information may be a minimum and maximum two-dimensional (2D) area, a target field of view distance, and a target field of view height.

[0057] In this example, the two-dimensional area may be an image of the floor surface where the vehicle (100) is located among the surround view images, and may be an area that is output to the image output device (not shown) of the vehicle (100) without distortion.

[0058] These two-dimensional areas can be designated as areas around the vehicle (100).

[0059] For example, as illustrated in FIG. 2, when the surround view image of the present example is generated by shooting data generated by actual cameras (not shown) installed at the front, rear, left, and right sides of the vehicle (100) in the position, the surround view image can be represented as a mesh having multiple levels.

[0060] The area enclosed by the lowest level among the multiple levels may be an area where an image of the floor surface that does not undergo distortion, that is, the surface in contact with the wheels of the vehicle (100), is displayed.

[0061] In this case, the gradient of the mesh can be the gradient of the curve in a mesh composed of curves and points.

[0062] A two-dimensional region can have vertical symmetry and horizontal symmetry relationships.

[0063] The minimum and maximum 2D areas of this example may be the minimum and maximum areas for the 2D area desired by the user (e.g., driver), respectively.

[0064] As illustrated in FIG. 3, the minimum two-dimensional area (AR11) can be determined by inputs of the minimum front-back distance (VDmin1) and the minimum left-right distance (HDmin1), and the maximum two-dimensional area (AR12) can be determined by inputs of the maximum front-back distance (VDmax1) and the maximum left-right distance (HDmax1).

[0065] Accordingly, the user can define a minimum two-dimensional area (AR11) by inputting a minimum front-back distance (VDmin1) and a minimum left-right distance (HDmin1) using the user input unit (10), and define a maximum two-dimensional area (AR12) by inputting a maximum front-back distance (VDmax1) and a maximum left-right distance (HDmax1).

[0066] The target viewing distance may be the minimum viewing distance that the driver wants to be displayed on an image output device attached to the interior of the vehicle (100), and the target viewing height may be the minimum viewing height that the driver wants to be displayed on an image output device.

[0067] Figure 4 illustrates an example of such a target field of view distance and target field of view height.

[0068] Referring to FIG. 4, the target viewing distance and target viewing height may be the distance (e.g., 10m) and height (160cm) to the front, rear, left, and right sides of the vehicle (100), respectively.

[0069] At this time, the forward target view distance (first forward target view distance) (FSD1) and the rear target view distance (second forward target view distance) (BSD1) (e.g., 10 m), which are the target view distances to the front and rear, may be the same, and the left target view distance (third forward target view distance) (LSD1) and the right target view distance (fourth forward target view distance) (RSD1) (e.g., 5 m), which are the target view distances to the left and right, may also be the same.

[0070] In addition, the target visibility heights at the front, rear, left, and right sides of the vehicle (100), such as the front target visibility height (FH1), rear target visibility distance (BH1), left target visibility height (LH1), and right target visibility height (RH1) (e.g., 160 cm), may all be the same.

[0071] In this way, when input information such as the values ​​of distances for minimum and maximum two-dimensional areas [e.g., front-rear minimum distance (VDmin1) and maximum distance (VDmax1) and left-right minimum distance (HDmin1) and maximum distance (HDmax1)] (hereinafter, the values ​​of distances for minimum and maximum two-dimensional areas are referred to as 'two-dimensional area distance range'), target viewing distance, and target viewing height is input from the user input unit (10), the user input unit (10) can output the input information to the image adjustment device (20).

[0072] In the present specification, the target viewing distance and the target viewing height may be target viewing information. Accordingly, the target viewing information may have first to fourth target viewing information each having a plurality of (e.g., k, k=1, 2, 3, 4) first to fourth target viewing distances and k first to fourth target viewing heights.

[0073] Accordingly, the image adjustment device (20) can perform an adjustment operation of the surround view image so that the surround view image corresponding to the target viewing distance and target viewing height desired by the driver can be output to the image output device.

[0074] The image adjustment device (20) may be equipped with an image adjustment unit (21) connected to a user input unit (10) and a memory (22) connected to the image adjustment unit (21).

[0075] The image adjustment unit (21) controls the overall operation of the image adjustment device (20) and may be a processor.

[0076] When the two-dimensional area distance range and target field of view information are input, this image adjustment unit (21) can perform a control operation to output a surround view image suitable for the two-dimensional area distance range and target field of view information.

[0077] The memory (22) may be a storage unit that stores data required for the operation of the image adjustment device (20) and data generated during operation.

[0078] This memory (22) may be at least one of a flash memory (22) type, a hard disk type, a multimedia card micro type, a card type memory (22) (e.g., SD or XD memory (22)), RAM (random access memory, RAM), SRAM (static random access memory, SRAM), ROM (read only memory, ROM), EEPROM (electrically erasable programmable read only memory, EEPROM), PROM (programmable read only memory, PROM), magnetic memory (22), a magnetic disk, and an optical disk.

[0079] The output unit (30) can operate under the control of the image adjustment unit (21) to generate an output related to at least one of visual and auditory.

[0080] This output unit (30) may be equipped with a display module and an audio output module.

[0081] The display module displays an image corresponding to the image data output from the image adjustment unit (21) on the display screen according to the operation of the image adjustment unit (21).

[0082] These display modules include at least one display device among a liquid crystal display, an organic light emitting diode display, a flexible display, and a 3D display.

[0083] The sound output module can output an audio signal output from the video adjustment unit (21) as voice by the operation of the video adjustment unit (21).

[0084] Next, with reference to FIGS. 5 to 11, the operation of a surround view image adjustment system having such a structure will be described.

[0085] First, when the operation of the surround view video adjustment system starts, a user such as a driver or mechanic can input minimum and maximum 2D areas and target field of view information to adjust the surround view video using the user input unit (10).

[0086] In this example, for inputting minimum and maximum two-dimensional areas and target field of view information (e.g., target field of view distance and target field of view height), the image adjustment unit (21) can output a user interface (UI) through the output unit (30), and the user can input the minimum and maximum two-dimensional areas and target field of view information using the user input unit (10) (see FIG. 12).

[0087] For the minimum 2D area (AR11), the front-back minimum distance (VDmin1) and the left-right minimum distance (HDmin1) can be input, and for the maximum 2D area (AR12), the front-back maximum distance (VDmax1) and the left-right maximum distance (HDmax1) can be input.

[0088] In this way, when minimum and maximum two-dimensional areas and target field of view information are input through the user input unit (10), the image adjustment unit (21) can store the input minimum and maximum two-dimensional areas and target field of view information in memory (22).

[0089] Then, the image adjustment unit (21) can control the operation of adjusting the display state of the current surround view image so that the surround view image (e.g., target surround view image) corresponding to the minimum and maximum two-dimensional areas and target field of view information can be displayed on the image output device of the vehicle (100).

[0090] Hereinafter, the operation of the image adjustment unit (21) will be explained in detail with reference to FIGS. 9 to 11.

[0091] First, when the operation starts, the image adjustment unit (21) can store information for the minimum and maximum 2D areas input from the user input unit (10), and the target viewing distances (i.e., first to fourth target viewing distances) and target viewing heights (i.e., first to fourth target viewing heights) for the front, rear, left, and right directions in the memory (22) (S1).

[0092] As previously described, the information of the minimum 2D area (AR11) may be the minimum forward / backward distance (VDmin1) and the minimum left / right distance (HDmin1), and the information of the maximum 2D area (AR12) may be the maximum forward / backward distance (VDmax1) and the maximum left / right distance (HDmax1).

[0093] Accordingly, when information for these minimum and maximum two-dimensional areas is input from the user input unit (10), the image adjustment unit (21) can determine the minimum two-dimensional area and the maximum two-dimensional area using the input information and store them in memory (22).

[0094] Next, the image adjustment unit (21) performs a variable initialization operation (S2) and sets variables necessary for the operation of the image adjustment unit (21), such as the values ​​of multiple parameters determining the inclination of the mesh for the surround view image and camera identification information for angle of view deviation, as initial values ​​and stores them in memory (22).

[0095] Then, the image adjustment unit (21) can set the minimum two-dimensional area (AR11) stored in the memory (22) as the initial two-dimensional area and store it in the memory (22) (S3).

[0096] Additionally, the image adjustment unit (21) can store variables required for the operation of the image adjustment unit (21), such as the values ​​of multiple parameters determining the inclination of the mesh for the surround view image and camera identification information for angle of view deviation, as initial values ​​in the memory (22).

[0097] Then, the image adjustment unit (21) can convert the target view information into three-dimensional coordinates using the target view distance and target view height, which are the target view information input from the user input unit (10) and stored in the memory (22) (S4).

[0098] Accordingly, by the three-dimensional coordinate transformation operation of this target view information, a plurality (e.g., k) of three-dimensional target view coordinates (e.g., first to fourth three-dimensional target view coordinates) can be generated, which are three-dimensional target view coordinates (X-axis value, Y-axis value, Z-axis value) for each direction (e.g., front, rear, left, and right) of the vehicle (100).

[0099] Accordingly, the first to fourth three-dimensional target view coordinates may include the first three-dimensional target view coordinate (e.g., three-dimensional forward target view coordinate) ([target_3d(1)]), the second three-dimensional target view coordinate (e.g., three-dimensional rear target view coordinate) [target_3d(2)], the third three-dimensional target view coordinate (three-dimensional left-side target view coordinate) [target_3d(3)], and the fourth three-dimensional target view coordinate (e.g., three-dimensional right-side target view coordinate) [target_3d(4)].

[0100] When multiple 3D target view coordinates [target_3d(1)~target_3d(4)] are generated, the image adjustment unit (21) can store these 3D target view coordinates [target_3d(1)~target_3d(4)] in memory (22).

[0101] As illustrated in FIG. 5, examples of each coordinate value of each 3D target view coordinate [target_3d(1)~target_3d(4)] may be (target view distance to the left or right, target view distance to the front or rear, target view height). However, the positions of the X-axis, Y-axis, and Z-axis and the types of each coordinate value for each 3D target view coordinate [target_3d(1)~target_3d(4)] are not limited to these and may be changed.

[0102] At this time, the left target field of view, the right target field of view, the forward target field of view, and the rear target field of view may have a designated sign [e.g., (+) or (-)] depending on the direction.

[0103] For example, regarding the forward target view distance and the rear target view distance, the forward target view distance can have a (+) value, and the rear target view distance can have a (-) value. Regarding the target view distances for the left and right sides, the left target view distance can have a (-) value, and the right target view distance can have a (+) value.

[0104] Accordingly, as shown in FIG. 5, the forward 3D target view coordinate [target_3d(1)] may be (0, forward target view distance, forward target view height), the rear 3D target view coordinate [target_3d(2)] may be (0, -(rear target view distance), rear target view height), the left 3D target view coordinate [target_3d(3)] may be ((-left target view distance), 0, left target view height), and the right 3D target view coordinate [target_3d(4)] may be (right target view distance, 0, right target view height).

[0105] Additionally, the image adjustment unit (21) can determine a point on the display screen (SC1) of the image output device to determine whether the image data of the surround view image to be finally output on the display screen of the image output device attached to the vehicle (100) has deviated from the field of view (FOV) of each actual camera attached to the vehicle (100), i.e., whether the field of view has deviated (S5).

[0106] At this time, the display screen (SC1) of the image output device may have multiple pixels arranged in the row direction (x) and the column direction (y), and as a result, the point on the display screen (SC1) for determining whether there is a deviation in the viewing angle (e.g., the point for determining the deviation in the viewing angle) has corresponding coordinates (x n , y m Can have ).

[0107] For example, the number of angle of view deviation judgment points may be multiple (e.g., r points) as shown in FIG. 6, for example, a total of 12 (r=1, 2, 3, ..., 12) (P1-P12).

[0108] For example, these angle of view deviation judgment points (P1-P12) may be the four corners (P1-P4) of the display screen (SC1), the center of each side (P5-P8) of the display screen (SC1), and the points (P9-P12) where the first direction straight line (V1) and the second direction straight line (H1) displayed on the display screen (SC1) respectively meet each side of the display screen (SC1). Here, the first direction straight line (V1) may be a straight line extending along the first direction and passing through the center point of the vehicle width, and the second direction straight line (H1) may be a straight line extending along the second direction and passing through the center point of the vehicle length.

[0109] At least one of the number and location of these angle of view deviation judgment points (P1-P12) can be changed as needed.

[0110] Next, the image adjustment unit (21) can convert the first to fourth three-dimensional target view coordinates [target_3d(1)~target_3d(4)], which are converted into three-dimensional coordinates, into the first to fourth two-dimensional target view coordinates [target_2d(1)~target_2d(4)], which are two-dimensional screen coordinates (S6). Here, the screen coordinates may be the coordinates of two-dimensional virtual data obtained by a virtual camera.

[0111] The 3D target view coordinates [target_3d(1)~target_3d(4)] converted into 3D coordinates are converted into 3D coordinates based on the world coordinate system.

[0112] An example of converting 3D target view coordinates [target_3d(1)~target_3d(4)] into pixel coordinates of virtual data, which are 2D coordinates, is as follows.

[0113] First, the coordinates of the point where the line connecting the focus of the virtual camera and each 3D target view coordinate [target_3d(1)~target_3d(4)] intersects the mesh are obtained, the coordinates of the point where the mesh intersects are converted into the coordinates of the virtual camera 3D Cartesian coordinate system, and then converted back into the coordinates of the virtual camera spherical coordinate system.

[0114] Then, the coordinates converted to the virtual camera spherical coordinate system can be converted back to the coordinates of the point projected onto the virtual data coordinate system, i.e., 2D coordinates (e.g., 2D virtual data coordinates) having X-axis and Y-axis coordinate values, which are 2D target field coordinates [target_2d(1)~target_2d(4)]. The operation of converting these coordinate systems is already known, so a detailed explanation thereof is omitted.

[0115] Through this operation, when each 3D target field of view coordinate [target_3d(1)~target_3d(4)] is converted into a 2D target field of view coordinate [target_2d(1)~target_2d(4)], the image adjustment unit (21) [returns] each coordinate value (x) of the converted 2D target field of view coordinate [target_2d(1)~target_2d(4)]. n , y m The coordinate value (X) of the corresponding 3D target view coordinates [target_3d(1)~target_3d(4)] a , Y b , Z c It can be stored in memory (22) corresponding to ) (S6).

[0116] Accordingly, referring to FIG. 7, each 3D target view coordinate [target_3d(1)~target_3d(4)] can correspond to the converted 2D target view coordinate [target_2d(1)~target_2d(4)]. In FIG. 7, it can be seen that some of the converted 2D target coordinates [target_2d(1)] are outside the display screen (SC1).

[0117] Next, the image adjustment unit (21) calculates the cost value (e.g., cost value) of each 2D target coordinate [target_2d(1)~target_2d(4)] converted into 2D coordinates, and can calculate the minimum cost value, which is the smallest value among these cost values ​​(S7).

[0118] The costs of the 2D target coordinates [target_2d(1)~target_2d(4)] can also be multiple (e.g., k), so that the first to fourth costs [cost(1)~cost(4)] can be provided.

[0119] At this time, the cost value may be the length (e.g., number of pixels) of each 2D target coordinate [target_2d(1)~target_2d(4)] extending inward or outward from the edge (SB1) of the display screen (SC1) of the image output device.

[0120] The out-of-bounds length of each 2D target coordinate [target_2d(1)~target_2d(4)] may be the distance to the nearest edge (SB1) from the corresponding 2D target coordinate [target_2d(1)~target_2d(4)].

[0121] If the 2D target coordinates [target_2d(1)~target_2d(4)] exist at the edge (SB1) of the display screen (SC1), the cost value for the 2D target coordinates may be '0'.

[0122] Additionally, the cost value may have a sign [positive (+) or negative (-)] depending on whether each 2D target coordinate [target_2d(1)~target_2d(4)] is inside or outside the edge (SB1) of the display screen (SC1), centered on the edge (SB1) of the display screen (SC1).

[0123] Accordingly, as illustrated in FIG. 8, examples of values ​​for the costs (e.g., first to fourth costs or first to fourth costs, rear two-dimensional target coordinates, left two-dimensional target coordinates, and right two-dimensional target coordinates) [cost(1), cost(2), cost(3), cost(4)] for each of the four two-dimensional target coordinates (e.g., first to fourth two-dimensional target coordinate costs, rear two-dimensional target coordinate costs, left two-dimensional target coordinate costs, and right two-dimensional target coordinate costs) [cost(1), cost(2), cost(3), cost(4)] may be -200, 300, 180, and 280, respectively.

[0124] In addition, among the values ​​of these costs [cost(1), cost(2), cost(3), cost(4)] (-200, 300, 180, 280, respectively), the minimum size can be -200, and therefore, in this case, the minimum cost value can be -200, which is the cost value for the first 2D target coordinate [target_2d(1)].

[0125] In this way, when the cost values ​​[cost(1)~cost(4)] and minimum cost values ​​for each of the first to fourth two-dimensional target coordinates [target_2d(1)~target_2d(4)] are determined, the image adjustment unit (21) can adjust the initial two-dimensional area determined as the minimum two-dimensional area (AR11) using the cost values ​​[cost(1)~cost(4)] and minimum cost values ​​for each of these two-dimensional target coordinates [target_2d(1)~target_2d(4)].

[0126] Through such adjustments, the final two-dimensional area can have a size suitable for the target field of view information (e.g., target field of view distance and target field of view height).

[0127] To this end, the image adjustment unit (21) can determine whether the first cost [cost(1)] and the second cost [cost(2)], which are costs related to the front and rear corresponding to the length direction of the vehicle (100), which is the first direction of the vehicle (100), are both greater than the corresponding setting value (e.g., inside setting value) (inside threshold) (S8). That is, the image adjustment unit (21) can determine whether the first cost [cost(1)] and the second cost [cost(2)] related to the front and rear 2D target coordinates [target_2d(1), target_2d(2)] of the vehicle (100) exist within the inside setting range (InS1) (see FIG. 8) having the inside setting value as the cost value (S8).

[0128] At this time, as illustrated in FIG. 8, the inner setting range (InS1) may be an area located inward from the edge (SB1) of the display screen (SC1). The inner setting range (InS1) may be determined by moving inward by a set number of pixels (e.g., 4) from the edge (SB1) of the display screen (SC1).

[0129] At this time, the edge (SB1) of the display screen (SC1) can form an outer setting range (OutS1), so that the outer setting range (OutS1) is a range enclosed by the edge (SB1) of the display screen (SC1). Accordingly, the value of the cost of the outer setting range (OutS1) [e.g., outer setting value (inside threshold)] may be smaller than the inner setting value and be '0'.

[0130] Accordingly, if both the forward cost [cost(1)] and the rear cost [cost(2)] exist within the inner setting range (InS1), the image adjustment unit (21) can extend the initial two-dimensional area in the first direction so that the cost adjacent to the inner setting range (InS1) among the forward cost [cost(1)] and the rear cost [cost(2)] lies on the inner setting range (InS1) line or is closest to the inner setting range (InS1) line, thereby extending the initial two-dimensional area forward and backward along the first direction. That is, the length (e.g., number of pixels) of the initial two-dimensional area is adjusted so that the initial two-dimensional area is extended in the first direction (S9).

[0131] As a result, the initial two-dimensional area can be expanded forward and backward together to the same size.

[0132] At this time, the range of expansion in the first direction can be adjusted so as not to be greater than the maximum forward and backward distance (HVmax1) of the maximum two-dimensional area (AR12) (see Fig. 3).

[0133] Likewise, the image adjustment unit (21) can determine whether the third cost value and the fourth cost value, which are the left-side cost [cost(3)] and the right-side cost [cost(4)], are greater than the inside threshold value, and determine whether the left-side cost [cost(3)] and the right-side cost [cost(4)] exist within the inside threshold range (InS1) (S10).

[0134] If both the left side cost [cost(3)] and the right side cost [cost(4)] exist within the inner setting range (InS1), the image adjustment unit (21) can extend the initial 2D area in the second direction so that the cost adjacent to the inner setting range (InS1) among the left side cost [cost(3)] and the right side cost [cost(4)] exists on the inner setting range (InS1) line or is located closest to the inner setting range (InS1) line, and then extend the initial 2D area to the left and right sides along the second direction, and then store the extended initial 2D area in memory (22) (S11).

[0135] In this case as well, the length of the initial 2D area is adjusted so that the 2D target coordinates can be expanded in the second direction, just as in the case of the front and rear. As a result, the initial 2D area can be expanded to the left and right directions by the same size.

[0136] In addition, the range of expansion in the second direction can also be adjusted so as not to be larger than the maximum left-right distance (HDmax1) of the maximum two-dimensional area (AR12) (see Fig. 3).

[0137] By this operation, the initial two-dimensional area can be expanded to the maximum extent to be almost the same size as the display screen (SC1).

[0138] In this way, when the expansion of the initial two-dimensional area into at least one of the first direction and the second direction is determined, the image adjustment unit (21) can create an area expansion two-dimensional area using the initial two-dimensional area in which the expansion state is determined into at least one of the first direction and the second direction, and store it in memory (22) (S12).

[0139] At this time, in steps (S8) and (S10), if at least one of the first cost value and the second cost value is less than or equal to the inner setting value, or at least one of the third cost value and the fourth cost value is less than or equal to the inner setting value, the image adjustment unit (21) can proceed to step (S12) to create an area expansion two-dimensional area using the current initial two-dimensional area and store it in memory (22) (S12).

[0140] Information of the generated area expansion 2D area [e.g., coordinate values ​​of each pixel] can be stored in memory (22) by the image adjustment unit (21).

[0141] In this example, the forward cost [cost(1)] and the backward cost [cost(2)] are considered simultaneously, and the leftward cost [cost(3)] and the rightward cost [cost(4)] are considered simultaneously, so that the initial 2D area can be expanded in the first direction or the second direction.

[0142] However, in an alternative example, the initial 2D area can be individually expanded to the front, back, left, and right directions by considering the front cost [cost(1)], back cost [cost(2)], left cost [cost(3)], and right cost [cost(4)], respectively.

[0143] At this time, the sizes of the areas extending forward and backward may differ from each other, and the sizes of the areas extending to the left and right may also differ from each other.

[0144] For example, if the left-side cost [cost(3)] is within the inner setting range (InS1), the initial 2D area can be extended to the left so that the left-side cost [cost(3)] is located on the inner setting range (InS1) line or closest to the inner setting range (InS1) line, and if the front-side cost [cost(1)] is within the inner setting range (InS1), the initial 2D area can be extended forward so that the front-side cost [cost(1)] is located on the inner setting range (InS1) line or closest to the inner setting range (InS1) line.

[0145] In this way, when determining whether to expand and the direction of expansion of the initial 2D area by considering each of the forward cost [cost(1)], backward cost [cost(2)], leftward cost [cost(3)], and rightward cost [cost(4)], the expansion operation of the initial 2D area can be controlled more precisely.

[0146] By adjusting this initial 2D area, an expanded 2D area can be created, and the mesh corresponding to the expanded 2D area can also be changed.

[0147] If the forward cost [cost(1)], the rear cost [cost(2)], the left cost [cost(3)], and the right cost [cost(4)] all fall outside the inner setting range (InS1), the expansion operation of the initial 2D area may not be performed, and in this case, the initial 2D area and the area expansion 2D area may be the same.

[0148] Then, the image adjustment unit (21) can determine whether there is a part of the virtual data that deviates from the angle of view of the actual camera attached to the vehicle (100) using each angle of view deviation judgment point (P1-P12). At this time, the virtual data may be virtual data having 2-dimensional coordinates generated by the transformation operation of the 3-dimensional target view coordinates [target_3d(1)~target_3d(4)] already described.

[0149] At this time, each angle of view deviation judgment point (P1-P12) is located within the shooting area of ​​at least one of the multiple actual cameras mounted on the front, rear, left, and right sides of the vehicle (100), respectively, but may be outside the angle of view of the corresponding actual camera. Here, the shooting area may be an area that must be captured by the actual camera.

[0150] In memory (22), among the four actual cameras mounted in the front, rear, left, and right directions, an actual camera having a shooting range to which each angle of view deviation judgment point (P1-P12) belongs may already be stored corresponding to the angle of view deviation judgment point (P1-P12). As a result, each angle of view deviation judgment point (P1-P12) and the actual camera corresponding to the angle of view deviation judgment point (P1-P12) may be stored in memory (22) so as to correspond to each other.

[0151] For this operation, the image adjustment unit (21) can convert each coordinate of the angle of view deviation judgment points (P1-P12) stored in memory (22) into coordinates of the camera coordinate system (i.e., the actual camera coordinate system) and store the converted coordinates of the angle of view deviation judgment points (P1-P12) in memory (22) (S13).

[0152] Next, the image adjustment unit (21) can determine whether each angle of view deviation judgment point (P1-P12) exists within the angle of view of the corresponding actual camera by using the coordinates converted into camera coordinates, that is, whether each angle of view deviation judgment point (P1-P12) exists within the resolution range of the corresponding actual camera (S14-S18).

[0153] If there is a point among each angle of view deviation judgment point (P1-P12) that is out of the angle of view of the corresponding actual camera (S14), the image adjustment unit (21) can change the value of the angle of view deviation camera identification information [fov_off(1)~fov_off(4)] corresponding to the angle of view deviation judgment point (P1-P12) from an initial value (e.g., '0') to a set value (e.g., '1') (S16). The angle of view deviation camera identification information [fov_off(1)~fov_off(4)] for each actual camera is already stored in memory (22), and the initial value can be '0'.

[0154] The number of angle of view deviation camera identification information [fov_off(1) to fov_off(4)] may be equal to the number of actual cameras attached to the vehicle (100). For example, the number of angle of view deviation camera identification information (e.g., first angle of view deviation camera identification information to fourth angle of view deviation camera identification information) [fov_off(1) to fov_off(4)] may be a total of 4, and the first angle of view deviation camera identification information [fov_off(1)] is for the first actual camera (i.e., front actual camera), the second angle of view deviation camera identification information [fov_off(2)] is for the second actual camera (i.e., rear actual camera), the third angle of view deviation camera identification information [fov_off(3)] is for the third actual camera (i.e., left actual camera), and the fourth angle of view deviation camera identification information [fov_off(4)] is for the fourth actual camera (i.e., right actual camera).

[0155] At this time, the value of the angle of view deviation camera identification information [fov_off(1)~fov_off(4)] of the corresponding actual camera corresponding to the angle of view deviation judgment point (P1-P12) that has not exceeded the angle of view may be maintained as a previous value (e.g., initial value) (S15).

[0156] Accordingly, the actual camera having the set value for the angle of view deviation camera identification information [fov_off(1)~fov_off(4)] may be a camera in which at least one of the corresponding points (P1-P12) included within its shooting range is out of its angle of view, for example, an angle of view deviation camera.

[0157] As a result, the image adjustment unit (21) can identify which of the actual cameras attached to the vehicle (100) is an angle of view deviation camera that has an angle of view deviation judgment point (P1-P12) located out of the angle of view, by using the value of the angle of view deviation camera identification information [fov_off(1)~fov_off(4)] corresponding to each actual camera.

[0158] Next, the image adjustment unit (21) can determine whether each 2D target field of view coordinate [target_2d(1)~target_2d(4)] exists within the shooting range of the corresponding actual camera by using the cost [cost(1)~cost(4)] of the 2D target field of view coordinate [target_2d(1)~target_2d(4)] and the value of the field of view deviation camera identification information [fov_off(1)~fov_off(4)] corresponding to the actual camera.

[0159] Accordingly, the image adjustment unit (21) can determine whether the corresponding target field of view information related to the actual camera is suitable by determining the values ​​of the cost [cost(1)~cost(4)] of the two-dimensional target field of view coordinates [target_2d(1)~target_2d(4)] and the values ​​of the field of view deviation camera identification information [fov_off(1)~fov_off(4)].

[0160] If the corresponding angle of view deviation camera identification information [fov_off(1)~fov_off(4)] corresponding to the actual camera has a set value, it may be a case where a pixel [i.e., point (P1-P12)] exists outside the angle of view of the actual camera.

[0161] In this case, since the point (e.g., P9) that is out of the field of view in the image output device of the vehicle (100) is located outside the shooting range of the actual camera, it is necessary to move the point that is out of the field of view into the field of view of the actual camera. Therefore, in the case of the present example, the slope of the mesh to which the area expansion 2D area created in step (S12) is applied is reduced so that the part out of the field of view including the point can be included within the field of view of the actual camera.

[0162] Additionally, the cost value of the 2D target view coordinates [target_2d(1)~target_2d(4)] related to the actual camera may be equal to or smaller than the outer setting value, and the 2D target view coordinates [target_2d(1)~target_2d(4)] may be outside the outer setting range (OutS1). In this state, the gradient of the mesh may need to be increased so that the 2D target view coordinates [target_2d(1)~target_2d(4)] are adjusted to move into the display screen (SC1).

[0163] Accordingly, when the image adjustment unit (21) needs to adjust the slope of the mesh in opposite directions (decrease in slope and increase in slope), the target view information for this may be target view information that cannot be achieved even by adjusting the slope of the mesh (i.e., unattainable target view information).

[0164] In this way, if the target field of view information is determined to be unattainable target field of view information, the image adjustment unit (21) determines that the target field of view information is unattainable target field of view information and can generate new target field of view information by changing the target field of view information by an adjustment value from the current target field of view information. Then, the image adjustment unit (21) can determine again whether the new target field of view information is suitable.

[0165] Accordingly, after step (S14) - step (S18), the image adjustment unit (21) can first determine whether the value of the cost [cost(1)] of the first (e.g., the i-th of k) 2D target view coordinates [target_2D(1)] is smaller than the inner setting value in order to determine whether the target view information is unattainable (S19).

[0166] If the value of the cost [cost(1)] of the first 2D target view coordinate [target_2D(1)] (i.e., the first cost value) is smaller than the inner setting value (S19), the image adjustment unit (21) can determine that the first 2D target view coordinate [target_2D(1)] is located outside the inner setting range (InS1) and that the slope of the corresponding mesh (i.e., the mesh with the area expansion 2D area applied) cannot be lowered.

[0167] Accordingly, if the value of the cost [cost(1)] of the first 2D target field of view coordinates [target_2D(1)] (i.e., the first cost value) is smaller than the inner setting value (S19), the image adjustment unit (21) can again determine whether the value of the first field of view deviation camera identification information [fov_off(1)], which is the field of view deviation camera identification information of the first camera (e.g., front camera), is the setting value (e.g., '1') (S20).

[0168] If the value of the camera identification information [fov_off(1)] of the first camera (e.g., front camera) has a set value, the image adjustment unit (21) can determine that the first target field of view information is unattainable target field of view information.

[0169] Accordingly, since the first target view information is unattainable, the image adjustment unit (21) adjusts the first target view information with at least one of the first target view distance and the first target view height (S21), then proceeds to step (S4) to determine whether the newly adjusted first target view information is suitable.

[0170] At this time, the target field of view distance and target field of view height can each be reduced by a fixed amount (e.g., adjustment size). The adjustment size for adjusting the target field of view distance and the adjustment size for adjusting the target field of view height may be the same or different from each other.

[0171] By the operation of the image adjustment unit (21), if the i-th angle of view deviation camera identification information among k angle of view deviation camera identification information has a set value and the i-th cost value among k cost values ​​is equal to or smaller than the inner set value, a new i-th target viewing distance and i-th target viewing height can be generated by reducing at least one of the i-th target viewing distance and i-th target viewing height by an adjustment size.

[0172] In this example, if the first target view height for the current target view information is greater than the set size, the image adjustment unit (21) reduces the target view height by the adjustment size to determine a new first target view distance, and then proceeds to step (S4) to determine whether the new first target view information is unattainable target view information.

[0173] However, if the first target viewing height is less than or equal to the set size, the image adjustment unit (21) can proceed to step (S4) after determining a new first target viewing distance by reducing the first target viewing distance among the first target viewing information by the adjustment size without further reducing the first target viewing height.

[0174] In this way, if the target view information is unattainable target view information, the image adjustment unit (21) sequentially adjusts at least one of the target view distance and the target view height so that the target view information becomes achievable target view information rather than unattainable target view information.

[0175] The operation of determining whether such target field of view information is unattainable and adjusting the target field of view information can be performed sequentially from the first target field of view information to the last target field of view information, the fourth target field of view information (S22-S23).

[0176] However, in step (S19), if the value of the first angle of view deviation camera identification information [fov_off(1)] is not '1', or in step (S20), if the value of the cost [cost(1)] of the first 2D target view coordinate [target_2d(1)] exceeds the inner setting value, that is, if the first target view information is achievable target view information, the image adjustment unit (21) can increase or decrease the slope of the corresponding mesh (i.e., the mesh of the minimum 2D dimension area) using the minimum cost value stored in memory (22) so that the 2D target view coordinate corresponding to the minimum cost value falls between the outer setting range (OutS1) and the inner setting range (InS1).

[0177] Accordingly, the image adjustment unit (21) can determine whether the minimum cost value is greater than the inner setting value, that is, whether all 2D target field coordinates [target_2D(1)~target_2d(4)] are located within the inner setting range (InS1) (S24).

[0178] If the minimum cost value is greater than the inner setting value (S24), the image adjustment unit (21) can reduce the slope of the mesh to lower the slope of the mesh (S25). As the slope of the mesh is reduced, the two-dimensional area displayed on the display screen (SC1) can be extended further toward the edge of the display screen (SC1). At this time, the magnitude of the reduction in slope can be reduced by a setting value, and this setting value can be determined by referring to the size of the cost.

[0179] However, conversely, if the minimum cost value is not greater than the inner setting value (S24), the image adjustment unit (21) can determine whether the minimum cost value is smaller than the outer setting value, that is, whether at least one of all 2D target field of view coordinates [target_2D(1)~target_2d(4)] is located outside the outer setting range (OutS1) (S26).

[0180] Accordingly, if the minimum cost value is smaller than the outer setting value (S26), the image adjustment unit (21) can increase the slope of the mesh to raise the slope of the mesh (S27). As the slope of the mesh increases, the 2D target view coordinate point displayed on the display screen (SC1) can move inward on the display screen (SC1). At this time, the magnitude of the increase in slope may also be determined by reference to the size of the cost.

[0181] The operation of the image adjustment unit (21) can be terminated when the minimum cost value is between the inner setting value and the outer setting value, and when each 2D target view coordinate is within the inner setting range (InS1) in step (S8) to step (S11), the 2D area is expanded so that it exists on the inner setting range (InS1) line or is located closest to the inner setting range (InS1) line, and all 2D target view coordinates [target_2D(1)~target_2d(4)] are located between the inner setting range (InS1) and the outer setting range (OutS1). In this case, it is determined that the first and second target view information has been secured (S100).

[0182] In step (S8) to step (S11), if the 2D target view coordinates cannot be moved to be located on the inner setting range (InS1) line or closest to the inner setting range (InS1) line due to the limitation of the maximum 2D area, the gradient of the mesh must be lowered to move the 2D target view coordinates outward. However, since the minimum cost value is between the inner setting value and the outer setting value, the gradient of the mesh cannot be lowered any further, so the operation may end in this state.

[0183] Since the 2D area remains within the minimum 2D area in the direction of the 2D target view coordinates having the minimum cost value, the image adjustment unit (21) cannot further reduce the 2D area. Therefore, the image adjustment unit (21) cannot use the method of reducing the 2D area in the direction of the 2D target view coordinates having the minimum cost value and lowering the slope of the mesh. Thus, if the minimum cost value is between the inner setting value and the outer setting value, and some cost values ​​are within the inner setting value, that is, if some 2D target view coordinates are within the inner setting range (InS1), the 2D area can no longer be adjusted, so the image adjustment unit (21) can terminate the operation. The fact that the 2D target view coordinates are within the inner setting range (InS1) may mean that a view distance and view height farther than the target view distance and target view height have been secured.

[0184] Accordingly, in steps (S25) and (S27), after adjusting the slope of the current mesh, the image adjustment unit (21) moves to step (S4) and can continue the control operation so that the position corresponding to the target field of view information for the front, rear, left, and right directions is located between the inner setting range (InS1) and the outer setting range (OutS1).

[0185] When the operation to determine the unattainable target viewing distance for each of the first to fourth target viewing distances is completed, the image adjustment unit (21) can adjust at least one of the target viewing distance and the target viewing height by adjusting the slope of the mesh according to the determination result if the target viewing distance among the first to fourth target viewing distances is not an unattainable target viewing distance.

[0186] For example, in step (S20), if the value of the identification information of the corresponding camera (e.g., front camera) does not have a set value, that is, has an initial value (e.g., '0'), the image adjustment unit (21) can determine whether the minimum cost value stored in the memory (22) is greater than or equal to the inner set value, that is, whether multiple target viewing distances (first to fourth target viewing distances) all exist within the inner set range (InS1).

[0187] Accordingly, if the value of the identification information of the camera (e.g., front camera) maintains its initial value and all target viewing distances are located within the inner setting range (InS1), the image adjustment unit (21) can reduce the slope of the mesh to lower the slope of the mesh (S25). As the slope of the mesh is reduced, the 2D target viewing coordinates [target_2d(1)~target_2d(4)] displayed on the display screen (SC1) can be increased further toward the edge of the display screen (SC1). At this time, the magnitude of the slope reduction may be determined by reference to the size of the cost.

[0188] However, the image adjustment unit (21) can determine in step (25) whether the minimum cost value is smaller than the outer setting value, that is, if the target viewing distance that deviates the most from the display screen among the multiple target viewing distances (first to fourth target viewing distances) deviates by more than the setting size, the image adjustment unit (21) can increase the slope of the mesh to increase the slope of the mesh (S27).

[0189] The decrease or increase in the slope of such a mesh can be reduced or increased in steps by a fixed amount.

[0190] The adjustment behavior of this mesh can be adjusted until the minimum cost value has a value between the inner set value and the outer set value.

[0191] However, in step (27), if the minimum cost value is greater than or equal to the outer setting value, the image adjustment unit (21) determines that the image within all target viewing distances can be displayed on the display screen while securing all target viewing distances. Accordingly, the image adjustment unit (21) can terminate the operation of adjusting the display state of the display screen to correspond to the target viewing distance (S100).

[0192] With the operation of the surround view video adjustment system of this example, the image output to the video output device located in the vehicle (100) may have a subject located within the range of each target viewing distance and each target viewing height located within the display screen of the video output device.

[0193] Referring to FIG. 13, it can be seen that the end of the subject (200) having a first target viewing distance and a first target viewing height is located between the outer setting range (OutS1) and the inner setting range (InS1).

[0194] The technical features disclosed in each embodiment of the present invention are not limited to that embodiment only, and as long as they are not mutually incompatible, the technical features disclosed in each embodiment may be combined and applied to different embodiments.

[0195] Therefore, in each embodiment, the technical features are described primarily, but as long as the technical features are not mutually incompatible, they may be combined and applied together.

[0196] The present invention is not limited to the embodiments described above and the attached drawings, and various modifications and variations may be possible from the perspective of those skilled in the art to which the present invention belongs. Accordingly, the scope of the present invention should be defined not only by the claims of this specification but also by equivalents thereof.

Claims

A storage unit storing a 1.2-dimensional area, k target viewing distances, k target viewing heights, and r angle of view deviation determination points; and Image adjustment unit connected to the above storage unit Includes, The above image adjustment unit is, 3D coordinate transformation is performed using k target view distances and k target view heights, respectively, to generate k 3D target view coordinates having 3D coordinates, and Calculate the cost value for each 3D target view coordinate to produce k cost values, and Using the calculated k cost values, the above two-dimensional region is expanded to at least one of the first direction and the second direction, and Determine whether each angle of view deviation judgment point exists within the actual camera's angle of view, change the value of the corresponding angle of view deviation camera identification information among the k angle of view deviation camera identification information that is outside the angle of view deviation judgment point to a set value, and A surround view image adjustment device that, among k field of view deviation camera identification information, the i-th field of view deviation camera identification information has a set value, and among k cost values, if the i-th cost value is equal to or smaller than the inner set value, reduces at least one of the i-th target field of view distance and i-th target field of view height by an adjustment size to generate a new i-th target field of view distance and i-th target field of view height.

2. In Paragraph 1, The 2D region includes a minimum 2D region and a maximum 2D region, and The above image adjustment unit is a surround view image adjustment device that expands the minimum two-dimensional area among the minimum two-dimensional area and the maximum two-dimensional area to at least one of the first direction and the second direction.

3. In Paragraph 2, A surround view image adjustment device in which the expansion range of the minimum two-dimensional area is not larger than the maximum two-dimensional area.

4. In Paragraph 1, The image adjustment unit transforms each 3D target field of view coordinate into k 2D target field of view coordinates having 2D coordinates, and The above cost value is a surround view image adjustment device in which the two-dimensional target field of view coordinates are the length extending inward or outward from the edge of the display screen.

5. In Paragraph 1, If, among the k cost values, at least one cost value related to the first direction is greater than the inner setting value, the two-dimensional area is expanded in the first direction such that the cost having the smaller value among the costs related to the first direction is located on the inner setting range line determined to be the inner side of the display screen or is located closest to the inner setting range line, and If, among the k cost values, at least one cost value related to the second direction is greater than the inner setting value, the two-dimensional area is expanded in the second direction such that the cost having the smaller value among the costs related to the second direction is located on the inner setting range line determined to be the inner side of the display screen or is located closest to the inner setting range line. The above inner setting value is a surround view image adjustment device in which the cost value of the above inner setting range is a value.

6. In Paragraph 1, The above image adjustment unit is a surround view image adjustment device that, among the i-th target viewing distance and the i-th target viewing height, if the i-th target viewing height is greater than the set size, reduces the i-th target viewing height by the adjustment size, and if the i-th target viewing height is less than or equal to the set size, reduces the i-th target viewing distance by the adjustment size.

7. In Paragraph 6, A surround view image adjustment device in which the adjustment size for adjusting the i-th target field of view height and the adjustment size for adjusting the i-th target field of view length are the same or different.

8. In Paragraph 1, The above r angle of view deviation determination points include the four corners of the display screen, the center of each side of the display screen, and each point where the straight line in the first direction and the straight line in the second direction of the vehicle displayed on the display screen meet each side of the display screen.

9. In Paragraph 1, A surround view image adjustment device that calculates a minimum cost value having the smallest value among the k cost values, and if the i-th angle of view deviation camera identification information does not have a set value and the i-th cost value exceeds an inner set value, uses the minimum cost value to increase or decrease the slope of a mesh to which a 2D area extended to at least one of the first direction and the second direction is applied.

10. In Paragraph 9, The above image adjustment unit is a surround view image adjustment device that reduces the slope of the mesh of the two-dimensional area when the above minimum cost value is greater than the inner setting value.

11. In Paragraph 10, The above image adjustment unit is a surround view image adjustment device that increases the slope of a mesh to which a two-dimensional area extended to at least one of the first direction and the second direction is applied, if the minimum cost value is not greater than the inner setting value and is smaller than the outer setting value.

12. In Paragraph 11, The above outer setting value is the cost value of the outer setting range, and The above outer setting range is a surround view image adjustment device, which is a range enclosed by the edges of the display screen.

Images