Image processing device

Abstract

An image processing device is provided that can create an image that is easy to detect a given object while suppressing an increase in calculation load. The image processing device comprises: a brightness image creation unit (22) that creates a brightness image (52) that represents the brightness distribution in a color image (51) obtained by a camera (15); a saturation image creation unit (23) that creates a saturation image (53) that represents the saturation distribution in the color image (51); and a resolution image creation unit (25) that combines the brightness image (52) and the saturation image (53) to create a resolution image (55) for detecting the given object.

Description

Technical Field

[0001] The present disclosure relates to an image processing device for processing a color image. Background Art

[0002] Generally, there is known an image processing device that detects a detection object such as a white line by analyzing a color image obtained by a camera as a brightness image. However, there is a case where, when the detection object is a color other than white, the image processing device cannot properly detect the detection object even if the image processing device analyzes the color as a brightness image. For this reason, it is considered to enable the image processing device to detect a detection object of a color other than white (for example, refer to Patent Document 1).

[0003] The image processing device respectively calculates the intensity of each color signal of RGB in the color image, and changes the combination of each color signal according to the color of the detection object, thereby calculating the combination that maximizes the contrast of the detection object, and combines each color signal according to the color of the detection object so as to detect the detection object.

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2003-32669 Summary of the invention

[0007] (Problems to be solved by the invention)

[0008] However, the image processing device needs to change the combination of each color signal according to the color of the detection object in the color image, so it is necessary to find a combination according to the color of the detection object to be analyzed, and find a combination that maximizes the contrast of the detection object for each color image to be analyzed. Therefore, in the image processing device, the calculation load for detecting the detection object increases.

[0009] The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide an image processing device capable of creating an image in which a given object is easily detected while suppressing an increase in calculation load.

[0010] (Technical solutions to solve problems)

[0011] In order to achieve the above-mentioned purpose, the image processing device disclosed in the present invention comprises: a brightness image creation unit, which creates a brightness image representing the brightness distribution in a color image obtained by a camera; a saturation image creation unit, which creates a saturation image representing the saturation distribution in the color image; and a resolution image creation unit, which synthesizes the brightness image and the saturation image to create a resolution image for detecting a given object.

[0012] (Effects of the Invention)

[0013] The image processing device of the present disclosure configured as described above can create an image in which a given object is easily detected while suppressing an increase in calculation load. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram showing the structure of an image processing device according to Embodiment 1 which is an example of an image processing device according to the present disclosure.

[0015] Figure 2 This is an explanatory diagram showing an example of a color image acquired by a camera.

[0016] Figure 3 It means in Figure 2 An illustration of a brightness image of an area surrounded by a dotted line in a color image of .

[0017] Figure 4 It means in Figure 2 An illustration of a saturation image of an area surrounded by a dotted line in a color image of .

[0018] Figure 5 It means in Figure 2 An illustration of an image of saturation extraction of an area surrounded by a dotted line in a color image of FIG.

[0019] Figure 6 This is an explanatory diagram showing a saturation image used to explain an example of extraction processing performed by the extracted image creation unit.

[0020] Figure 7 It is an explanatory diagram for explaining the contraction process and the expansion process using a filter as an example of the extraction process.

[0021] Figure 8 Yes Figure 6 An illustration of the situation in which a saturated image is subjected to a shrinkage process.

[0022] Fig. 9 Yes Figure 6 An illustration of the situation where a saturated image is subjected to two shrinkage processes.

[0023] Fig.10 Yes Figure 6 An illustration of a saturated image that is subjected to two contraction processes and then one expansion process.

[0024] Fig.11 Yes Figure 6 An explanatory diagram showing a situation in which a saturated image is subjected to two shrinkage processes and then two dilation processes.

[0025] Fig.12 It means from Figure 6 Subtract the saturation image Fig.11 Illustration of a saturation extraction image created by processing an image.

[0026] Fig.13 It means in Figure 2 An illustration of an analyzed image of an area surrounded by a dotted line in a color image of .

[0027] Fig.14 This is a flowchart showing an example of image processing executed by the control unit.

[0028] Fig.15 Yes Figure 6 An illustration of a processed image created by performing a contraction process and then a dilation process on a saturation image.

[0029] Fig.16 It means from Figure 6 Subtract the saturation image Fig.15 An illustration of a processed image created by processing an image of .

[0030] Fig.17 It means from Fig.12 The saturation of the extracted image is subtracted Fig.16 Illustration of a saturation extraction image created by processing an image. DETAILED DESCRIPTION

[0031] Hereinafter, an image processing device 10 according to Embodiment 1 as an example of an image processing device according to the present disclosure will be described with reference to the drawings.

[0032] (Example 1)

[0033] use Figures 1 to 17 The image processing device 10 of the first embodiment of the present disclosure is described. The image processing device 10 performs image processing for creating an image that facilitates detection of a given object in a color image obtained by a camera 15. In the first embodiment, the image processing device 10 is mounted on a vehicle 1 (see Figure 2 ) for in-vehicle use. Furthermore, the image processing device 10 is not limited to an in-vehicle image processing device, and may also be, for example, a smart phone, a tablet terminal, and a PDA (personal data assistant). In this case, image processing is provided by an application installed in these devices. For example, the image processing device 10 has: a parking assistance function that assists the operation of parking in a parking lot; a lane keeping function that maintains the lane in which the vehicle is traveling; a navigation function that displays a map and / or accompanying information, searches for a recommended route from a starting point (or current location) to a destination, guides the route, and displays road traffic information, etc.

[0034] like Figure 1 As shown, the image processing device 10 includes a control unit 11 and a storage unit 12 , and a display unit 13 , an operation unit 14 , and a camera 15 are connected to the control unit 11 via an in-vehicle network such as CAN (Controller Area Network) as a communication protocol.

[0035] The storage unit 12 stores in advance a program for executing image processing, a program for executing other functions, and information required for these programs that can be used without being overwritten. The information stored in the storage unit 12 can be read out by the control unit 11 as appropriate.

[0036] The display unit 13 displays an analysis image 55 (see FIG. 1 ) described below, which is created by image processing. Fig.13 ), the display unit is composed of, for example, a liquid crystal display, an organic EL display, etc. In addition, when having the above functions, the display unit 13 appropriately displays auxiliary images for the parking assist function, display images for the lane keeping function, route images for the navigation function, etc.

[0037] The operation unit 14 receives a command input from the user and can specify whether to execute image processing. In addition, when the above functions are provided, the operation unit 14 can specify whether to execute the parking assist function and the lane keeping function, and can input a destination for the navigation function. In the first embodiment, the configuration of the operation unit 14 includes the display unit 13 as a component equipped with a touch panel function, and various switches provided around the display unit.

[0038] The camera 15 obtains a color image of a given range around the vehicle 1 for image processing. In the first embodiment, the camera 15 is arranged at the front, rear, left, and right positions of the vehicle 1 to obtain a color image of the entire periphery of the vehicle 1 and form a color bird's-eye view image 51 (see Figure 2 ). In addition, the camera 15 may include a single wide-angle fisheye lens installed at the front of the vehicle 1 to obtain a color image of the front (front side in the driving direction), or may be installed in a driving recorder or an anti-collision mechanism. The camera 15 outputs image data representing the obtained color image to the control unit 11.

[0039] The control unit 11 includes an overhead image creation unit 21, a brightness image creation unit 22, a saturation image creation unit 23, an extracted image creation unit 24, an analytical image creation unit 25, an object detection unit 26, and a memory unit 27. Each part of the control unit 11 other than the memory unit 27 is composed of a program. In addition, each of these parts can be exclusively composed of an electronic circuit, and is not limited to the structure of the first embodiment. In the memory unit 27, numerical data required in the calculation process of each part of the control unit 11, program variables for the processing results during the calculation process, etc. are appropriately written, and these contents are appropriately read by each part of the control unit 11. The memory unit 27 can also store programs for each process executed by the control unit 11.

[0040] When a color image (image data) is input from the camera 15, the bird's-eye view image creation unit 21 converts the color image into a viewing angle and creates a color bird's-eye view image 51 (see FIG. 5 ) that looks like the ground from above. Figure 2 The bird's-eye image creation unit 21 obtains the design values ​​(external parameters) of the installation position and angles (pitch angle, yaw angle, roll angle, camera height) of the camera 15, the focal length, pixel size, optical axis center, distortion function and other (internal parameters) of the camera 15 from the storage unit 12. The bird's-eye image creation unit 21 uses the obtained distortion function to perform a distortion correction process for correcting the distortion of the image caused by the lens. The bird's-eye image creation unit 21 uses a known geometric transformation formula of the camera and creates the output values ​​(signal values ​​including the brightness value of each color) of the pixels at all coordinate positions in the bird's-eye image 51 represented by the world coordinates based on the output values ​​(signal values) of the corresponding pixels of the distortion-corrected image. At this time, if there is no pixel of the distortion-corrected image corresponding to a specific pixel of the bird's-eye image 51, the output value of the specific pixel is obtained based on the output values ​​of the surrounding pixels by a known brightness interpolation process. Furthermore, the bird's-eye view image creation unit 21 creates a bird's-eye view image 51 around the vehicle 1 by combining adjacent captured images of the four cameras 15, namely the front, rear, left and right cameras, into one image by linearly interpolating the brightness of corresponding coordinate values ​​to eliminate the uncomfortable feeling of seams.

[0041] like Figure 2 As shown, the bird's-eye view image 51 of the first embodiment shows a state where the vehicle 1 is located in a parking lot, and a shadow 2 of the vehicle 1 is displayed above the vehicle 1 in a front view, and a plurality of lines 3 drawn on a ground surface such as asphalt, which represent parking spaces, are displayed above the shadow. If the bird's-eye view image creation unit 21 creates the bird's-eye view image 51 (its image data), the above-mentioned processes may be executed simultaneously, and other processes may be performed, and the present invention is not limited to the first embodiment.

[0042] The brightness image creating unit 22 and the saturation image creating unit 23 create a brightness image 52 (see FIG. 5 ) described later based on the color bird's-eye view image 51. Figure 3 ), saturation image 53 (refer to Figure 4 ). Therefore, the bird's-eye view image 51 functions as a color image acquired by the camera. The brightness image creation unit 22 and the saturation image creation unit 23, together with the extracted image creation unit 24 and the analytical image creation unit 25, create images (brightness image 52, saturation image 53, saturation extracted image 54, analytical image 55 described later) of equal size (number of pixels) to each other. Furthermore, if each creation unit (22, 23, 24, 25) creates each image with equal size, it is possible to perform the following operations on the image. Figure 2 A portion of the overhead image 51 shown (see Figure 2 and Figure 3 ) may perform various processing on the entire bird's-eye view image 51.

[0043] The brightness image creation unit 22 performs brightness image creation processing for obtaining the brightness value (brightness distribution) of each pixel based on the brightness data in the bird's-eye view image 51 (color image) to create a brightness image 52 (see FIG. 5 ) which is a monochrome image represented by the magnitude (value of 0 to 255) of the brightness value of each pixel. Figure 3 ). The brightness image creation unit 22 uses the Y value in each pixel when the bird's-eye view image 51 is YUV data, and uses the value obtained by adding the brightness value of each color (RGB) in each pixel when the bird's-eye view image 51 is RGB data. The brightness image 52 represents the brightness distribution in the bird's-eye view image 51 (color image) using the brightness value of each pixel. Figure 3 The intensity image 52 shown corresponds to Figure 2 The area surrounded by the dashed line in the overhead image 51 shown in FIG. 5 is shown in FIG. 5. Here, in the brightness image 52, when each line 3 in the overhead image 51 is white, the brightness value of each line 3 becomes sufficiently large and is clearly displayed. Figure 3 In the brightness image 52 shown, each line 3 in the bird's-eye view image 51 is yellow and the brightness value of each line 3 is not too large, so it appears thin (in Figure 3 In order to express thinness, the lines drawn are made very thin).

[0044] The saturation image creation unit 23 obtains the saturation value (saturation distribution) of each pixel based on the saturation data in the bird's-eye view image 51 (color image). The saturation image creation unit 23 performs a saturation image creation process that replaces the saturation value of each pixel with the brightness value to create a monochrome image represented by the brightness value of each pixel after the replacement, that is, a saturation image 53 (see Figure 4 ). Figure 4 The saturation image 53 shown represents Figure 2 The area surrounded by the dot-dash line in the bird's-eye view image 51 is shown. The saturation image creating unit 23 of the first embodiment creates the saturation image 53 in the following manner.

[0045] First, when the bird's-eye view image 51 is YUV data, the saturation image creating unit 23 performs conversion processing on each pixel into RGB data using, for example, the following equation (1). When the bird's-eye view image 51 is RGB data, the saturation image creating unit 23 does not perform conversion processing.

[0046] R value = Y value + 1.402 × V value

[0047] G value = Y value - 0.344 × U value - 0.714 × V value (1)

[0048] B value = Y value + 1.772 × U value

[0049] Next, the saturation image creation unit 23 sets the largest value among the R value, G value, and B value of each pixel as the maximum value V1, and sets the smallest value as the minimum value V2. Then, the saturation image creation unit 23 performs saturation calculation processing, which uses the following formula (2) based on the HSV of the cylindrical model to obtain the saturation value (S value) of each pixel.

[0050] S value = (V1 - V2) / V1 (2)

[0051] Thus, the saturation image creating unit 23 can obtain the normalized S value for each pixel.

[0052] Next, the saturation image creation unit 23 performs a replacement process for obtaining a Sb value for replacing the saturation value of each pixel with the brightness value by multiplying the normalized S value by a coefficient A as shown in the following equation (3). The coefficient A is the brightness value obtained by using the normalized S value, and is set to 255 in accordance with the scale of the brightness image 52, which is a monochrome image, in the first embodiment.

[0053] Sb value = S value × A (3)

[0054] Then, the saturation image creation unit 23 creates a saturation image 53 (see FIG. 5 ) which is a monochrome image represented by the magnitude of the brightness value of each pixel, using the obtained Sb value of each pixel. Figure 4 ). The saturation image 53 represents the saturation distribution in the bird's-eye view image 51 (color image) using the brightness value of each pixel after the conversion. Figure 4 The saturation image 53 shown is Figure 3 The brightness image 52 shown also represents Figure 2 The area surrounded by the dotted line in the overhead image 51 shown in FIG. 5A. Here, in the saturation image 53, even if each line 3 in the overhead image 51 is yellow, the saturation value of each line 3 is large enough and can be clearly displayed. In addition, the saturation image creation unit 23 is not limited to the configuration of the first embodiment as long as it can create a monochrome image, i.e., the saturation image 53, in which the magnitude of the saturation value of each pixel is replaced with the magnitude of the brightness value, based on the saturation data in the overhead image 51 (color image).

[0055] The extracted image creation unit 24 performs extraction processing to create a saturation extracted image 54 (see Figure 5 ), the extraction process extracts a pattern corresponding to the detection object in the saturation image 53 created by the saturation image creation unit 23. In addition, the pattern is displayed in the saturation image 53, including lines and marks drawn on the ground, shadows formed on the ground, etc. Figure 5 The saturation extracted image 54 is shown in the saturation image 53 (refer to Figure 4 ) in which each line 3 is a detection object, shadows 2 different from the detection object are removed by extracting each line 3.

[0056] The extraction image creation unit 24 of the first embodiment creates a saturation extraction image 54 by performing contraction processing and expansion processing on the saturation image 53 to remove patterns larger than the detection object. Figures 6 to 12 to explain. Figure 6 A saturation image 53 is shown as an example for facilitating understanding of the contraction process and the expansion process. The left side shows a pattern D1 larger than the detection object, the middle shows the pattern D2 as the detection object, and the right side shows a pattern D3 smaller than the detection object. In the saturation image 53, each pattern (D1, D2, D3) has an equal vertical size (10 pixels) and different width sizes for comparison. The pattern D1 is a line with a width of 8 pixels in the saturation image 53, and the saturation value (brightness value) of one pixel in the periphery is lower than that of the center. The pattern D2 (detection object) is a line with a width of 4 pixels in the saturation image 53, and the saturation value (brightness value) of one pixel in the periphery is lower than that of the center. The pattern D3 is a line with a width of 2 pixels in the saturation image 53, and has a saturation value (brightness value) equal to that of the center of the pattern D1 and the pattern D2.

[0057] like Figure 7 As shown, the extracted image creating section 24 uses the filter F to perform the contraction process and the expansion process. Figure 7For explanation, the lower end of the pattern D1 of the saturation image 53 is enlarged and the filter F is configured in three modes (hereinafter referred to as F1, F2, and F3) to perform contraction processing and expansion processing. In this example, the extracted image creation unit 24 uses the filter F of 9 pixels in a 3×3 frame, and converts the pixel located in the center frame to the lowest saturation value (brightness value) among the pixels of the surrounding 8 frames in the case of contraction processing, and converts it to the highest saturation value (brightness value) among the pixels of the surrounding 8 frames in the case of expansion processing.

[0058] The filter F1 of the first mode is configured to align the center frame with the low saturation pixel G1 at the lower left end of the pattern D1 to convert the pixel G1. The filter F1 makes the pixel G1 have a saturation value of 0 (zero) on the outside of the pattern D1 in the case of contraction processing, and makes the pixel G1 have a high saturation value in the center of the pattern D1 in the case of expansion processing.

[0059] The filter F2 of the second mode is configured to align the center frame with the pixel G2 so that the entire frame is located in the high saturation value in the center of the pattern D1 to convert the pixel G2. The filter F2 makes the pixel G2 a high saturation value regardless of the shrinking process or the dilation process.

[0060] The filter F3 of the third mode is configured to align the center frame with the pixel G3 of 0 (zero) saturation value on the left outer side of the pattern D1 to convert the pixel G3. The filter F3 makes the pixel G3 0 (zero) saturation value on the outer side of the pattern D1 in the case of contraction processing, and makes the pixel G3 low saturation value on the periphery of the pattern D1 in the case of expansion processing.

[0061] As described above, the extracted image creation unit 24 reduces the pattern of the saturation image 53 by one pixel by performing the contraction process using the filter F on all pixels of the original saturation image 53. In addition, as described above, the extracted image creation unit 24 enlarges the pattern of the saturation image 53 by one pixel by performing the dilation process using the filter F on all pixels of the original saturation image 53. Furthermore, as described above, when the saturation value is 0 (zero) in all frames of the filter F, the extracted image creation unit 24 uses the saturation value of 0 (zero) in both the contraction process and the dilation process. Here, since the contraction process and the dilation process are performed on all pixels of the target area in the saturation image 53, the pattern is reduced or increased by one pixel in all directions. Therefore, the extracted image creation unit 24 can remove the detection object by performing the contraction process half the number of times of the smaller dimension of the longitudinal dimension and the width dimension in the detection object. Taking advantage of this, the extracted image creating unit 24 creates the processed image 56 by executing the contraction process a number of times that the detection object can be removed, and then executing the expansion process a same number of times.

[0062] First, the extraction image creation unit 24 Figure 6 The saturation image 53 shown is subjected to a contraction process, such as Figure 8 As shown in FIG. 5 , the saturation image 53 is obtained by reducing the pattern D1 by one circle, reducing the pattern D2 by one circle and making it a low saturation value, and making the pattern D3 have a saturation value of 0 (zero). Next, the extraction image creation unit 24 removes the saturation value. Figure 8 The saturation image 53 shown is subjected to a contraction process, such as Fig. 9 As shown, the saturation image 53 is obtained by reducing the pattern D1 by one circle, removing the pattern D2 with a saturation value of 0 (zero), and maintaining the state of removing the pattern D3.

[0063] Next, the extraction image creation unit 24 Fig. 9 The saturation image 53 shown is subjected to a dilation process, such as Fig.10 As shown in FIG. 1 , the pattern D1 is enlarged by one circle, and the saturation value of the removed pattern D2 and pattern D3 is set to 0 (zero) so that the saturation image 53 is maintained in the removed state. Next, the extraction image creation unit 24 Fig.10 The saturation image 53 shown is subjected to a dilation process, such as Fig.11 As shown, the saturation image 53 is obtained by enlarging the pattern D1 and causing no subsequent changes to the removed patterns D2 and D3. Therefore, the extracted image creation unit 24 creates a processed image 56 as the saturation image 53, which leaves only the pattern (in this example, the pattern D1) larger than the pattern D2 as the detection object from the saturation image 53.

[0064] Then, the extracted image creation unit 24 generates the image from Figure 6 The original saturation image 53 is subtracted Fig.11 The processed image 56 (the saturation value (brightness value) of each pixel) is created Fig.12 The saturation extracted image 54 is shown. The saturation extracted image 54 removes a pattern (pattern D1) larger than the pattern D2 to be detected from the saturation image 53. In this way, the extracted image creation unit 24 creates the saturation extracted image 54 from which the pattern larger than the detection target is removed by performing the contraction process and the expansion process on the saturation image 53.

[0065] The analytical image creation unit 25 basically generates the brightness image 52 (see Figure 3 ) and the saturation image 53 (refer to Figure 4 ) are synthesized to create an analysis image 55 for detecting a given object (detection target) (refer to Fig.13 ). The analysis image creation unit 25 adds the brightness value in the brightness image 52 and the brightness value in the saturation image 53 (replacing the saturation value) for each pixel in the brightness image 52 and the saturation image 53 of equal size (number of pixels), thereby creating a monochrome image represented by the size of the brightness value of each pixel, namely, the analysis image 55.

[0066] The analysis image creation unit 25 of the first embodiment generates the saturation extracted image 54 (see Figure 5 ) replaces the saturation image 53 created by the saturation image creation unit 23 and is synthesized with the brightness image 52 to create Fig.13 The analytical image 55 shown. The analytical image creating unit 25 synthesizes the brightness image 52 and the saturation extraction image 54 in the same manner as the saturation image 53. From the above, the brightness image creating unit 22, the saturation image creating unit 23, the extraction image creating unit 24, and the analytical image creating unit 25 function as an image processing device that realizes an image processing function of creating the analytical image 55 based on the color image (bird's-eye view image 51) acquired by the camera 15.

[0067] The object detection unit 26 analyzes the analytical image 55 created by the analytical image creation unit 25 to detect a pattern corresponding to the detection object in the analytical image 55. The object detection unit 26 performs edge extraction processing on the analytical image 55, and extracts a theoretical comparison area representing the size and shape of the detection object in the image obtained by the edge extraction processing, thereby extracting the detection object (pattern). In addition, as long as the pattern corresponding to the detection object can be extracted, the object detection unit 26 can also extract it by other methods, and is not limited to the configuration of the first embodiment. In the first embodiment, since each line 3 is used as the detection object, the object detection unit 26 detects each line 3 in the analytical image 55.

[0068] Next, use Fig.14 An example of image processing for detecting a detection target in a color image acquired by the camera 15 under the control of the control unit 11 in the image processing device 10 will be described. Fig.14 FIG. 1 is a flowchart showing the image processing performed by the control unit 11 in the first embodiment. The image processing is performed by the control unit 11 based on the program stored in the storage unit 12. Fig.14 Each step (process) of the flowchart is described below. Fig.14 The flowchart is started by turning on the power of the image processing device 10 to output the color image (data thereof) acquired by the camera 15 to the control unit 11 .

[0069] In step S1, the bird's-eye view image 51 is created, and the process proceeds to step S2. In step S1, the bird's-eye view image creation unit 21 creates the color bird's-eye view image 51 (see FIG. 1 ) based on the color image (image data) input from the camera 15. Figure 2 ).

[0070] In step S2, the brightness image 52 is created, and the process proceeds to step S3. In step S2, the brightness image creation unit 22 creates the brightness image 52 (see FIG. 5 ) which is a monochrome image represented by the brightness value of each pixel based on the bird's-eye view image 51. Figure 3 ).

[0071] In step S3, the saturation image 53 is created, and the process proceeds to step S4. In step S3, the saturation image creation unit 23 creates the saturation image 53 (see FIG. 1 ), which is a monochrome image represented by the magnitude of the brightness value after replacing the magnitude of the saturation value of each pixel, based on the bird's-eye view image 51. Figure 4 ). Furthermore, step S2 and step S3 may be performed in reverse order or simultaneously, and are not limited to the configuration of embodiment 1.

[0072] In step S4, the saturation extraction image 54 is created, and the process proceeds to step S5. In step S4, the extraction image creation unit 24 creates the saturation extraction image 54 by removing patterns larger than the detection object based on the saturation image 53 (see Figure 5 ).

[0073] In step S5, the analysis image 55 is created, and the process proceeds to step S6. In step S5, the analysis image creation unit 25 creates the analysis image 55 by combining the brightness image 52 created in step S2 and the saturation extraction image 54 created in step S4 (see FIG. 5 ). Fig.13 ). Steps S2 to S5 are steps of creating the analysis image 55 based on the color image (bird's-eye view image 51 ) acquired by the camera 15 .

[0074] In step S6, the detection object in the analysis image 55 is detected to end the image processing. In step S6, the object detection unit 26 analyzes the analysis image 55 created in step S5 to detect the detection object (in the example shown in the figure, each line 3) in the analysis image 55. The control unit 11 uses the detection result for various functions. For example, the control unit 11 can realize the parking assist function or the lane keeping function based on the position of each line 3 as the detection object.

[0075] In this way, the control unit 11 facilitates detection of a given object based on analysis in the object detection unit 26 by synthesizing the luminance image 52 created from the bird's-eye view image 51 based on the color image acquired by the camera 15 with the saturation extraction image 54 (saturation image 53) to create the analysis image 55. The control unit 11 displays the analysis image 55 created by the image processing or the detection result of the detection object on the display unit 13 as appropriate.

[0076] Here, the detection object is, for example, a figure drawn on the ground like the lines 3 of Example 1. Since the figure is mostly drawn on the ground in white and has a high brightness value, it can be properly detected by analyzing the color image obtained by the camera 15 as a brightness image. However, there are many cases where the figure (detection object) is drawn in yellow, red, green, etc. in addition to white. Therefore, there are many cases where the brightness value of the detection object other than white is low, and even if it can be clearly identified by human vision, there are many cases where it cannot be properly detected by analyzing it as a brightness image. In addition, when a color image is obtained from the camera 15 in a dark environment, the overall brightness value decreases, so that sufficient contrast cannot be obtained when it is used as a brightness image. Even if it can be identified by human vision, there are cases where the detection object cannot be properly detected by analyzing the brightness image.

[0077] Therefore, conventional image processing devices respectively obtain the intensity of each color signal of RGB in a color image to change the combination of each color signal according to the color of the detection object, thereby obtaining a combination with the maximum contrast of the detection object. Furthermore, conventional image processing devices can detect the detection object by combining each color signal according to the color of the detection object. However, since conventional image processing devices perform such calculation processing, the calculation load for detecting the detection object becomes large.

[0078] In contrast, the image processing device 10 of the present disclosure creates a saturation image 53 based on the color image (bird's-eye image 51) acquired by the camera 15 and the brightness image 52, regardless of the color of the detection object, and synthesizes these images to create a resolution image 55. Here, a detection object other than white is considered to have a high saturation value even if the brightness value is low, if it can be clearly recognized by human eyes. Then, the image processing device 10 can create a resolution image 55 that emphasizes both the portion with high brightness value and the portion with high saturation value by synthesizing the brightness image 52 and the saturation image 53. Thus, the image processing device 10 can detect both the portion with high brightness and the portion with high saturation, that is, both the detection object of white and the detection object other than white, by only analyzing the resolution image 55, and can appropriately detect the detection object regardless of the color of the detection object.

[0079] In addition, the image processing device 10 makes the saturation image 53 a monochrome image, which is a monochrome image represented by the magnitude of the brightness value of each pixel replaced by the magnitude of the saturation value (saturation distribution) of each pixel of the bird's-eye view image 51 (color image). Therefore, the image processing device 10 can make both the brightness image 52 and the saturation image 53 monochrome images represented by the magnitude of the brightness value, and the analysis image 55 synthesized from the two can also be a monochrome image represented by the magnitude of the brightness value. Therefore, the image processing device 10 can analyze the analysis image 55 to detect the detection object in exactly the same way as the conventional method of analyzing the brightness image 52 to detect the detection object. Therefore, the image processing device 10 can appropriately detect the detection object regardless of the color of the detection object by a simple process of only analyzing the analysis image 55 as in the past.

[0080] Furthermore, the image processing device 10 creates an analysis image 55 by synthesizing the saturation extraction image 54 with the brightness image 52 instead of the saturation image 53. Here, in the saturation image 53, if colored light is irradiated in the original bird's-eye view image 51 (color image), for example, the ground is illuminated by the setting sun or lights, then the area different from the detection object ( Figure 4 The saturation of the shadow 2) in the example increases, or the overall saturation increases. Then, when such a saturation image 53 is synthesized with the brightness image 52 to create an analytical image 55, the brightness value of the area that is not the detection object in the analytical image 55 also becomes high, and proper detection of the detection object sometimes becomes difficult. In contrast, the image processing device 10 removes a pattern larger than the detection object in the saturation image 53 to create a saturation extraction image 54, and synthesizes the saturation extraction image 54 with the brightness image 52 to create an analytical image 55. Therefore, even if it is a saturation image 53 illuminated by colored light, the image processing device 10 can remove a pattern larger than the detection object in the analytical image 55, so that the area larger than the detection object with a high brightness value can be eliminated in the analytical image 55, and the detection object can be properly detected.

[0081] The image processing device 10 according to one embodiment of the image processing device of the present disclosure can obtain the following effects.

[0082] The image processing device 10 includes: a luminance image creation unit 22 that creates a luminance image 52 that indicates the luminance distribution in the color image (the bird's-eye view image 51 in the first embodiment) acquired by the camera 15; a saturation image creation unit 23 that creates a saturation image 53 that indicates the saturation distribution in the color image; and an analysis image creation unit 25 that creates an analysis image 55 for detecting a given object by synthesizing the luminance image 52 and the saturation image 53. Therefore, the image processing device 10 can create an analysis image 55 that emphasizes both a portion with a high luminance value and a portion with a high saturation value, and thus can appropriately detect the detection object regardless of the color of the detection object by simply analyzing the analysis image 55. Thus, since the image processing device 10 only needs to analyze the analysis image 55 created from the luminance image 52 and the saturation image 53 regardless of the color of the detection object, it can appropriately detect the detection object while preventing an increase in calculation load compared to conventional image processing devices.

[0083] In particular, the image processing device 10 obtains a normalized S value (saturation value) for each pixel based on the color image (bird's-eye image 51), and then multiplies the S value by a coefficient A to obtain an Sb value, which is a brightness value based on the scale of the brightness image 52. Then, the image processing device 10 creates a saturation image 53 using the Sb value of each pixel. Therefore, even if the color image (bird's-eye image 51) is acquired in a dark environment, the image processing device 10 can create a saturation image 53 represented by the brightness value of each pixel after the saturation distribution is converted, similar to the case of acquiring in a bright environment. Therefore, the image processing device 10 can perform appropriate detection of the detection object regardless of the brightness when the color image (bird's-eye image 51) is acquired.

[0084] Furthermore, the image processing device 10 includes an extracted image creation unit 24 that creates a saturation extracted image 54 by extracting a pattern corresponding to the detection object from the saturation image 53, and an analysis image creation unit 25 creates an analysis image 55 by synthesizing the saturation extracted image 54 with the brightness image 52 instead of the saturation image 53. Therefore, in the image processing device 10, even in the saturation image 53 irradiated with colored light, a pattern corresponding to the detection object can be extracted from the saturation extracted image 54, so that a region different from the detection object having a high brightness value can be removed from the analysis image 55, and appropriate detection of the detection object can be performed.

[0085] Furthermore, in the image processing device 10, the saturation image 53 is subjected to contraction processing and expansion processing by the extraction image creation unit 24, thereby removing patterns larger than the detection object to create the saturation extraction image 54. Therefore, the image processing device 10 can create the saturation extraction image 54 from which patterns larger than the detection object are removed by a simple process, and can eliminate a region larger than the detection object having a high brightness value in the analysis image 55, and can appropriately detect the detection object.

[0086] The image processing device 10 uses the color image as an overhead image 51 viewed from above. Therefore, the image processing device 10 can appropriately detect the detection object in the overhead image 51 for assisting driving in the vehicle 1 and can provide more appropriate driving assistance.

[0087] Therefore, the image processing device 10 as the first embodiment of the image processing device according to the present disclosure can create an image that facilitates detection of a given object while suppressing an increase in calculation load.

[0088] Although the image processing device of the present disclosure has been described above based on the first embodiment, the specific configuration is not limited to the first embodiment, and design changes and additions are permitted without departing from the gist of the inventions involved in each of the claims.

[0089] For example, in the first embodiment, the extraction image creation unit 24 creates the saturation extraction image 54 by performing contraction processing and expansion processing on the saturation image 53, thereby removing patterns larger than the detection object (see Fig.12 , Fig.13 ). However, the extraction image creation unit 24 may also create a saturation extraction image 54 by performing a contraction process and an expansion process on the saturation image 53, thereby removing patterns smaller than the detection object, that is, creating a saturation extraction image 54 by removing patterns of a size different from the detection object, and is not limited to the configuration of Example 1. Figure 6 , Fig.12 In addition, we also use Figures 15 to 17 Let me illustrate this example.

[0090] The extracted image creation unit 24 performs the dilation process to create a processed image 561 (see FIG. 5 ) as the saturation image 53 after performing the contraction process the maximum number of times without removing the detection object. Fig.15 ).exist Figure 6 In the example of the saturation image 53 shown in FIG. 1 , the pattern D2 to be detected is a line with a width of 4 pixels, so the expansion process is performed once after the contraction process is performed once. Fig.15 As shown, the extracted image creation unit 24 creates a processed image 561 in which the pattern D1 and Figure 6 The original saturation image 53 is the same as that of the original saturation image 53, and the brightness value of the center of the pattern D2 is Figure 6 The original saturation image 53 is reduced and the pattern D3 is removed. Figure 6 The original saturation image 53 is subtracted Fig.15 The processed image 561 (the saturation value (brightness value) of each pixel) is created Fig.16 The processed image 562 (saturation image 53) shown in FIG. Fig.12 The saturation extracted image 54 is subtracted from Fig.16 The processed image 562 (the saturation value (brightness value) of each pixel) is created Fig.17 The saturation extracted image 54 is shown. Fig.17 The saturation extraction image 54 in the image is obtained by removing patterns (D1, D3) of different sizes from the pattern D2 as the detection object from the saturation image 53. The image processing device 10 creates an analysis image 55 by synthesizing the saturation extraction image 54 created as described above with the brightness image 52, thereby enabling more appropriate detection of the detection object.

[0091] In addition, in the first embodiment, the extracted image creation unit 24 creates the saturation extracted image 54 by performing contraction processing and expansion processing on the saturation image 53. That is, in the first embodiment, the extracted image creation unit 24 creates the saturation extracted image 54 in which the detection object is extracted as a result by removing patterns larger than the detection object (including the case where patterns smaller than the detection object are also removed as described above). However, the extracted image creation unit 24 is not limited to the configuration of the first embodiment as long as it can create the saturation extracted image 54 in which the pattern corresponding to the detection object is extracted from the saturation image 53. As an example, the extracted image creation unit 24 can create the saturation extracted image 54 by performing image analysis on the saturation image 53 to extract a pattern of a size corresponding to the detection object. In this case, the extracted image creation unit 24 extracts the detection object (pattern) by performing edge extraction processing on the saturation image 53 and extracting a theoretical comparison area representing the size and shape of the detection object in the image obtained by the edge extraction processing. In addition, in the extraction image creation unit 24, as long as it is possible to extract a pattern corresponding to the detection object, it can also be extracted by other methods, and is not limited to this configuration. According to this configuration, the image processing device 10 can create a saturation extraction image 54 after extracting a pattern corresponding to the detection object in a simple process, and can eliminate a region different from the detection object with a high brightness value in the analysis image 55, and can appropriately detect the detection object.

[0092] Furthermore, in Embodiment 1, the extracted image creation section 24 performs the contraction processing and the expansion processing using the filter F of 9 pixels in the 3×3 frame. However, if the filter F performs the contraction processing and the expansion processing by adopting the saturation value (brightness value) of the pixel located in the surrounding multiple frames in the saturation value (brightness value) of the pixel located in the center frame, the number of frames (number of pixels) can be set appropriately without being limited to the configuration of Embodiment 1.

[0093] In the first embodiment, the saturation image creation unit 23 sets the coefficient A to be multiplied by the S value (saturation value) when calculating the Sb value to 255. However, the coefficient A can be appropriately set in consideration of the balance between the brightness value (brightness image 52) and the saturation value (saturation image 53), and is not limited to the configuration of the first embodiment.

[0094] In the first embodiment, the extracted image creation unit 24 creates the saturation extracted image 54 by performing an extraction process on the saturation image 53 created by the saturation image creation unit 23. However, if the extracted image creation unit 24 creates the saturation extracted image 54 based on the saturation image 53, the saturation extracted image 54 may be created by performing an extraction process after smoothing the saturation image 53, and is not limited to the configuration of the first embodiment. According to this configuration, even in the case where a large amount of noise is included in the saturation image 53 created by the saturation image creation unit 23, a saturation extracted image 54 in which a pattern corresponding to the detection object is more appropriately extracted can be created.

[0095] In the first embodiment, the extraction image creation unit 24 creates the saturation extraction image 54 by performing the contraction process and the expansion process on the saturation image 53 created by the saturation image creation unit 23. However, the extraction image creation unit 24 is not limited to the configuration of the first embodiment, as long as it creates the saturation extraction image 54 based on the saturation image 53, and it may perform the contraction process and the expansion process after binarizing (binary processing) the saturation image 53 to create the saturation extraction image 54. According to this configuration, it is possible to easily remove a pattern different in size from the detection object while simplifying the contraction process and the expansion process.

[0096] In the first embodiment, the bird's-eye view image 51 is used as a color image as a source of the brightness image 52 and the saturation image 53. However, if the color image becomes a source for the brightness image creating unit 22 to create the brightness image 52 and becomes a source for the saturation image creating unit 23 to create the saturation image 53, the color image may be obtained by a camera that acquires an image of a portion of the surroundings of the vehicle 1 or by a camera installed at a position different from the vehicle 1, and is not limited to the configuration of the first embodiment.

[0097] CROSS-REFERENCE TO RELATED APPLICATIONS

[0098] This application claims priority based on Japanese Patent Application No. 2019-037647 filed with the Japan Patent Office on March 1, 2019, the entire disclosure of which is incorporated herein by reference.

Claims

1. An image processing device, characterized in that: have: a brightness image creating unit that creates a brightness image representing brightness distribution in a color image acquired by a camera; a saturation image creating unit, which creates a saturation image representing the saturation distribution in the color image and replaces the magnitude of the saturation value with the magnitude of the brightness value; an analysis image creation unit that creates an analysis image for detecting a given object by synthesizing the brightness image and the saturation image; as well as an extracted image creation unit, which creates a saturation extracted image by removing a pattern larger than the detection object from the saturation image by performing a contraction process and an expansion process on the saturation image, The analytical image creation unit creates the analytical image by adding and synthesizing the saturation extraction image to the brightness image instead of the saturation image.

2. The image processing device according to claim 1, characterized in that The extracted image creation unit creates the saturation extracted image by performing the contraction process and the expansion process on the saturation image to remove a pattern having a size different from that of the detection object.

3. The image processing device according to claim 1 or 2, characterized in that The color image is a bird's-eye view image viewed from above.

Images