Image recognition device and image recognition program

Abstract

To provide an image recognition device and an image recognition program that can improve the accuracy of road boundary line detection while suppressing the processing load. [Solution] The image recognition device 11 includes an image acquisition unit 51 that acquires image data of the area around the vehicle from an in-vehicle camera 20; an estimation unit 52 that uses a trained inference model to estimate a region of interest that includes lane markings displayed on the road surface from the image data, and estimates road surface information and line information relating to the lane markings included in the region of interest; a lane marking detection unit 53 that calculates the brightness difference in the region of interest estimated by the estimation unit 52, detects edge points based on whether the brightness difference is greater than or equal to a brightness threshold, and detects lane markings based on the edge points; and a threshold setting unit 54 that changes the brightness threshold based on the road surface information and line information estimated by the estimation unit 52.

Description

Technical Field

[0001] The present disclosure relates to an image recognition device and an image recognition program for recognizing partition lines from an image.

Background Art

[0002] Various image recognition devices have been conventionally proposed as image recognition devices that recognize partition lines such as parking frames provided on a road surface based on captured images. For example, Patent Document 1 proposes an image recognition device that corrects the luminance value of pixels satisfying a predetermined yellow condition to improve the recognition accuracy of yellow lines. Further, Patent Document 2 proposes an image recognition device that detects edge points having a luminance difference equal to or greater than a threshold value to recognize partition lines, and that changes the threshold value according to the distance from the vehicle. According to the image recognition device of Patent Document 2, the detection accuracy of partition lines can be improved even if a region far from the vehicle has low contrast.

Prior Art Literature

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of Invention

Problems to be Solved by the Invention

[0004] However, the image recognition device of Patent Document 1 corrects the luminance values over the entire screen including portions unrelated to yellow lines, which has a problem of increasing the processing load of the image recognition device. Furthermore, Patent Document 1 requires yellow discrimination based on saturation, which also increases the processing load in this respect.

[0005] Furthermore, the image recognition device described in Patent Document 2 changed the threshold according to the distance from the vehicle, which meant that if there were low-contrast areas near the vehicle, it was not possible to improve the detection accuracy in those areas.

[0006] As described above, conventional image recognition devices have problems such as increased processing load or insufficient improvement in detection accuracy because they do not properly recognize image areas that require correction.

[0007] The present invention has been made in view of the above problems, and its objective is to provide an image recognition device and an image recognition program that can improve the detection accuracy of road markings while suppressing the processing load. [Means for solving the problem]

[0008] An image recognition device to solve the above problem is: An image acquisition unit that acquires images of the area around the vehicle from an imaging device, An estimation unit that uses a trained inference model to estimate a region of interest from the captured image that includes lane markings displayed on the road surface, and estimates road surface information and line information relating to the lane markings included in the region of interest, A boundary line detection unit calculates the luminance difference in the region of interest estimated by the estimation unit, detects edge points based on whether the luminance difference is greater than or equal to a luminance threshold, and detects the boundary lines based on the edge points. The system includes a threshold setting unit that changes the brightness threshold based on the road surface information and line information estimated by the estimation unit.

[0009] According to the above configuration, the detection process for lane markings is performed by narrowing down the area using an estimation model, thereby improving detection accuracy while suppressing the processing load. Furthermore, the estimation model is used to estimate road surface information and line information in the area of ​​interest, and the brightness threshold used for lane marking detection is changed based on the road surface information and line information. This improves the accuracy of lane marking detection. In addition, because an inference model is used, the processing load can be reduced compared to changing the brightness threshold using a rule-based method.

[0010] The image recognition program to solve the above problem is: An image recognition program for an image recognition device that recognizes lane markings from captured images of the area around a vehicle, The aforementioned image recognition device, An image acquisition step involves obtaining images of the area around the vehicle from an imaging device, An estimation step in which a trained inference model is used to estimate a region of interest from the captured image that includes lane markings displayed on the road surface, and to estimate road surface information and line information relating to the lane markings included in the region of interest, A boundary line detection step which involves calculating the luminance difference in the region of interest estimated by the estimation step, detecting edge points based on whether the luminance difference is greater than or equal to a luminance threshold, and detecting boundary lines based on said edge points, Based on the road surface information and line information estimated in the estimation step, a threshold setting step is performed to change the brightness threshold.

[0011] According to the above configuration, the detection process for lane markings is performed by narrowing down the area using an estimation model, thereby improving detection accuracy while suppressing the processing load. Furthermore, the estimation model is used to estimate road surface information and line information in the area of ​​interest, and the brightness threshold used for lane marking detection is changed based on the road surface information and line information. This improves the accuracy of lane marking detection. In addition, because an inference model is used, the processing load can be reduced compared to changing the brightness threshold using a rule-based method. [Brief explanation of the drawing]

[0012] [Figure 1] A diagram showing the configuration of the driver assistance system. [Figure 2] An overhead view showing the corner of a parking space. [Figure 3] An overview showing the area of ​​interest. [Figure 4] An overview showing the area of ​​interest in the modified example. [Figure 5] A flowchart illustrating the flow of the parking assistance process. [Figure 6] A flowchart illustrating the process of detecting parking spaces. [Modes for carrying out the invention]

[0013] Hereinafter, embodiments of the image recognition device and image recognition program described in this disclosure will be described in detail with reference to the drawings. In principle, the same or corresponding parts in the drawings will be denoted by the same reference numerals between embodiments and modifications, and their descriptions will not be repeated.

[0014] As shown in Figure 1, the vehicle 100, which is the vehicle itself, is equipped with a driver assistance device 10 that provides driving assistance for the vehicle 100. The driver assistance device 10 includes an image recognition device 11 that performs image recognition on captured images (image data) input from an onboard camera 20 which is an imaging device, and a vehicle control device 12 which is a vehicle control unit that performs driving control of the vehicle.

[0015] The in-vehicle camera 20 captures the surrounding conditions of the vehicle 100, and may be a monocular camera or a compound-eye camera. The in-vehicle camera 20 captures moving images at a predetermined frame rate to generate image data, and inputs the image data to the image recognition device 11. That is, the in-vehicle camera 20 generates image data related to a moving image (video) composed of a plurality of frames (still images). Note that the number, position, and type of the in-vehicle camera 20 may be changed arbitrarily. In addition, although the image data in the present embodiment is a moving image, it may be a still image. In the present embodiment, the in-vehicle camera 20 includes a front camera 21a whose imaging range is the front of the vehicle 100, a right side camera 21b whose imaging range is the right side of the vehicle 100, a left side camera 21c whose imaging range is the left side of the vehicle 100, and a rear camera 21d whose imaging range is the rear of the vehicle.

[0016] The image recognition device 11 is mainly configured by a microcomputer including an arithmetic processing unit such as a CPU and a storage device such as various memories. The functions provided by the microcomputer can be provided by software recorded in a substantial memory device and a computer executing the software, software alone, hardware alone, or a combination thereof. For example, when the microcomputer is provided by an electronic circuit that is hardware, it can be provided by a digital circuit including a large number of logic circuits or an analog circuit. For example, the microcomputer executes a program stored in a non-transitory tangible storage medium included in itself. The program includes, for example, a program that realizes an image recognition function for recognizing the surrounding conditions of the vehicle 100 based on the image data input from the in-vehicle camera 20. By executing the program, a method corresponding to the program is executed. The non-transitory tangible storage medium is, for example, a non-volatile memory. Note that the program stored in the non-transitory tangible storage medium can be downloaded and updated via a communication network such as the Internet, for example, by OTA (Over The Air) or the like.

[0017] Similar to the image recognition device 11, the vehicle control device 12 is configured mainly with a microcomputer. The vehicle control device 12 has various functions related to drive control of the vehicle 100, and with these functions, controls the actuator 30 based on sensor information input from outside the vehicle control device 12. Note that image data input from the image recognition device 11 or the like is included in the sensor information. The sensor information also includes information from various sensors for measuring the traveling state of the vehicle, such as a vehicle speed sensor and a yaw rate sensor, as well as information from sensors for detecting other vehicles and obstacles, such as millimeter wave radars and laser radars (LiDAR). The sensor information may further include various sensors for detecting various operation amounts by the driver, such as an accelerator sensor that detects the accelerator operation amount by the driver, a brake sensor that detects the brake operation amount, and a steering angle sensor that detects the steering amount (steering angle) of the steering wheel by the driver.

[0018] As a representative function among the various functions related to drive control of the vehicle 100, for example, there is an Advanced Parking Assist function (APA). This Advanced Parking Assist function is a type of function related to driving assistance. Note that other vehicle control functions related to driving assistance may also be provided. This vehicle control function is realized by the vehicle control device 12 executing a program stored in the storage device of the driving assistance device 10 or the vehicle control device 12. Note that this program may be downloadable and updatable via a communication network.

[0019] The actuator 30 includes, for example, an actuator for driving the vehicle 100, such as the main motor of the vehicle 100. It also includes actuators for controlling the behavior of the vehicle 100, such as an actuator for operating the steering wheel, an actuator for operating the brake pedal, and an actuator for operating the accelerator pedal. The actuator 30 may also include devices for operating equipment of the vehicle 100, such as a display, speakers, indicators, and headlights. The driving and operation of the vehicle 100 are controlled by this actuator 30. When the vehicle control device 12 performs the advanced parking assist function, it controls the driving of the vehicle 100 to park the vehicle 100 within the parking space 40 by controlling these actuators 30.

[0020] Incidentally, as a prerequisite for implementing the advanced parking assist function, the image recognition device 11 needs to recognize parking spaces displayed on the road surface from image data from the in-vehicle camera 20. However, parking spaces can be set up in various locations, and the lines that make up each side of the parking space can also take on various forms. For example, parking spaces may be set up on roads paved with ordinary asphalt, or on concrete surfaces such as apartment building parking lots. They may also be set up on surfaces covered with grass or gravel. Furthermore, the lines may be white or yellow. For this reason, if the recognition of parking spaces and lines is performed uniformly, it may be difficult to recognize them depending on the situation.

[0021] Therefore, in this embodiment, the image recognition device 11 is configured to improve the detection accuracy of the parking space 40 by using an inference model in combination. The image recognition device 11 of this embodiment will be described below.

[0022] The image recognition device 11 includes the functions of an image acquisition unit 51, an estimation unit 52, a road boundary detection unit 53, and a threshold setting unit 54. These functions are realized by the arithmetic processing unit of the image recognition device 11 executing an image recognition program stored in a memory device.

[0023] The image acquisition unit 51 acquires image data (captured images) of the area around the vehicle from the in-vehicle camera 20. The estimation unit 52 uses a trained inference model to estimate from the image data the area containing the lane markings displayed on the road surface. Furthermore, the estimation unit 52 uses the inference model to estimate road surface information and line information related to the lane markings included in that area. The inference model is, for example, a machine learning-trained inference model such as a deep neural network. Here, the estimation unit 52 in this embodiment will be described in more detail.

[0024] The estimation unit 52 of this embodiment first estimates the positions of the corners 41-44 (the four corners) of the parking space 40 displayed on the road surface from the image data using an inference model (see Figure 2). In other words, the inference model of this embodiment is pre-machine-trained so that it can estimate the corners 41-44 from the image data of the parking space 40 displayed on the road surface. Figure 2 is an overhead view (bird's-eye view) showing the positions of the corners 41-44 of the left parking space 40.

[0025] Next, the estimation unit 52 uses an inference model to estimate the image region containing the demarcation lines 45-48 that make up each side of the parking space 40, based on the positions of the corners 41-44. Hereinafter, the image region containing the demarcation lines 45-48 that make up each side of the parking space 40 will simply be referred to as the "region of interest".

[0026] Specifically, first, the estimation unit 52 determines pairs of adjacent corners 41-44 from the identified corners 41-44. At this time, it avoids determining pairs of corners 41-44 that are diagonally opposite each other. In other words, it determines pairs other than the pair of the upper left corner 41 and the lower right corner 43, and the pair of the upper right corner 44 and the lower left corner 42 in Figure 2. Then, as shown in Figures 3(a) to 3(d), the estimation unit 52 estimates the rectangular area centered on the straight line connecting the paired corners 41-44 as the region of interest 45a-48a for each pair. Figures 3(a) to 3(d) are overhead views (bird's-eye views) showing the locations of the regions of interest 45a-48a of the left parking space 40.

[0027] As described above, the inference model of this embodiment has undergone machine learning in advance so that it can estimate the regions of interest 45a to 48a from the positions of the corners 41 to 44 of the parking space 40.

[0028] Note that the estimation method for the regions of interest 45a to 48a is just one example and can be changed as needed. For example, the estimation unit 52 may use an inference model to estimate a rectangular region 140 containing all corners 41 to 44 as the region of interest, as shown by the dashed line in Figure 4. Alternatively, for each of the corners 41 to 44, a certain region centered on a corner 41 to 44 may be estimated as the region of interest.

[0029] The estimation unit 52 then uses an inference model to estimate road surface information and line information related to lane markings included in each region of interest 45a to 48a. Road surface information refers to the material or color of the road surface. In this embodiment, material refers to information indicating whether the road surface is asphalt, concrete, brick, grass, soil, gravel, etc. Line information refers to the color information of lane markings (parking spaces). Therefore, the inference model in this embodiment has undergone machine learning in advance so that it can estimate road surface information and line information in regions of interest 45a to 48a.

[0030] Furthermore, the estimation unit 52 uses an inference model to estimate whether the road surface or the lane markings 45 to 48 is brighter in each region of interest 45a to 48a. Therefore, the inference model in this embodiment has undergone prior machine learning so that it can estimate whether the road surface or the lane markings 45 to 48 is brighter in each region of interest 45a to 48a.

[0031] The boundary line detection unit 53 extracts edge points, which are pixels with large changes in brightness values, in the regions of interest 45a to 48a identified by the estimation unit 52. Specifically, while scanning in a predetermined direction (left-right or up-down) within the regions of interest 45a to 48a, it searches for pixels where the change in brightness value is greater than or equal to a brightness threshold, and extracts up-edge points and down-edge points.

[0032] An up-edge point is the point where the brightness value changes from a low brightness value to a high brightness value, more specifically, the point where the amount of change in brightness value from a low brightness value to a high brightness value is greater than or equal to the brightness threshold. A down-edge point is the point where the brightness value changes from a high brightness value to a low brightness value, more specifically, the point where the amount of change in brightness value from a high brightness value to a low brightness value is greater than or equal to the brightness threshold. The boundary line detection unit 53 extracts edge points from regions of interest 45a to 48a by repeatedly scanning in a predetermined direction and shifting its position in an orthogonal direction perpendicular to the predetermined direction.

[0033] Subsequently, the parking space line detection unit 53 extracts parking space lines 45 to 48 by connecting the extracted edge points. Specifically, from each extracted edge point, a roughly rectangular area formed by multiple edge points is extracted as parking space lines 45 to 48. More specifically, an up edge point and a down edge point located on the opposite side of that up edge point and closest to it are paired. Then, a roughly rectangular area in which these pairs are arranged at intervals within a predetermined interval threshold is extracted as parking space lines 45 to 48. The parking space line detection unit 53 detects the parking space 40 by connecting the parking space lines 45 to 48 extracted in this way.

[0034] The threshold setting unit 54 changes the luminance threshold according to the combination of line information and road surface information estimated by the estimation unit 52. More specifically, the threshold setting unit 54 identifies the contrast between the lane lines 45-48 and the road surface from the estimated combination of line information and road surface information, and lowers the luminance threshold if the contrast is lower than the reference value. If the contrast is equal to or greater than the reference value, the luminance threshold is left at the initial value (standard value).

[0035] For example, if the road surface information is asphalt (close to black) and the line information is white, the threshold setting unit 54 determines that the contrast is above the standard value (high contrast). On the other hand, if the road surface information is asphalt (close to black) and the line information is yellow, the threshold setting unit 54 determines that the contrast is below the standard value (low contrast). Similarly, if the road surface information is concrete (close to gray) and the line information is white, the threshold setting unit 54 determines that the contrast is below the standard value (low contrast).

[0036] In this embodiment, a map is pre-stored that associates line information and road surface information with contrast. The threshold setting unit 54 refers to this map to identify the contrast and sets the luminance threshold based on that contrast. The map is created through experiments or simulations and is pre-stored.

[0037] Furthermore, the threshold setting unit 54 changes the sign of the luminance threshold depending on whether the road surface or the lane markings 45 to 48 are brighter in the region of interest 45a to 48a (i.e., the direction of the luminance difference). In other words, if the lane markings 45 to 48 are brighter than the road surface (positive contrast), the threshold setting unit 54 sets the sign of the luminance threshold to positive (remaining in the initial state), and if the lane markings 45 to 48 are darker than the road surface (negative contrast), the sign of the luminance threshold is set to negative. Whether the road surface or the lane markings 45 to 48 are brighter (i.e., the direction of the luminance difference) is estimated by the estimation unit 52 as described above.

[0038] Next, the parking assistance process in this embodiment will be explained with reference to Figures 5 and 6. The parking assistance process is performed by the image recognition device 11 and the vehicle control device 12.

[0039] The image acquisition unit 51 of the image recognition device 11 acquires image data from the in-vehicle camera 20 (step S101). Next, the estimation unit 52 of the image recognition device 11 estimates various information from the acquired image data using an inference model (step S102). In step S102, as described above, the estimation unit 52 estimates the regions of interest 45a to 48a, as well as road surface information and line information in each region of interest 45a to 48a. The estimation unit 52 also estimates whether the road surface or the lane markings 45 to 48 are brighter (direction of brightness difference) in the regions of interest.

[0040] Next, the image recognition device 11 performs the parking space detection process shown in Figure 6 (step S103). As shown in Figure 6, the threshold setting unit 54 of the image recognition device 11 first determines whether the lane markings 45 to 48 are brighter than the road surface in the region of interest 45a to 48a (step S200). If the lane markings 45 to 48 are brighter than the road surface (positive contrast), the threshold setting unit 54 sets the sign of the luminance threshold to positive (step S201). Note that the initial value of the luminance threshold is set to positive. On the other hand, if the determination in step S201 is negative, that is, if the lane markings 45 to 48 are darker than the road surface (negative contrast), the threshold setting unit 54 sets the sign of the luminance threshold to negative (step S202).

[0041] After the processing in step S201 or step S202 is performed, the threshold setting unit 54 refers to the map and identifies the contrast from the combination of line information and road surface information estimated by the estimation unit 52 (step S203). Then, the threshold setting unit 54 determines whether the identified contrast is above a reference value (step S204). If this determination result is positive, the threshold setting unit 54 sets the normal luminance threshold (step S205). On the other hand, if the determination result in step S204 is negative, the threshold setting unit 54 sets a luminance threshold for low contrast (step S206). Specifically, the luminance threshold is set lower than the normal luminance threshold.

[0042] Then, as described above, the lane line detection unit 53 of the image recognition device 11 extracts edge points, which are pixels with large changes in brightness values ​​in the region of interest 45a to 48a (step S207). When extracting edge points, the brightness threshold set by the threshold setting unit 54 in steps S201 to S206 is used. After that, the lane line detection unit 53 extracts lane lines 45 to 48 by connecting the extracted edge points (step S208), and detects the parking space 40 by connecting the extracted lane lines 45 to 48 (step S209). This completes the parking space detection process in step S103.

[0043] Returning to Figure 5, once the parking space detection process in step S103 is complete, the vehicle control device 12 controls the actuator 30 of the vehicle 100 to move the vehicle 100 into the parking space detected by the image recognition device 11 and park it within the parking space (step S104). Then, the parking assistance process ends.

[0044] The effects of the image recognition device 11 of the above embodiment will be explained.

[0045] The estimation unit 52 uses an inference model to estimate regions of interest 45a to 48a from the image data, and detects the demarcation lines 45 to 48 within these regions of interest 45a to 48a. In this way, by narrowing down the region using the estimation model and then performing the detection process for the demarcation lines 45 to 48, it is possible to improve detection accuracy while suppressing the processing load.

[0046] Furthermore, an estimation model is used to estimate road surface information and line information in the region of interest 45a to 48a, and the luminance threshold used for detecting lane markings 45 to 48 is changed based on the road surface information and line information. Since the luminance threshold is changed according to the characteristics of the road surface and lane markings 45 to 48 in the region of interest 45a to 48a, the detection accuracy of lane markings 45 to 48 can be improved. In addition, since the road surface information and line information in the region of interest 45a to 48a are estimated using an inference model, the processing load can be reduced compared to changing the luminance threshold using a rule-based method.

[0047] The estimation unit 52 uses an inference model to estimate the positions of the corners 41-44 of the parking space 40 displayed on the road surface, and estimates the regions of interest 45a-48a that include the lane markings 45-48 that constitute each side of the parking space 40 from the positions of the corners 41-44. In this way, characteristic parts of the parking space 40, namely the corners 41-44, are estimated, and the regions of interest 45a-48a are estimated from the corners 41-44, so that the regions of interest 45a-48a can be set within an appropriate range. Therefore, the detection accuracy of the lane markings 45-48 and the parking space 40 can be improved.

[0048] The estimation unit 52 estimates information about the color of the lane markings 45-48 as line information and information about the material or color of the road surface as road surface information. The threshold setting unit 54 then changes the luminance threshold according to the combination of line information and road surface information. Specifically, the threshold setting unit 54 identifies the contrast between the lane markings 45-48 and the road surface from the combination of line information and road surface information, and lowers the luminance threshold if the contrast is lower than the reference value. This makes it possible to set an appropriate luminance threshold according to the combination of the road surface and lane markings, thereby improving the detection accuracy of the lane markings 45-48. Furthermore, since the line information and road surface information are estimated using an inference model only for the region of interest 45a-48a, the processing load can be reduced. In addition, the amount of training required when training the inference model can be reduced.

[0049] Furthermore, the threshold setting unit 54 changed the sign of the brightness threshold depending on whether the road surface or the lane markings 45-48 were brighter. This improves the detection accuracy of the lane markings 45-48. In this case, since the determination of whether the road surface or the lane markings 45-48 are brighter is estimated using an inference model only in the region of interest 45a-48a, the processing load can be reduced. In addition, the amount of training required to train the inference model can be reduced.

[0050] (modified version) In the above embodiment, an inference model was used to estimate the regions of interest 45a to 48a that include the demarcation lines 45 to 48 that constitute each side of the parking space 40, starting from the positions of the corners 41 to 44 of the parking space 40. However, the regions of interest that include the demarcation lines 45 to 48 that constitute each side of the parking space 40, starting from the positions of the corners 41 to 44 of the parking space 40, may be identified based on predetermined rules.

[0051] In the above embodiment, the regions of interest 45a to 48a may be connected and set as a single region of interest.

[0052] In the above embodiment, the threshold setting unit 54 identifies the contrast between the lane lines 45-48 and the road surface, and lowers the luminance threshold when the contrast is lower than the reference value. However, the luminance threshold may be changed according to the contrast. In this case, it is desirable to lower the luminance threshold when the contrast is low compared to when it is high, and to raise the luminance threshold when the contrast is high compared to when it is low. Specifically, a map that associates combinations of line information and road surface information with luminance thresholds can be stored in advance, and the threshold setting unit 54 can set the luminance threshold by referring to this map. Alternatively, as another example, a map that associates contrast with luminance thresholds can be stored in advance, and the threshold setting unit 54 can set the luminance threshold by referring to this map. The map can be created by experimentation or simulation and stored in advance.

[0053] • In the above embodiment, a parking space 40 was detected, but this method may also be applied to detecting lane markings or road boundary markings indicating the lane in which the vehicle 100 is traveling.

[0054] The following is an addendum regarding the technical ideas that can be derived from the above embodiments and modifications. [Configuration 1] An image acquisition unit (51) acquires captured images of the area around the vehicle from the imaging device (20), An estimation unit (52) uses a trained inference model to estimate a region of interest (45a to 48a) from the captured image that includes lane markings (45 to 48) displayed on the road surface, and estimates road surface information and line information relating to the lane markings included in the region of interest, A boundary line detection unit (53) calculates the luminance difference in the region of interest estimated by the estimation unit, detects edge points based on whether the luminance difference is greater than or equal to a luminance threshold, and detects the boundary line based on the edge points, An image recognition device (11) comprising a threshold setting unit (54) that changes the brightness threshold based on the road surface information and line information estimated by the estimation unit. [Configuration 2] The aforementioned image recognition device detects parking spaces (40) displayed on the road surface. The image recognition device according to Configuration 1, wherein the estimation unit uses the inference model to estimate the positions of the corners (41-44) of the parking space displayed on the road surface, and estimates the region of interest which includes the demarcation lines that constitute each side of the parking space from the positions of the corners. [Configuration 3] The estimation unit estimates information regarding the color of the lane lines as line information, and estimates information regarding the material or color of the road surface as road surface information. The image recognition device according to configuration 1 or 2, wherein the threshold setting unit changes the brightness threshold according to the combination of the line information and road surface information estimated by the estimation unit. [Structure 4] The image recognition device according to configuration 3, wherein the threshold setting unit identifies the contrast between the lane markings and the road surface from the combination of the line information and road surface information estimated by the estimation unit, and lowers the brightness threshold when the contrast is low compared to when it is high. [Composition 5] The estimation unit estimates whether the road surface or the lane markings are brighter in the region of interest. The image recognition device according to any one of configurations 1 to 4, wherein the threshold setting unit changes the sign of the luminance threshold depending on whether the road surface or the lane markings are brighter. [Composition 6] An image recognition program for an image recognition device (10) that recognizes lane markings from captured images of the area around a vehicle, The aforementioned image recognition device, An image acquisition step involves obtaining an image of the area around the vehicle from the imaging device (20), An estimation step in which a trained inference model is used to estimate a region of interest (45a to 48a) containing lane markings (45 to 48) displayed on the road surface from the captured image, and to estimate road surface information and line information relating to the lane markings included in the region of interest, A boundary line detection step which involves calculating the luminance difference in the region of interest estimated by the estimation step, detecting edge points based on whether the luminance difference is greater than or equal to a luminance threshold, and detecting boundary lines based on said edge points, An image recognition program that performs a threshold setting step to change the brightness threshold based on the road surface information and line information estimated in the estimation step. [Explanation of Symbols]

[0055] 11...Image recognition device, 12...Vehicle control device, 20...In-vehicle camera, 30...Actuator, 40...Parking space, 41-44...Corner, 45-48...Parking line, 45a-48a...Region of interest, 51...Image acquisition unit, 52...Estimation unit, 53...Parking line detection unit, 54...Threshold setting unit, 100...Vehicle.

Claims

1. An image acquisition unit (51) acquires captured images of the area around the vehicle from the imaging device (20), An estimation unit (52) uses a trained inference model to estimate a region of interest (45a to 48a) from the captured image that includes lane markings (45 to 48) displayed on the road surface, and estimates road surface information and line information relating to the lane markings included in the region of interest, A boundary line detection unit (53) calculates the luminance difference in the region of interest estimated by the estimation unit, detects edge points based on whether the luminance difference is greater than or equal to a luminance threshold, and detects the boundary lines based on the said edge points, An image recognition device (11) comprising a threshold setting unit (54) that changes the brightness threshold based on the road surface information and line information estimated by the estimation unit.

2. The aforementioned image recognition device detects parking spaces (40) displayed on the road surface. The image recognition device according to claim 1, wherein the estimation unit uses the inference model to estimate the positions of the corners (41-44) of the parking space displayed on the road surface, and estimates the region of interest which includes the demarcation lines that constitute each side of the parking space from the positions of the corners.

3. The estimation unit estimates information regarding the color of the lane lines as line information, and estimates information regarding the material or color of the road surface as road surface information. The image recognition apparatus according to claim 1 or 2, wherein the threshold setting unit changes the brightness threshold according to the combination of the line information and the road surface information estimated by the estimation unit.

4. The image recognition device according to claim 3, wherein the threshold setting unit identifies the contrast between the lane line and the road surface from the combination of the line information and road surface information estimated by the estimation unit, and when the contrast is low, it lowers the brightness threshold compared to when it is high.

5. The estimation unit estimates whether the road surface or the lane markings are brighter in the region of interest. The image recognition device according to claim 1 or 2, wherein the threshold setting unit changes the sign of the luminance threshold depending on whether the road surface or the lane markings are brighter.

6. An image recognition program for an image recognition device (10) that recognizes lane markings from captured images of the area around a vehicle, The aforementioned image recognition device, An image acquisition step involves obtaining an image of the area around the vehicle from the imaging device (20), An estimation step in which a trained inference model is used to estimate a region of interest (45a to 48a) that includes lane markings (45 to 48) displayed on the road surface from the captured image, and to estimate road surface information and line information relating to the lane markings included in the region of interest, A boundary line detection step which involves calculating the luminance difference in the region of interest estimated by the estimation step, detecting edge points based on whether the luminance difference is greater than or equal to a luminance threshold, and detecting boundary lines based on said edge points, An image recognition program that performs a threshold setting step to change the brightness threshold based on the road surface information and line information estimated in the estimation step.

Images