Image processing device, image processing method, and program

Abstract

To provide an image processing device, image processing method and program for estimating road demarcation lines stably usable for travel control of mobile objects.SOLUTION: An image processing device 100 to be mounted on a self vehicle M includes: a setting part for setting an initial value of at least one parameter out of parameters of a function to approximate a road boundary in an image captured by a mounted camera and representing a forward area, based on an identification value of another parameter out of the parameters; a model parameter updating part for sequentially updating a parameter at a prescribed time instant based on a parameter of the prescribed time instant at the previous time instant and a constraint condition set to the parameter at the previous time instant; and a drive control part for executing drive control or drive support of a mobile object based on the road boundary approximated by a function having the updated parameter.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program. [Background technology]

[0002] Conventionally, there are known technologies that estimate the road markings on the road a vehicle is traveling on and control the vehicle's movement based on the estimated road markings. For example, Patent Document 1 discloses a technology that selects multiple three-dimensional objects from images captured by a camera mounted on a vehicle, estimates road markings based on the positions of the selected three-dimensional objects, and sets a target speed for the vehicle according to the curvature of the estimated road markings. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-60885 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The technology described in Patent Document 1 estimates road markings by fitting curves to multiple positions extracted from images of the road on which a vehicle is traveling. However, with this method, due to constraints on the degrees of freedom of the curves to be fitted, the estimated road markings can have large errors, and fluctuations in the positions extracted from the images from time to time can cause deviations in the estimated road markings. As a result, it was sometimes not possible to stably utilize the estimated road markings for vehicle driving control.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide an image processing device, an image processing method, and a program that can estimate road markings that can be reliably used for controlling the movement of a moving object. [Means for solving the problem]

[0006] The image processing apparatus, image processing method, and program according to this invention employ the following configuration. (1) An image processing device according to one aspect of the present invention includes: a setting unit that sets the initial value of at least one parameter of a quadratic function that approximates the road boundary in an image representing the area in front of a moving body, captured by a camera mounted on the moving body, based on the identification value of the other parameter; an update unit that sequentially updates the parameter at a predetermined time based on the parameter at the previous time and constraint conditions set for the parameter; and a control unit that performs driving control or driving assistance for the moving body based on the road boundary approximated by the function having the updated parameter.

[0007] (2): In the embodiment of (1) above, the function is a quadratic function, the setting unit sets the initial value of the linear coefficient of the quadratic function based on the identification value of the road boundary recognized with respect to the lateral direction of the image, and the updating unit updates the parameters at a predetermined time by setting the forgetting gain of the quadratic coefficient and the linear coefficient of the parameters at the previous time to a value less than 1.

[0008] (3) In the embodiment of (1) or (2) above, the update unit divides the image into predetermined intervals, sequentially updates the parameters of the function that approximates the road boundary for each divided region, and then synthesizes them to sequentially update the parameters of the function that approximates the road boundary.

[0009] (4): In any of the embodiments described in (1) to (3) above, the update unit obtains the probability value using a trained model that has been trained to output a probability value indicating the probability of the existence of a road boundary for each coordinate of the image in response to the input of the image, and updates the parameters based on the probability value and the coordinate.

[0010] (5) In the embodiment of (4) above, the update unit sequentially updates the parameters to minimize the error between the component of the coordinate in the first direction and the estimated value of the component in the first direction calculated based on the component of the coordinate in the second direction.

[0011] (6): An image processing method according to another aspect of the present invention involves a computer setting an initial value for at least one parameter of a function that approximates the road boundary in an image representing the area in front of the moving body, captured by a camera mounted on the moving body, based on the identification value of the other parameter; sequentially updating the parameter at a predetermined time based on the parameter at the previous time and constraints set for the parameter; and performing driving control or driving assistance for the moving body based on the road boundary approximated by the function having the updated parameter.

[0012] (7) A program according to another aspect of the present invention causes a computer to set an initial value for at least one parameter of a function that approximates the road boundary in an image representing the area in front of the mobile body, captured by a camera mounted on the mobile body, based on the identification value of the other parameters, to sequentially update the parameter at a predetermined time based on the parameter at the previous time and constraints set for the parameter, and to perform driving control or driving assistance for the mobile body based on the road boundary approximated by the function having the updated parameter. [Effects of the Invention]

[0013] According to (1) to (7), it is possible to estimate road markings that can be reliably used for controlling the movement of a moving object. [Brief explanation of the drawing]

[0014] [Figure 1]This figure shows an example of the operating environment for the image processing device 100 installed in the vehicle M. [Figure 2] This figure shows an example of the configuration of the image processing device 100. [Figure 3] This figure shows an example of how the candidate point extraction unit 110 extracts candidate points for the road boundary. [Figure 4] This figure shows an example of a method for rearranging candidate points for the track boundary. [Figure 5] This figure illustrates the outline of the track boundary model updated by the model parameter update unit 120. [Figure 6] This figure shows an example of the flow of the model parameter update process executed by the model parameter update unit 120. [Figure 7] This graph illustrates how to calculate baseline values ​​for model parameters. [Figure 8] This is a sequence diagram showing an example of the processing flow performed by the image processing device 100. [Modes for carrying out the invention]

[0015] The following describes embodiments of the image processing apparatus, image processing method, and program of the present invention with reference to the drawings. In this embodiment, the image processing apparatus is, for example, a terminal device such as a smartphone having a camera and a display. However, the present invention is not limited to such a configuration, and the image processing apparatus may be at least a computer device that receives and processes an image captured by a camera and outputs the processing result to a display. In that case, the functions of the present invention are realized by the cooperation of the camera, the display, and the image processing apparatus.

[0016] [composition] Figure 1 shows an example of the operating environment for the image processing device 100 mounted on the vehicle M. The vehicle M is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its power source is an internal combustion engine such as a diesel engine or gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or power discharged from a secondary battery or fuel cell.

[0017] As shown in Figure 1, the image processing device 100 is installed on the vehicle M so that the camera 10 can capture images of the area in front of the vehicle M in the direction of travel. The image processing device 100 is held, for example, by an on-board holder (not shown) attached to the dashboard of the vehicle M, and captures images of the area in front of the vehicle M. The vehicle M is an example of a "moving object". In the following description, in this embodiment, an example is described in which the image processing device 100 is mounted on the vehicle M, which is a moving object, but more generally, a moving object includes devices having cameras that are mounted on a vehicle, such as a drive recorder or a smartphone.

[0018] Figure 2 shows an example of the configuration of the image processing device 100. As shown in Figure 2, the image processing device 100 includes, for example, a camera 10, a display unit 20, a candidate point extraction unit 110, a model parameter update unit 120, and an operation control unit 130. The candidate point extraction unit 110, the model parameter update unit 120, and the operation control unit 130 are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance on a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device equipped with a non-transient storage medium), or it may be stored on a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed when the storage medium is inserted into the drive device. Camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Display unit 20 is, for example, a display device such as a touch panel or a liquid crystal display.

[0019] [Extraction of candidate points] The candidate point extraction unit 110 extracts candidate points for the boundary of the road on which the vehicle M travels (road boundary) based on an image representing the area in front of the vehicle M captured by the camera 10. Figure 3 shows an example of how the candidate point extraction unit 110 extracts candidate points for the road boundary. As shown in Figure 3, when the candidate point extraction unit 110 acquires an image captured by the camera 10, it inputs the acquired image into a trained model (Deep Neural Network; DNN) that has been trained to output a probability value (0 or more and 1 or less) indicating whether or not each pixel (coordinate) of the image is a road boundary, according to the image input. The candidate point extraction unit 110 extracts pixels whose output probability value is a positive value as candidate points for the road boundary. Alternatively, the candidate point extraction unit 110 may extract pixels whose output probability value is greater than or equal to a threshold (e.g., 0.5) as candidate points for the road boundary.

[0020] [Updating model parameters] The model parameter update unit 120 sorts the candidate points of the track boundary extracted by the candidate point extraction unit 110 in descending order of probability value, and updates the model parameters of the track boundary model, described later, using the candidate points of the track boundary in descending order of probability value. Figure 4 shows an example of a method for sorting candidate points of the track boundary. In Figure 4, k represents a point in the recognition cycle for recognizing the track boundary, N(k) represents the number of candidate points of the track boundary acquired at time k, x'(n,k) represents the x-coordinate of each candidate point before sorting, y'(n,k) represents the y-coordinate of each candidate point before sorting, x(n,k) represents the x-coordinate of each candidate point after sorting, and y(n,k) represents the y-coordinate of each candidate point after sorting. As described later, by sequentially updating the model parameters of the track boundary model using the candidate points of the track boundary in descending order of probability value, a highly accurate track boundary model can be output even in a shorter time, even if downsampling occurs within a single recognition cycle.

[0021] Figure 5 is a diagram illustrating the outline of the track boundary model updated by the model parameter update unit 120. The model parameter update unit 120 sequentially substitutes candidate track boundary points, which have been sorted in descending order of probability, into the track boundary model defined by the following equation (1), thereby obtaining an estimated value of the track boundary y _hat We obtain (n,k).

[0022]

number

[0023] In equation (1), a i (n,k), b i (n,k), c i (n,k) represents the quadratic coefficient, linear coefficient, and constant term of a quadratic function (hereinafter sometimes referred to as the "element function") that approximates the path boundary in the image, and w i This represents a weight function that outputs weights between 0 and 1, depending on the x-coordinate of the input candidate point. More specifically, the weight function is defined by the following equations (2) to (4). In equations (2) to (4), x wi The values ​​(i=1~m) are fixed values ​​that are set in advance. i The sum of (x)(i=1~m) is set to always be 1.

[0024]

number

[0025]

number

[0026]

number

[0027] FIG. 5 shows the element functions f1, f2, f3 and the weight functions w1, w2, w3 when m = 3 in the formulas (1) to (4). That is, the running boundary model in the present embodiment divides an image into a plurality of regions based on the x coordinate, approximates the running boundary for each divided region by a quadratic function, and synthesizes the quadratic functions for each region to approximate the running boundary in the entire image.

[0028] Thus, compared with the conventional method (batch operation method least squares method) of approximating candidate points of the running boundary in an image with a single quadratic function, in the present embodiment, the running boundary is approximated by a quadratic function for each partial region of the image, and the final approximation curve is obtained by synthesizing the approximated quadratic functions, so that the running boundary in the image can be expressed with higher accuracy.

[0029] In the present embodiment, the value and number of the x coordinates x wi (i = 1 to m) for dividing the region are set as fixed values set in advance. However, the present invention is not limited to such a configuration, and the value and number of the x coordinates x wi may be set as the number of clusters obtained by clustering the extracted candidate points and their boundary points, for example.

[0030] Next, referring to FIG. 6, the update process of the model parameters executed by the model parameter update unit 120 will be described. FIG. 6 is a diagram showing an example of the flow of the update process of the model parameters executed by the model parameter update unit 120.

[0031] First, the model parameter update unit 120 rearranges the candidate points (x'(n, k), y'(n, k)) of the running boundary extracted by the candidate point extraction unit 110 in descending order of probability value to obtain candidate points (x(n, k), y(n, k)). The model parameter update unit 120 generates a vector ξ(n, k) = [x(n, k) 2 , x(n, k), 1] from the candidate point (x(n, k), and substitutes x(n, k) into the weight function w i to obtain a weight value w iWe obtain (x(n,k)).

[0032] Next, the model parameter update unit 120 updates the vector ξ(n,k) and the model parameter θ. i (n,k)=[a i (n,k),b i (n,k),c i Calculate the inner product between (n,k) and the weight value w i By multiplying by (x(n,k)), the element function f for each region is obtained. i (x=w i (x(n,k))(a i (n,k)x(n,k) 2 +b i (n,k)x(n,k)+c i The (n,k)) is obtained. The model parameter update unit 120 performs the element function f for each region. i By taking the sum of (x), the output estimate y is expressed by equation (1). _hat We obtain (n,k). Note that the model parameter θ is multiplied at this time. i (n,k)=[a i (n,k),b i (n,k),c i The initial values ​​for (n,k) will be discussed later.

[0033] Next, the model parameter update unit 120 updates the output estimated value y _hat The identification error between (n,k) and the y-coordinate y(n,k) of the candidate point is given by e id (n,k=w i (x(n,k))(y(n,k)-y _hat The output is calculated as (n,k). _hat The error between (n,k) and the y-coordinate y(n,k) of the candidate point is weighted w i By multiplying by (x(n,k)), the identification error can be reflected for each region. The model parameter update unit 120 calculates the identification error e id The model parameter θ decreases in the direction of reducing the squared error of (n,k). i Adaptive gain K to modify (n,k) p We define it by the following equation (5).

[0034]

number

[0035] In equation (5), P'(n,k) represents a 3x3 covariance matrix, which is defined by the following equation (6).

[0036]

number

[0037] In equation (6), I represents a 3x3 identity matrix, and λ1 and λ2 represent the setting parameters of the successive identification algorithm. λ1 and λ2 are constant values ​​greater than 0 and less than or equal to 1. When applying the least squares method, λ1=1 and λ2=1 are set; when applying the weighted least squares method, λ1=λ (0<λ≦1) and λ2=1 are set; and when applying the fixed-gain method, λ1=1 and λ2=0 are set. When applying the fixed-gain method, the adaptive gain K p This is expressed by the following equations (7) and (8).

[0038]

number

[0039]

number

[0040] In equation (8), P represents the identification gain matrix. P1, P2, and P3 represent the identification gains and are positive fixed values. The model parameter update unit 120 calculates the identification error e id For (n,k), the adaptive gain K p By multiplying by the following equation (9), the model parameter correction amount ddθ is obtained. i We obtain (n,k).

[0041]

number

[0042] Next, the model parameter update unit 120 updates the final correction amount, which will be described later, to the previous value dθ. i For (n-1,k), the forgetting gain Δ fgt Multiply by this, and the resulting multiplied value is the model parameter correction amount ddθ in equation (9). i By adding to (n,k), the model parameter correction amount dθ is expressed by the following equation (10). raw_i We obtain (n,k). In this way, we obtain the final correction amount dθ from the previous attempt. i (n-1,k) is the forgetting gain Δ fgt The correction amount dθ is calculated using the value obtained by multiplying by the result. raw_i By defining (n,k), we can suppress rapid changes in the track boundary model.

[0043]

number

[0044] In equation (10), the forgetting gain Δ fgt This is a 3x3 diagonal matrix represented by the following equation (11). In equation (11), δ fgt_1 , δ fgt_2、 δ fgt_3 0 < δ fgt_1 , δ fgt_2 <1, δ fgt_3 It is a constant value that satisfies =1. That is, the forgetting gain Δ fgt is, a i (n,k) and b i This is configured to apply the forgetting effect to (n,k).

[0045]

number

[0046] Next, the model parameter update unit 120 adjusts the model parameter correction amount dθ raw_i By applying a limiter process (an example of a "constraint") represented by equations (12) to (14) below to (n,k), the amount of modification to the model parameter is corrected, and the final modification amount dθ is represented by equation (15) below. i We obtain (n,k). In equations (12) to (14), da L da H db L db H dc L dc H This is a pre-set fixed value that is configured to prevent the track boundary model from having an unrealistic shape.

[0047]

number

[0048]

number

[0049]

number

[0050]

number

[0051] Next, the model parameter update unit 120 adjusts the correction amount dθ i (n,k) is the reference value θ of the model parameter, which is expressed by the following equation (16). base_i By adding (n,k), the model parameter value θ is expressed by the following equation (17). i We obtain (n,k). The calculated model parameter value θ i (n,k) is the model parameter θ for the next input value n+1.i It is used as an identification value (initial value) for calculating (n+1,k).

[0052]

number

[0053]

number

[0054] [Calculation of baseline values ​​for model parameters] Next, referring to Figure 7, the reference value θ of the model parameter base_i The method for calculating (n,k) will be explained. Figure 7 is a graph illustrating the method for calculating the reference value of the model parameter. The model parameter update unit 120 calculates the reference value θ of the model parameter. base_i (n,k) is set by the following equations (18) to (20).

[0055]

number

[0056]

number

[0057]

number

[0058] Equations (18) and (19) represent the model parameter a, respectively. base_i (n,k), c base_i This represents the initial value of (n,k). As shown in equations (18) and (19), the curvature of the track boundary model can be in both left and right directions, so the model parameter a base_iThe initial value of (n,k) may be zero, and the model parameter c corresponds to the y-intercept of the track boundary model. i Since it can be in both left and right directions, the model parameter c base_i The initial value of (n,k) can be zero.

[0059] In equation (20), c i (n-1,k) is the model parameter c calculated in the previous calculation. i The function g represents the identification value of the model parameter c. i This represents the scaling function that gives a line passing through the identification value (i.e., the y-intercept) and the vanishing point VP of the image. That is, as shown in the left part of Figure 7, the identification value c i The larger the value of (n-1,k), the higher the baseline value of the model parameter b. base_i (n,k) takes on a smaller value, and as shown in the right part of Figure 7, the line slopes to the left. Using the method described above, the reference value θ of the model parameter base_i By setting (n,k), it is possible to prevent the track boundary model from becoming an unrealistic shape, even if the number of DNN output values ​​N(k) in a given recognition cycle is significantly small due to events such as bad weather or dark visibility.

[0060] When the model parameter update unit 120 determines the track boundary model for each recognition cycle, the driving control unit 130 performs automatic driving or driving assistance for its own vehicle M based on the determined track boundary model. More specifically, for example, the driving control unit 130 transforms the track boundary model in the camera coordinate system into a bird's-eye view coordinate system to obtain a track boundary model in the bird's-eye view coordinate system. Using the track boundary model in the bird's-eye view coordinate system, the driving control unit 130 generates a target trajectory and action plan for its own vehicle M, and drives its own vehicle M according to the generated target trajectory and action plan. Also, for example, the driving control unit 130 uses the track boundary model in the bird's-eye view coordinate system to assist the steering or issue warnings when the occupant of its own vehicle M is manually driving, so as not to deviate from the determined track boundary model.

[0061] FIG. 8 is a sequence diagram showing an example of the processing flow executed by the image processing apparatus 100. As shown in FIG. 8, at time k-2, the image processing apparatus 100 rearranges the output values x’(1,k-2), y’(1,k-2), x’(2,k-2), y’(2,k-2), ···, x’(N(k-2),k-2)), y’(N(k-2),N(k-2)) of the DNN in descending order of probability values to obtain x(1,k-2), y(1,k-2), x(2,k-2), y(2,k-2), ···, x(N(k-2),k-2), y(N(k-2),k-2). The image processing apparatus 100 constructs a vector ξ(n,k-2)=[x(n,k-2) 2 ,x(n,k-2),1] and inputs it into the sequential identification algorithm shown in FIG. 6 to sequentially update the model parameter value θ i (n,k-2).

[0062] During the update of the model parameter value θ i (n,k-2), when the resampling timing T ds has elapsed, the image processing apparatus 100 downsamples the model parameter value (for example, θ ds (N(k-2)-1,k-2)) at the time when the resampling timing T i has elapsed and determines it as the final model parameter value in the recognition cycle k-2. The image processing apparatus 100 causes the display unit 20 to display the gait boundary model in which the determined model parameter value θ i (N(k-2)-1,k-2) is set. Thus, unlike the batch operation method of the least squares method, in the present embodiment, by sequentially updating the model parameter value θ i (n,k) in order from the output values with high probability values, even when the amount of data is large and the calculation cannot be completed by the batch operation method of the least squares method, a highly accurate gait boundary model can be estimated.

[0063] When the recognition cycle reaches k-1, the image processing apparatus 100 rearranges the output values x’(1,k-1), y’(1,k-1), x’(2,k-1), y’(2,k-1), ···, x’(N(k-1),k-1), y’(N(k-1),k-1) of the DNN in descending order of probability values to obtain x(1,k-1), y(1,k-1), x(2,k-1), y(2,k-1), ···, x(N(k-1),k-1), y(n(k-1),k-1). The image processing apparatus 100 constructs a vector ξ(n,k-1)=[x(n,k-1) 2 , x(n,k-1), 1] and inputs it into the sequential identification algorithm shown in FIG. 6 to update the model parameter value θ i (n,k-1). At this time, the image processing apparatus 100 uses the model parameter value θ i (N(k-2)-1,k-2)) determined in the recognition cycle k-2 as an initial value for calculating the estimated value y _hat (1,k-1). When the update of the model parameter value θ i (n,k-1) is completed up to the output values of x(N(k-1),k-1), y(N(k-1),k-1), the image processing apparatus 100 holds the model parameter value θ i (N(k-1),k-1) until the next recognition cycle, the recognition cycle k, and causes the display unit 20 to display the gait boundary model in which the held model parameter value θ i (N(k-1),k-1) is set. Thereafter, when the recognition cycle reaches k, similarly, the image processing apparatus 100 uses the output values x’(1,k), y’(1,k), x’(2,k), y’(2,k), ···, x’(N(k),k), y’(N(k),k) of the DNN to update the model parameter value θ i (n,k). At this time, the image processing apparatus 100 uses the model parameter value θ i (N(k-1),k-1) determined in the recognition cycle k-1 as an initial value for calculating the estimated value y _hat (1,k).

[0064] As described above, according to this embodiment, the image processing device divides the forward region on a coordinate system based on the vehicle at predetermined intervals, generates a function that approximates the road markings in each region based on the probability value indicating the probability of the existence of road markings for each coordinate within the region obtained by the division and the said coordinate, and generates a function that approximates the road markings in the forward region by combining the functions generated for each region. This makes it possible to estimate road markings that can be reliably used for driving control of a moving object.

[0065] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes the program stored in the memory device, The initial value of at least one parameter of a quadratic function that approximates the road boundary in an image representing the area in front of the moving object, captured by a camera mounted on the moving object, is set based on the identification value of the other parameters. The parameters at a predetermined time are sequentially updated based on the parameters at the previous time point and the constraints set for those parameters. Based on the road boundary approximated by the function having the updated parameters, the driving control or driving assistance of the moving body is performed. An image processing device configured in such a way.

[0066] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0067] 10 Cameras 20 Display section 100 Image Processing Devices 110 Candidate point extraction part 120 Model parameter update section 130 Operation Control Unit

Claims

1. A setting unit sets the initial value of at least one parameter of a function that approximates the road boundary in an image representing the area in front of the moving object, captured by a camera mounted on the moving object, based on the identification values ​​of other parameters of the same function. An update unit that sequentially updates the parameter at a predetermined time based on the parameter at the previous time and the constraint conditions set for the parameter, A control unit that performs driving control or driving assistance for the moving body based on the road boundary approximated by the function having the updated parameters, Image processing device.

2. A setting unit that sets the initial value of at least one parameter of a function that approximates the road boundary in an image representing the area in front of the moving body, captured by a camera mounted on the moving body, based on the identification value of the other parameter, An update unit that sequentially updates the parameter at a predetermined time based on the parameter at the previous time and the constraint conditions set for the parameter, A control unit that performs driving control or driving assistance for the mobile body based on the road boundary approximated by the function having the updated parameters, The aforementioned function is a quadratic function, The setting unit sets the initial value of the linear coefficient of the quadratic function based on the identification value of the road boundary recognized in the lateral direction of the image. The update unit updates the parameters at a predetermined time by setting the forgetting gains of the quadratic coefficient and the linear coefficient of the parameters at the previous time to a value less than 1. Image processing device.

3. The update unit divides the image into predetermined intervals, sequentially updates the parameters of the function that approximates the road boundary for each divided region, and then synthesizes them to sequentially update the parameters of the function that approximates the road boundary. The image processing apparatus according to claim 1 or 2.

4. The update unit obtains the probability value, which indicates the probability of the existence of a road boundary for each coordinate in the image, using a trained model that has been trained to output a probability value indicating the probability of the existence of a road boundary for each coordinate in the image, and updates the parameters based on the probability value and the coordinate. The image processing apparatus according to any one of claims 1 to 3.

5. The update unit sequentially updates the parameters to minimize the error between the component of the coordinate in the first direction and the estimated value of the component in the first direction calculated based on the component of the coordinate in the second direction. The image processing apparatus according to claim 4.

6. Computers The initial value of at least one parameter of a function that approximates the road boundary in an image representing the area in front of the moving body, captured by a camera mounted on the moving body, is set based on the identification values ​​of the other parameters of the same function. The parameters at a predetermined time are sequentially updated based on the parameters at the previous time point and the constraints set for those parameters. Based on the road boundary approximated by the function having the updated parameters, the driving control or driving assistance of the moving body is performed. Image processing methods.

7. On the computer, The initial value of at least one parameter of a function that approximates the road boundary in an image representing the area in front of the moving body, captured by a camera mounted on the moving body, is set based on the identification values ​​of the other parameters of the same function. The parameters at a predetermined time are sequentially updated based on the parameters at the previous time point and the constraints set for those parameters. Based on the road boundary approximated by the function having the updated parameters, the driving control or driving assistance of the moving body is performed. program.

Images