Image processing device, image processing method, and program

Abstract

To provide an image processing device, image processing method and program for estimating road demarcation lines stably usable for travel control of mobile objects.SOLUTION: An image processing device 100 to be mounted on a self vehicle M includes: a setting part for setting an initial value of at least one parameter out of parameters of a function to approximate a road boundary in an image captured by a mounted camera and representing a forward area, based on an identification value of another parameter out of the parameters; a model parameter updating part for sequentially updating a parameter at a prescribed time instant based on a parameter of the prescribed time instant at the previous time instant and a constraint condition set to the parameter at the previous time instant; and a drive control part for executing drive control or drive support of a mobile object based on the road boundary approximated by a function having the updated parameter.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, there is known a technique for estimating road dividing lines on a road on which a vehicle is traveling and controlling the traveling of the vehicle based on the estimated road dividing lines. For example, Patent Document 1 discloses a technique for selecting multiple three-dimensional objects from an image captured by a camera mounted on the vehicle, estimating road dividing lines based on the positions of the selected three-dimensional objects, and setting a target speed of the vehicle according to the curvature of the estimated road dividing lines. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-60885 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 estimates road-dividing lines by fitting curves to multiple positions extracted from captured images of the road on which a vehicle is traveling. However, this method can result in large errors in the estimated road-dividing lines due to limitations on the degree of freedom of the curves to be fitted. In addition, the estimated road-dividing lines can become blurred due to fluctuations in the positions extracted from the images at each point in time. As a result, the estimated road-dividing lines cannot always be reliably used for vehicle driving control.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide an image processing device, an image processing method, and a program that can estimate road dividing lines that can be stably used for driving control of a moving body. [Means for solving the problem]

[0006] The image processing device, image processing method, and program according to the present invention employ the following configuration. (1): An image processing device according to one embodiment of the present invention includes an acquisition unit that acquires, in response to an input of an image showing the area ahead of a moving body captured by a camera mounted on the moving body, a probability value indicating the probability of the existence of a road boundary for each coordinate of the image using a trained model that outputs the probability value and the corresponding coordinates; a sorting unit that sorts the multiple coordinates in descending order of the probability value; an update unit that uses the multiple coordinates in the sorted order to sequentially update parameters of a function that approximates the road boundary in the image; and a control unit that performs driving control or driving assistance of the moving body based on the road boundary approximated by the function defined by the updated parameters.

[0007] (2) In the aspect (1) above, the acquisition unit acquires coordinates of the image where the probability value is equal to or greater than a threshold value as candidate points for the road boundary.

[0008] (3): In the above aspect (1) or (2), the update unit divides the image at predetermined intervals, sequentially updates the parameters of the element function that approximates the road boundary for each divided area, and synthesizes the parameters, thereby sequentially updating the parameters of the function that approximates the road boundary.

[0009] (4): In any of the above aspects (1) to (3), the update unit sequentially updates the parameters so as to minimize an error between a first direction component of the coordinate and an estimated value of the first direction component calculated based on a second direction component of the coordinate.

[0010] (5): Another aspect of the image processing method of the present invention involves a computer receiving an image of a region ahead of a moving body captured by a camera mounted on the moving body, using a trained model that outputs a probability value indicating the probability of the existence of a road boundary for each coordinate in the image, acquiring the probability value and the corresponding coordinate, sorting the coordinates in descending order of probability value, sequentially updating parameters of a function that approximates the road boundary in the image using the coordinates in the sorted order, and performing driving control or driving assistance for the moving body based on the road boundary approximated by the function defined by the updated parameters.

[0011] (6): Another aspect of the present invention provides a program that causes a computer to, in response to an input of an image showing the area ahead of a moving body captured by a camera mounted on the moving body, acquire the probability value and the corresponding coordinates using a trained model that outputs a probability value indicating the probability of the existence of a road boundary for each coordinate of the image, sort the coordinates in descending order of the probability value, use the coordinates in the sorted order to sequentially update parameters of a function that approximates the road boundary in the image, and perform driving control or driving assistance of the moving body based on the road boundary approximated by the function defined by the updated parameters. [Effects of the Invention]

[0012] According to (1) to (6), it is possible to estimate road dividing lines that can be stably used for driving control of a moving object. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing an example of a usage environment of an image processing device 100 mounted on a vehicle M. FIG. [Figure 2] 1 is a diagram illustrating an example of a configuration of an image processing device 100. FIG. [Figure 3] FIG. 10 is a diagram showing an example of a method in which the candidate point extraction unit 110 extracts candidate points for lane boundaries. [Figure 4] FIG. 10 is a diagram illustrating an example of a method for rearranging candidate points for lane boundaries. [Figure 5] 1 is a diagram for explaining an outline of a lane boundary model updated by a model parameter update unit 120. FIG. [Figure 6] 10 is a diagram showing an example of the flow of a model parameter update process executed by a model parameter update unit 120. FIG. [Figure 7] 10 is a graph illustrating a method for calculating a reference value of a model parameter. [Figure 8] 2 is a sequence diagram showing an example of the flow of processing executed by the image processing device 100. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, with reference to the drawings, embodiments of an image processing device, an image processing method, and a program of the present invention will be described. In this embodiment, the image processing device is, for example, a terminal device such as a smartphone having a camera and a display. However, the present invention is not limited to such a configuration, and the image processing device may be at least a computer device that receives images captured by the camera, processes them, and outputs the processed results to a display. In this case, the camera, the display, and the image processing device cooperate to realize the functions of the present invention.

[0015] [composition] 1 is a diagram showing an example of a usage environment of an image processing device 100 mounted on a vehicle M. The vehicle M may be, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

[0016] As shown in FIG. 1 , the image processing device 100 is installed in the host vehicle M so that the camera 10 can capture an image of the area ahead of the host vehicle M in the traveling direction of the host vehicle M. The image processing device 100 is held, for example, by an in-vehicle holder (not shown) attached to the dashboard of the host vehicle M, and captures an image of the area ahead of the host vehicle M. The host vehicle M is an example of a "mobile body." In the following description, in this embodiment, an example will be described in which the image processing device 100 is mounted on the host vehicle M, which is a mobile body, but more generally, a mobile body includes a device having a camera mounted on a vehicle, such as a drive recorder or a smartphone.

[0017] 2 is a diagram illustrating an example of the configuration of an image processing device 100. As illustrated in FIG. 2, the image processing device 100 includes, for example, a camera 10, a display unit 20, a candidate point extraction unit 110, a model parameter update unit 120, and an operation control unit 130. The candidate point extraction unit 110, the model parameter update unit 120, and the operation control unit 130 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. Camera 10 is, for example, a digital camera that uses a solid-state imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). Display unit 20 is, for example, a display device such as a touch panel or a liquid crystal display.

[0018] [Extract candidate points] The candidate point extraction unit 110 extracts candidate points for the boundary (lane boundary) of the lane along which the vehicle M is traveling, based on an image showing the area ahead of the vehicle M captured by the camera 10. FIG. 3 is a diagram illustrating an example of a method by which the candidate point extraction unit 110 extracts candidate points for lane boundaries. As shown in FIG. 3, when the candidate point extraction unit 110 acquires an image captured by the camera 10, the candidate point extraction unit 110 inputs the acquired image to a trained model (Deep Neural Network; DNN) that is trained to output a probability value (0 to 1) indicating whether or not each pixel (coordinate) of the image is a lane boundary in response to the image input. The candidate point extraction unit 110 extracts pixels for which the output probability value is a positive value as candidate points for lane boundaries. Alternatively, the candidate point extraction unit 110 may extract pixels for which the output probability value is equal to or greater than a threshold value (e.g., 0.5) as candidate points for lane boundaries.

[0019] [Update model parameters] The model parameter update unit 120 sorts the lane boundary candidate points extracted by the candidate point extraction unit 110 in descending order of probability value and updates the model parameters of the lane boundary model, described below, using the lane boundary candidate points with the highest probability value. FIG. 4 illustrates an example of a method for sorting lane boundary candidate points. In FIG. 4, k represents a point in a recognition cycle for recognizing lane boundaries, N(k) represents the number of lane boundary candidate points acquired at point k, x'(n,k) represents the x-coordinate of each candidate point before sorting, y'(n,k) represents the y-coordinate of each candidate point before sorting, x(n,k) represents the x-coordinate of each candidate point after sorting, and y(n,k) represents the y-coordinate of each candidate point after sorting. As described below, by sequentially updating the model parameters of the lane boundary model using the lane boundary candidate points with the highest probability value, a highly accurate lane boundary model can be output even if downsampling occurs within a single recognition cycle, even in a shorter time.

[0020] 5 is a diagram for explaining an outline of the lane boundary model updated by the model parameter update unit 120. The model parameter update unit 120 sequentially substitutes the lane boundary candidate points, sorted in descending order of probability value, into the lane boundary model defined by the following equation (1), to calculate the estimated value y _hat Obtain (n,k).

[0021]

number

[0022] In formula (1), a i (n,k), b i (n,k), c i (n, k) represent the quadratic coefficient, linear coefficient, and constant term of the quadratic function (hereinafter sometimes referred to as the "element function") that approximates the lane boundary in the image, respectively, and w i represents a weighting function that outputs a weight between 0 and 1 according to the x coordinate of the input candidate point. More specifically, the weighting function is defined by the following equations (2) to (4). In equations (2) to (4), x wi The values ​​of (i=1 to m) are fixed values ​​set in advance. i The sum of (x) (i=1 to m) is set to always be 1.

[0023]

number

[0024]

number

[0025]

number

[0026] 5 shows the element functions f1, f2, f3 and weighting functions w1, w2, w3 in equations (1) to (4) when m = 3. That is, the lane boundary model in this embodiment divides the image into multiple regions based on the x coordinate, approximates the lane boundary for each divided region with a quadratic function, and combines the quadratic functions for each region to approximate the lane boundary for the entire image.

[0027] In this way, compared to the conventional method (batch calculation least squares method) of approximating candidate points for lane boundaries in an image with a single quadratic function, in this embodiment, the lane boundaries are approximated with quadratic functions for each partial region of the image, and the approximated quadratic functions are combined to obtain the final approximate curve, thereby enabling the lane boundaries in the image to be represented with greater accuracy.

[0028] In this embodiment, the x-coordinate x wi The values ​​and numbers of (i=1 to m) are fixed values ​​set in advance. However, the present invention is not limited to such a configuration. wi The value and number of may be set as, for example, the number of clusters obtained by clustering the extracted candidate points and their boundary points.

[0029] Next, the model parameter update processing executed by the model parameter update unit 120 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the flow of the model parameter update processing executed by the model parameter update unit 120.

[0030] First, the model parameter update unit 120 sorts the candidate points (x'(n,k), y'(n,k)) of the lane boundary extracted by the candidate point extraction unit 110 in descending order of probability value to obtain candidate points (x(n,k), y(n,k)). The model parameter update unit 120 calculates the vector ξ(n,k)=[x(n,k) 2 ,x(n,k),1] and weight x(n,k) by the weight function w i Substituting into the weight value w iObtain (x(n,k)).

[0031] Next, the model parameter update unit 120 calculates the vector ξ(n,k) and the model parameters θ i (n,k)=[a i (n,k),b i (n,k),c i (n,k)] and calculate the weight value w i By multiplying (x(n,k)), the element function f i (x)=w i (x(n,k))(a i (n,k) x (n,k) 2 +b i (n,k)x(n,k)+c i (n, k)) is obtained by the model parameter update unit 120. i (x) to obtain the output estimate y _hat (n, k) is obtained. In this case, the model parameter θ i (n,k)=[a i (n,k),b i (n,k),c i The initial value of (n, k)] will be described later.

[0032] Next, the model parameter update unit 120 updates the output estimate y _hat The identification error between (n, k) and the y-coordinate y(n, k) of the candidate point is e id (n,k)=w i (x(n,k))(y(n,k)-y _hat (n,k)) and the output estimate y _hat The error between (n,k) and the y-coordinate y(n,k) of the candidate point is weighted by w i By multiplying (x(n,k)), the identification error can be reflected for each region. id The model parameter θ in the direction that reduces the square error of (n,k) i Adaptive gain K that modifies (n,k) p is defined by the following equation (5).

[0033]

number

[0034] In equation (5), P'(n, k) represents a covariance matrix with three rows and three columns, and is a matrix defined by the following equation (6).

[0035]

number

[0036] In equation (6), I represents a 3-by-3 identity matrix, and λ1 and λ2 represent the setting parameters of the recursive identification algorithm. λ1 and λ2 are constant values ​​greater than 0 and less than or equal to 1. When applying the least squares method, λ1=1 and λ2=1 are set; when applying the weighted least squares method, λ1=λ(0<λ≦1) and λ2=1 are set; and when applying the fixed gain method, λ1=1 and λ2=0 are set. When applying the fixed gain method, the adaptive gain K p is expressed by the following equations (7) and (8).

[0037]

number

[0038]

number

[0039] In equation (8), P represents the identification gain matrix. P1, P2, and P3 represent identification gains, which are positive fixed values. The model parameter update unit 120 updates the identification error e id For (n,k), adaptive gain K p By multiplying by , the correction amount ddθ of the model parameter expressed by the following equation (9) is obtained. i Obtain (n,k).

[0040]

number

[0041] Next, the model parameter update unit 120 updates the previous value dθ of the final correction amount, which will be described later. i For (n-1,k), the forgetting gain Δ fgt The multiplied value is used as the correction amount ddθ of the model parameter in equation (9). i By adding it to (n, k), the correction amount dθ of the model parameter expressed by the following equation (10) is obtained. raw_i (n, k) is obtained. In this way, the final correction amount dθ obtained last time is i (n-1,k) is the forgetting gain Δ fgt The correction amount dθ is calculated by multiplying raw_i By defining (n,k), it is possible to suppress sudden fluctuations in the track boundary model.

[0042]

number

[0043] In equation (10), the forgetting gain Δ fgt is a diagonal matrix with three rows and three columns expressed by the following equation (11). In equation (11), δ fgt_1 , δ fgt_2、 δ fgt_3 is 0<δ fgt_1 , δ fgt_2 <1, δ fgt_3 = 1. That is, the forgetting gain Δ fgt is a i (n,k) and b i It is set to apply the forgetting effect to (n, k).

[0044]

number

[0045] Next, the model parameter update unit 120 updates the model parameter by a correction amount dθ raw_i By applying limiter processing (an example of a "constraint condition") expressed by the following equations (12) to (14) to (n, k), the modification amount of the model parameters is corrected, and the final modification amount dθ expressed by the following equation (15) is obtained. i (n, k) is obtained. In equations (12) to (14), da L , da H , db L , db H , d.c. L , d.c. H is a fixed value that is set in advance to prevent the lane boundary model from taking on an unrealistic shape.

[0046]

number

[0047]

number

[0048]

number

[0049]

number

[0050] Next, the model parameter update unit 120 calculates the correction amount dθ i (n, k) is given the reference value θ of the model parameter expressed by the following equation (16): base_i By adding (n, k), the current model parameter value θ i (n, k) is obtained. The calculated model parameter value θ i (n,k) is the model parameter θ for the next input value n+1i It is used as an identification value (initial value) for calculating (n+1, k).

[0051]

number

[0052]

number

[0053] [Calculation of reference values ​​for model parameters] Next, referring to FIG. 7, the reference value θ of the model parameter base_i A method for calculating (n, k) will be described. FIG. 7 is a graph for explaining a method for calculating the reference values ​​of the model parameters. The model parameter update unit 120 calculates the reference values ​​θ of the model parameters. base_i (n, k) is set by the following equations (18) to (20).

[0054]

number

[0055]

number

[0056]

number

[0057] Equations (18) and (19) respectively represent the model parameters a base_i (n,k), c base_i (n, k) represents the initial value. As shown in Equation (18) and Equation (19), the curvature of the lane boundary model can be in both the left and right directions. Therefore, the model parameter a base_iThe initial value of (n, k) can be zero, and the model parameter c corresponding to the y-intercept of the road boundary model i Since can be in both the left and right directions, the model parameter c base_i The initial values ​​of (n, k) may be zero.

[0058] In equation (20), c i (n-1,k) is the previously calculated model parameter c i The function g represents the identified value of the model parameter c i This represents a scaling function that gives a straight line passing through the identified value (i.e., y-intercept) of the image and the vanishing point VP of the image. That is, as shown in the left part of Figure 7, the identified value c i The larger the value of (n-1,k), the lower the reference value of the model parameter b base_i (n, k) takes a smaller value, and the line leans to the left, as shown in the right part of Figure 7. Using the method described above, the reference value θ base_i By setting (n, k), it is possible to prevent the lane boundary model from becoming unrealistic in shape, even if the number of DNN output values ​​N(k) in a recognition cycle is significantly small, for example, due to events such as bad weather or low visibility.

[0059] When the model parameter update unit 120 determines the lane boundary model for each recognition cycle, the driving control unit 130 performs automatic driving or driving assistance for the host vehicle M based on the determined lane boundary model. More specifically, for example, the driving control unit 130 performs bird's-eye view conversion of the lane boundary model in the camera coordinate system to obtain a lane boundary model in the bird's-eye view coordinate system. The driving control unit 130 uses the lane boundary model in the bird's-eye view coordinate system to generate a target trajectory and action plan for the host vehicle M, and causes the host vehicle M to travel in accordance with the generated target trajectory and action plan. Furthermore, for example, the driving control unit 130 uses the lane boundary model in the bird's-eye view coordinate system to provide steering assistance or issue a warning when an occupant of the host vehicle M manually drives the vehicle M so as not to deviate from the determined lane boundary model.

[0060] 8 is a sequence diagram showing an example of the flow of processing executed by the image processing device 100. As shown in FIG. 8, at time point k-2, the image processing device 100 rearranges the output values ​​x'(1,k-2), y'(1,k-2), x'(2,k-2), y'(2,k-2), . . . , x'(N(k-2),k-2)), y'(N(k-2),N(k-2)) of the DNN in descending order of probability value to obtain x(1,k-2), y(1,k-2), x(2,k-2), y(2,k-2), . . . , x(N(k-2),k-2), y(N(k-2),k-2). The image processing device 100 rearranges the vector ξ(n,k-2)=[x(n,k-2) 2 ,x(n,k-2),1] is constructed and input to the recursive identification algorithm shown in Figure 6 to obtain the model parameter values ​​θ i (n,k-2) is updated sequentially.

[0061] Model parameter value θ i During the update of (n,k-2), the resampling timing T ds When the time has elapsed, the image processing device 100 ds The model parameter values ​​(e.g., θ i (N(k-2)-1, k-2)) is downsampled and determined as the final model parameter value in the recognition cycle k-2. i The lane boundary model in which (N(k-2)-1, k-2) is set is displayed on the display unit 20. In this way, unlike the least squares method of the batch calculation type, in this embodiment, the model parameter value θ is calculated by using the output values ​​in descending order of probability value. i By sequentially updating (n, k), a highly accurate lane boundary model can be estimated even when the amount of data is large and the calculation cannot be completed using the least squares method with a batch calculation method.

[0062] When the recognition cycle k-1 arrives, the image processing device 100 rearranges the DNN output values ​​x'(1,k-1), y'(1,k-1), x'(2,k-1), y'(2,k-1), . . . , x'(N(k-1),k-1), y'(N(k-1),k-1) in descending order of probability value to obtain x(1,k-1), y(1,k-1), x(2,k-1), y(2,k-1), . . . , x(N(k-1),k-1), y(n(k-1),k-1). The image processing device 100 rearranges the vector ξ(n,k-1)=[x(n,k-1) 2 ,x(n,k-1),1] is constructed and input to the recursive identification algorithm shown in Figure 6 to obtain the model parameter values ​​θ i (n, k-1). At this time, the image processing apparatus 100 updates the model parameter value θ i (N(k-2)-1,k-2)) is the estimated value y _hat (1,k-1) is used as the initial value for calculating the model parameter value θ i When the update of (n, k−1) is completed, the image processing device 100 updates the model parameter value θ i (N(k-1),k-1) and the retained model parameter values ​​θ i The lane boundary model with (N(k-1), k-1) set is displayed on the display unit 20. Thereafter, when the recognition cycle k begins, the image processing device 100 similarly calculates the model parameter values ​​θ using the output values ​​x'(1, k), y'(1, k), x'(2, k), y'(2, k), . . . , x'(N(k), k), y'(N(k), k). i (n, k). At this time, the image processing apparatus 100 updates the model parameter value θ i (N(k-1),k-1) is the estimated value y _hat Used as the initial value for calculating (1,k).

[0063] According to the present embodiment described above, the image processing device divides the area ahead of the vehicle in a coordinate system based on the vehicle at predetermined intervals, generates a function that approximates the road-dividing lines in each area based on the coordinates and a probability value indicating the probability of the existence of a road-dividing line for each coordinate within each area obtained by dividing the area, and combines the functions generated for each area to generate a function that approximates the road-dividing lines in the area ahead. This makes it possible to estimate road-dividing lines that can be reliably used for driving control of a moving object.

[0064] The above-described embodiment can be expressed as follows. a storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device, In response to an input of an image showing a region ahead of a moving body captured by a camera mounted on the moving body, a trained model is used to output a probability value indicating the probability of a road boundary being present for each coordinate of the image, and the probability value and the corresponding coordinate are acquired; sorting the plurality of coordinates in descending order of the probability value; using the plurality of coordinates in the sorted order to sequentially update parameters of a function that approximates the road boundary in the image; performing driving control or driving assistance for the moving object based on the road boundary approximated by the function defined by the updated parameters; The image processing device is configured as follows.

[0065] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0066] 10 Camera 20 Display section 100 Image processing device 110 Candidate point extraction part 120 Model parameter update unit 130 Operation control unit

Claims

1. an acquisition unit that acquires, in response to an input of an image showing a region ahead of the moving body captured by a camera mounted on the moving body, the probability value indicating the existence probability of a road boundary for each coordinate of the image using a trained model that outputs the probability value and the corresponding coordinate; a sorting unit that sorts the plurality of coordinates in descending order of the probability value; an update unit that uses the plurality of coordinates in the sorted order to sequentially update parameters of a function that approximates the road boundary in the image; a control unit that performs driving control or driving assistance of the moving object based on the road boundary that is approximated by the function defined by the updated parameters, Image processing device.

2. The acquisition unit acquires coordinates of the image where the probability value is equal to or greater than a threshold as candidate points for the road boundary. The image processing device according to claim 1 .

3. the updating unit divides the image at predetermined intervals, sequentially updates parameters of element functions that approximate road boundaries for each divided area, and synthesizes the parameters, thereby sequentially updating parameters of the functions that approximate road boundaries.

3. The image processing device according to claim 1 or 2.

4. the updating unit sequentially updates the parameters so as to minimize an error between a first direction component of the coordinate and an estimated value of the first direction component calculated based on a second direction component of the coordinate. The image processing device according to claim 1 .

5. The computer In response to an input of an image showing a region ahead of a moving body captured by a camera mounted on the moving body, a trained model is used to output a probability value indicating the probability of a road boundary being present for each coordinate of the image, and the probability value and the corresponding coordinate are acquired; sorting the plurality of coordinates in descending order of the probability value; using the plurality of coordinates in the sorted order to sequentially update parameters of a function that approximates the road boundary in the image; performing driving control or driving assistance for the moving object based on the road boundary approximated by the function defined by the updated parameters; Image processing methods.

6. On the computer, In response to an input of an image showing a region ahead of a moving body captured by a camera mounted on the moving body, a trained model is used to output a probability value indicating the probability of a road boundary being present for each coordinate of the image, and the probability value and the corresponding coordinate are acquired; sorting the plurality of coordinates in descending order of the probability value; using the plurality of coordinates in the sorted order to sequentially update parameters of a function that approximates the road boundary in the image; performing driving control or driving assistance for the mobile object based on the road boundary approximated by the function defined by the updated parameters; program.

Images