Image processing apparatus, image processing method, and storage medium

Abstract

Provided are an image processing device, an image processing method, and a storage medium capable of estimating a road division line that enables stable travel control of a mobile body. An image processing device includes: an element function generation unit that divides an image representing a front region of a mobile body, captured by a camera mounted on the mobile body, at a prescribed interval, and generates an element function approximating a travel road boundary in each region based on a probability value representing a presence probability of the travel road boundary in coordinates within each divided region and the coordinates; a synthesis function generation unit that synthesizes the element functions generated for the regions, thereby generating a synthesis function approximating the travel road boundary in the front region; and a control unit that performs driving control or driving support of the mobile body based on the travel road boundary approximated by the generated synthesis function.

Description

Technical Field

[0001] This invention relates to an image processing apparatus, an image processing method, and a storage medium. Background Technology

[0002] Previously, there were known techniques for estimating road markings on the road a vehicle travels on and controlling the vehicle's movement based on these estimated road markings. For example, Japanese Patent Application Publication No. 2021-60885 disclosed a technique in which multiple three-dimensional objects are selected from an image captured by a camera mounted on a vehicle, road markings are estimated based on the positions of the selected three-dimensional objects, and a target speed for the vehicle is set according to the curvature of the estimated road markings.

[0003] In the technology described in Japanese Patent Application Publication No. 2021-60885, multiple positions extracted from an image of a road on which a vehicle is traveling are fitted with a curve to deduce road markings. However, this method suffers from several drawbacks: the constraints on the degrees of freedom of the fitted curve lead to large errors in the deduced road markings; furthermore, the variations in the positions extracted from the image at each time point result in blurred or indistinct road markings. Consequently, the deduced road markings sometimes cannot be reliably used for vehicle driving control. Summary of the Invention

[0004] The present invention was made in consideration of such circumstances, and one of its objectives is to provide an image processing apparatus, an image processing method, and a storage medium capable of stably utilizing road markings for the driving control of moving bodies.

[0005] The image processing apparatus, image processing method, and storage medium involved in this invention adopt the following structure.

[0006] (1): One aspect of the present invention relates to an image processing apparatus, wherein the image processing apparatus comprises: an element function generation unit that divides an image representing a region in front of the mobile body captured by a camera mounted on the mobile body at predetermined intervals, and generates an element function that approximates the road boundary in each region based on a probability value representing the probability of the existence of a road boundary in each region and the coordinates therein; a synthesis function generation unit that synthesizes the element functions generated in each region to generate a synthesis function that approximates the road boundary in the region in front; and a control unit that performs driving control or driving support for the mobile body based on the road boundary approximated by the generated synthesis function.

[0007] (2): Based on the above (1) scheme, the feature function generation unit uses a learned model that has been trained to output a probability value representing the probability of the existence of the driving road boundary according to the coordinates based on the input of the image to obtain the probability value, and generates the feature function based on the probability value and the coordinates.

[0008] (3): Based on the above scheme (1), the synthesis function generation unit multiplies the element function generated according to each region with the weighting function defined according to each region and sums them to generate the synthesis function.

[0009] (4): Based on any of the above schemes (1) to (3), the feature function is a quadratic function. The feature function generation unit updates the parameters of the quadratic function that approximates the boundary of the driving road by using coordinates sequentially starting from the coordinate with the larger probability value, thereby generating the feature function.

[0010] (5): Another aspect of the present invention relates to an image processing method, wherein the image processing method causes a computer to perform the following processing: segmenting an image representing a region in front of the mobile body captured by a camera mounted on the mobile body at a predetermined interval, and generating an element function that approximates the road boundary in each region based on a probability value representing the probability of the existence of a road boundary in each segmented region and the coordinates thereon; synthesizing the element functions generated for each region to generate a composite function that approximates the road boundary in the region in front; and performing driving control or driving support for the mobile body based on the road boundary approximated by the generated composite function.

[0011] (6): Another aspect of the present invention relates to a storage medium storing a program, wherein the program causes a computer to perform the following processing: segmenting an image representing a region in front of the mobile body captured by a camera mounted on the mobile body at predetermined intervals, and generating an element function that approximates the road boundary in each region based on a probability value representing the probability of the existence of a road boundary in each segmented region and the coordinates thereon; synthesizing the element functions generated for each region to generate a composite function that approximates the road boundary in the region in front of the mobile body; and performing driving control or driving support for the mobile body based on the road boundary approximated by the generated composite function.

[0012] Based on (1) to (6), it can be inferred that road markings can be stably used for driving control of moving bodies. Attached Figure Description

[0013] Figure 1This diagram illustrates an example of the usage environment of the image processing device mounted on the vehicle M.

[0014] Figure 2 This is a diagram illustrating an example of the structure of an image processing device.

[0015] Figure 3 This is a diagram illustrating an example of a method for extracting candidate points from the boundary of a driving path by a candidate point extraction unit.

[0016] Figure 4 This diagram illustrates an example of a method for rearranging candidate points at the boundaries of a driving road.

[0017] Figure 5 This is a diagram used to illustrate a simplified view of the driving road boundary model updated by the model parameter update unit.

[0018] Figure 6 This is a diagram illustrating an example of the process of updating model parameters performed by the model parameter update unit.

[0019] Figure 7 It is a line graph used to illustrate the method for calculating the baseline values ​​of model parameters.

[0020] Figure 8 This is a sequence diagram illustrating an example of the processing flow performed by an image processing device. Detailed Implementation

[0021] Hereinafter, embodiments of the image processing apparatus, image processing method, and storage medium of the present invention will be described with reference to the accompanying drawings. In this embodiment, the image processing apparatus is, for example, a terminal device such as a smartphone having a camera and a display. However, the present invention is not limited to such a structure; the image processing apparatus may be any computer device that at least receives and processes images captured by a camera and outputs the processing results to a display. In this case, the camera, display, and image processing apparatus cooperate to achieve the function of the present invention.

[0022] [structure]

[0023] Figure 1 This diagram illustrates an example of the operating environment of the image processing device 100 mounted on the vehicle M. The vehicle M is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electricity generated by a generator connected to the internal combustion engine, or electricity discharged from a secondary battery or fuel cell.

[0024] like Figure 1As shown, the image processing device 100 is installed in the vehicle M in such a way that it can use the camera 10 to capture images of the area in front of the vehicle M in relation to its direction of travel. The image processing device 100 is held, for example, by a vehicle-mounted bracket (not shown) mounted on the dashboard of the vehicle M, and captures images of the area in front of the vehicle M. The vehicle M is an example of a "moving body". In the following description, in this embodiment, an example of the image processing device 100 being mounted on the vehicle M, which is a moving body, is described. However, more generally, a moving body includes a device with a camera mounted on a vehicle, such as a dashcam or a smartphone.

[0025] Figure 2 This is a diagram illustrating an example of the structure of the image processing apparatus 100. (As shown...) Figure 2 As shown, the image processing apparatus 100 includes, for example, a camera 10, a display unit 20, a candidate point extraction unit 110, a model parameter update unit 120, and a driving control unit 130. The candidate point extraction unit 110, the model parameter update unit 120, and the driving control unit 130 are implemented, for example, by executing programs (software) using a hardware processor such as a CPU (Central Processing Unit). Some or all of these components can be implemented by hardware (including the circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or through a combination of software and hardware. The program can be pre-stored in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transitory storage medium), or stored in a removable storage medium such as a DVD or CD-ROM (a non-transitory storage medium), and installed by mounting the storage medium onto a drive device. The camera 10 is, for example, a digital camera that utilizes a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The display unit 20 is, for example, a display device such as a touch panel or a liquid crystal display.

[0026] [Extraction of candidate points]

[0027] The candidate point extraction unit 110 extracts candidate points of the boundary (road boundary) of the driving path of the vehicle M based on the image representing the area in front of the vehicle M captured by the camera 10. Figure 3This diagram illustrates an example of a method used by the candidate point extraction unit 110 to extract candidate points for the boundary of the driving path. (See diagram for example.) Figure 3 As shown, after acquiring an image captured by the camera 10, the candidate point extraction unit 110 inputs the acquired image into a learned Deep Neural Network (DNN). The learned DNN is trained to output a probability value (between 0 and 1) for each pixel (coordinate) of the image representing whether it is a road boundary. The candidate point extraction unit 110 extracts pixels with positive output probability values ​​as candidate points for road boundaries. Alternatively, the candidate point extraction unit 110 may extract pixels with output probability values ​​of a threshold (e.g., 0.5) or higher as candidate points for road boundaries.

[0028] [Model parameter update]

[0029] The model parameter update unit 120 rearranges the candidate points of the driving road boundary extracted by the candidate point extraction unit 110 in order starting from the candidate point with the highest probability value, and uses the candidate points in sequence to update the model parameters of the driving road boundary model described later. Figure 4 This diagram illustrates an example of a method for rearranging candidate points along the boundaries of a driving path. Figure 4 In this diagram, k represents a time point in the recognition loop for identifying the road boundary, N(k) represents the number of candidate points for the road boundary obtained at time point k, x'(n,k) represents the x-coordinate of each candidate point before rearrangement, y'(n,k) represents the y-coordinate of each candidate point before rearrangement, x(n,k) represents the x-coordinate of each candidate point after rearrangement, and y(n,k) represents the y-coordinate of each candidate point after rearrangement. As described later, starting from the candidate point of the road boundary with the highest probability value, the model parameters of the road boundary model are updated sequentially using candidate points. Thus, even if downsampling occurs within a recognition loop, a high-accuracy road boundary model can be output even in a shorter time.

[0030] Figure 5 This is a diagram illustrating a simplified view of the driving road boundary model updated by the model parameter update unit 120. The model parameter update unit 120 sequentially substitutes the candidate points of the driving road boundary, which have been rearranged in order from the candidate points with the highest probability values, into the driving road boundary model defined by the following equation (1), thereby obtaining the estimated value y_ of the driving road boundary. hat (n, k).

[0031]

[0032]

[0033] In equation (1), a i (n, k), b i (n, k), c i (n, k) represent the quadratic coefficient, linear coefficient, and constant term of the quadratic function (sometimes referred to as the "element function") that approximates the road boundary in the image, respectively. i This represents a weighting function that assigns weights greater than 0 and less than 1 to the x-coordinates of the input candidate points. More specifically, the weighting function is defined by the following equations (2) to (4). In equations (2) to (4), x wi The values ​​of (i = 1 to m) are fixed values ​​that are set in advance. i The sum of (x)(i=1~m) is set to always be 1.

[0034]

[0035]

[0036]

[0037] Figure 5 Let f1, f2, f3 and w1, w2, w3 be the element functions when m = 3 in equations (1) to (4). That is, in this embodiment, the driving road boundary model divides the image into multiple regions based on the x-coordinate, approximates the driving road boundary of each region using a quadratic function, and synthesizes the quadratic functions of each region to approximate the driving road boundary of the entire image.

[0038] In contrast to the conventional method of approximating the candidate points of the road boundary in the image using a single quadratic function (the least squares method of summation), this embodiment approximates the road boundary using a quadratic function for each region of the image and synthesizes the approximate quadratic functions to obtain the final approximate curve. Therefore, it can represent the road boundary in the image with higher accuracy.

[0039] It should be noted that, in this embodiment, the x-coordinate x is used to segment the region. wi The values ​​and number of (i = 1 to m) are fixed values ​​set in advance. However, the present invention is not limited to such a structure, and the x-coordinates for dividing the region are also considered. wi The value and number can also be set to the number of clusters and their boundary points obtained by clustering the extracted candidate points.

[0040] Next, refer to Figure 6 This section explains the model parameter update process performed by the model parameter update unit 120. Figure 6 This is a diagram illustrating an example of the process of updating model parameters performed by the model parameter update unit 120.

[0041] First, the model parameter update unit 120 rearranges the candidate points (x'(n,k), y'(n,k)) of the driving road boundary extracted by the candidate point extraction unit 110 in order starting from the candidate points with higher probability values, thus obtaining candidate points (x(n,k), y(n,k)). The model parameter update unit 120 then generates a vector ξ(n,k) = [x(n,k)] based on the candidate points (x(n,k)]. 2 ,x(n,k),1], and substitute x(n,k) into the weighting function w i And thus the weight value w is obtained. i (x(n, k)).

[0042] Next, the model parameter update unit 120 calculates the vector ξ(n, k) and the model parameter θ. i (n, k) = [a i (n, k), b i (n, k), c i The inner product between [n, k] is multiplied by the weight value w. i (x(n, k)), from which the element function f of each region is obtained. i (x)=w i (x(n,k))(a i (n, k)x(n, k) 2 +b i (n, k)x(n, k)+c i (n, k)). The model parameter update unit 120 takes the feature function f of each region. i The sum of (x) is used to obtain the output estimate y_ expressed by equation (1). hat (n, k). It should be noted that the model parameters θ being multiplied at this point... i (n, k) = [a i (n, k), b i (n, k), c i The initial values ​​of [n, k] are described later.

[0043] Next, the model parameter update unit 120 outputs the estimated value y_ hat The fixed error between (n, k) and the y-coordinate y(n, k) of the candidate point is taken as e id (n, k) = w i (x(n,k))(y(n,k)-y_ hat It can be calculated from (n, k). The estimated value y_ can be obtained by outputting the estimated value. hat The error between (n, k) and the y-coordinate y(n, k) of the candidate point is multiplied by the weight value w.i (x(n, k)) is used to reflect the uniform error in each region. The model parameter update unit 120 defines the uniform error e using the following equation (5). id Direction correction model parameter θ for reducing the squared error of (n, k) i The adaptive gain K of (n, k) p .

[0044]

[0045] In equation (5), P'(n, k) represents a three-row, three-column covariance matrix, which is defined by the following equation (6).

[0046]

[0047] In equation (6), I represents a 3x3 identity matrix, and λ1 and λ2 represent the parameters set for the successive fixed-gain algorithm. λ1 and λ2 are constant values ​​greater than 0 and less than 1. When applying the least squares method, λ1 = 1 and λ2 = 1 are set; when applying the weighted least squares method, λ1 = λ (0 < λ ≤ 1) and λ2 = 1 are set; and when applying the fixed-gain method, λ1 = 1 and λ2 = 0 are set. In the case of applying the fixed-gain method, the adaptive gain K is... p It is represented by the following equations (7) and (8).

[0048]

[0049]

[0050] In equation (8), P represents the constant gain matrix. P1, P2, and P3 represent constant gains, which are positive fixed values. The model parameter update unit 120 updates the constant error e. id (n, k) multiplied by the adaptation gain K p Thus, the correction amount ddθ of the model parameters is obtained as expressed by the following equation (9). i (n, k).

[0051] ddθ i (n, k) = K p (n, k)e id_i (n, k)(i = 1 to m)

[0052] ...Formula (9)

[0053] Next, the model parameter update unit 120 updates the previous value dθ of the final correction amount, which will be described later. i (n-1, k) multiplied by forget gain Δ fgt And make the obtained product value the correction amount ddθ of the model parameters in equation (9). iBy adding (n, k), we obtain the correction amount dθ of the model parameters as expressed by the following equation (10). raw_i (n, k). Thus, by using the final correction amount dθ obtained last time... i (n-1, k) multiplied by forget gain Δ fgt The obtained value is used to define the correction amount dθ raw_i (n, k) can suppress drastic changes in the boundary model of the driving road.

[0054] dθ raw_i (n, k) = Δ fgt dθ i (n-1, k)+K p (n, k)e id_i (n, k)

[0055] ...Formula (10)

[0056] In equation (10), forget the gain Δ fgt It is a 3x3 diagonal matrix represented by the following equation (11). In equation (11), δ fB_1 δ fgt_2 δ fgt_3 It satisfies 0 < δ fgt_1 δ fgt_2 <1, δ fgt_3 =1 is a constant value. That is, forget about the gain Δ. fgt Set as to a i (n, k) and b i (n, k) applies the forgetting effect.

[0057]

[0058] Next, the model parameter update unit 120 corrects the model parameters by adjusting dθ. raw_i (n, k) is subjected to the constraint treatment expressed by the following equations (12) to (14) (as an example of "constraint conditions"), thereby correcting the correction amount of the model parameters, and obtaining the final correction amount dθ expressed by the following equation (15). i (n, k). In equations (12) to (14), da L da H db L db H ,dc L ,dc H It is a pre-set fixed value, set to prevent the driving road boundary model from becoming an unrealistic shape.

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] Next, the model parameter update unit 120 adjusts the correction amount dθ. i (n, k) and the baseline values ​​θ of the model parameters expressed by the following equation (16) base_i Adding (n, k), we obtain the current model parameter value θ as expressed by the following equation (17). i (n, k). The calculated model parameter values ​​θ i (n, k) is used to calculate the model parameters θ for the next input value n+1. i The same fixed value (initial value) for (n+1, k).

[0065]

[0066] θ i (n, k) = θ base_i (n, k)+dθ i (n, k)

[0067] ...Formula (17)

[0068] [Baseline values ​​for model parameters calculated]

[0069] Next, refer to Figure 7 To illustrate how the baseline value θ of the model parameters is calculated. base_i The method of (n, k). Figure 7 This is a line graph illustrating the method for calculating the baseline values ​​of model parameters. The model parameter update unit 120 sets the baseline values ​​θ of the model parameters using the following equations (18) to (20). base_i (n, k).

[0070] a base_i (n, k) = 0

[0071] ...Formula (18)

[0072] c base_i (n, k) = 0

[0073] ...Formula (19)

[0074] b base_i (n, k) = g(c i (n-1, k))

[0075] =αc i (n-1,k)+β

[0076] ...Formula (20)

[0077] Equations (18) and (19) respectively represent the model parameter a base_i (n, k), c base_i The initial values ​​of (n, k). As shown in equations (18) and (19), the curvature of the driving road boundary model may take the left and right directions, therefore the model parameter a base_i The initial value of (n, k) can be zero. Additionally, the model parameter c is equivalent to the y-intercept of the driving road boundary model. i It can take either the left or right direction, therefore the model parameter c base_i The initial value of (n, k) can be zero.

[0078] In equation (20), c i (n-1, k) represents the model parameters c calculated in the last time. i The function g represents the value given by the model parameter c. i The scaling function of the line at the vanishing point VP of the image, where the y-intercept and the y-intercept are constant values. That is, as shown... Figure 7 As shown on the left side, with the same fixed value c i The larger the value of (n-1, k), the higher the baseline value of the model parameters, b. base_i (n, k) takes the smallest possible value, such as Figure 7 As shown on the right side, the straight line slopes further to the left. The baseline value θ of the model parameters is set using the method described above. base_i (n, k), for example, even if the number of DNN output values ​​N(k) in a certain recognition loop is significantly small due to phenomena such as bad weather or dark vision, it can prevent the driving road boundary model from becoming an unrealistic shape.

[0079] After the model parameter update unit 120 determines the road boundary model in each recognition cycle, the driving control unit 130 performs autonomous driving or driving support for the vehicle M based on the determined road boundary model. More specifically, for example, the driving control unit 130 performs a bird's-eye view transformation on the road boundary model in the camera coordinate system to obtain the road boundary model in the bird's-eye view coordinate system. Using the road boundary model in the bird's-eye view coordinate system, the driving control unit 130 generates a target trajectory and action plan for the vehicle M, and drives the vehicle M according to the generated target trajectory and action plan. In addition, for example, when the occupants of the vehicle M are manually driving, the driving control unit 130 uses the road boundary model in the bird's-eye view coordinate system to assist steering and issue warnings to avoid deviating from the determined road boundary model.

[0080] Figure 8This is a sequence diagram illustrating an example of the processing flow performed by the image processing device 100. For example... Figure 8 As shown, at time point k-2, the image processing device 100 rearranges the DNN output values ​​x'(1,k-2), y'(1,k-2), x'(2,k-2), y'(2,k-2), ..., x'(N(k-2),k-2)), y'(N(k-2),N(k-2)) in order from the output value with the highest probability value, resulting in x(1,k-2), y(1,k-2), x(2,k-2), y(2,k-2), ..., x(N(k-2),k-2), y(N(k-2),k-2). The image processing device 100 constructs a vector ξ(n,k-2) = [x(n,k-2)] in order from the output value with the highest probability value. 2 ,x(n,k-2),1], and towards Figure 6 The successive fixed-type algorithm input shown is used to successively update the model parameter values ​​θ. i (n, k-2).

[0081] In the model parameter value θ i The update of (n, k-2) goes through a resampling time T. ds In the case of image processing device 100, the image processing device 100 pairs the images after resampling time T ds Model parameter values ​​at subsequent time points (e.g., θ) i The model parameter values ​​in the identification loop k-2 are determined by downsampling (N(k-2)-1, k-2). The image processing device 100 causes the display unit 20 to display the determined model parameter values ​​θ. i The driving path boundary model is N(k-2)-1, k-2. Thus, unlike the least-two multiplication method of the generalization approach, in this embodiment, the model parameter values ​​θ are updated sequentially using the output values ​​starting from the output values ​​with higher probability values. i (n, k), thus even when the amount of data is large and the least squares method of the general operation method cannot be completed, a high-precision driving road boundary model can still be estimated.

[0082] When the image processing device 100 reaches recognition loop k-1, it rearranges the DNN output values ​​x'(1,k-1), y'(1,k-1), x'(2,k-1), y'(2,k-1), ..., x'(N(k-1),k-1), y'(N(k-1),k-1) in order of probability, resulting in x(1,k-1), y(1,k-1), x(2,k-1), y(2,k-1), ..., x(N(k-1),k-1), y(n(k-1),k-1). The image processing device 100 constructs a vector ξ(n,k-1) = [x(n,k-1)] in order of probability. 2 ,x(n,k-1),1], and towards Figure 6 The successive identically determined algorithm inputs are used to update the model parameter values ​​θ. i (n, k-1). It should be noted that at this point, the image processing device 100 will use the model parameter values ​​θ determined in recognition loop k-2. i (N(k-2)-1,k-2)) is used to calculate the estimated value y_ hat The initial values ​​are (1, k-1). The model parameter values ​​θ are obtained by averaging the output values ​​of x(N(k-1), k-1) and y(N(k-1), k-1). i When the update of (n, k-1) is complete, the image processing device 100 holds the model parameter value θ until the next recognition loop k. i (N(k-1), k-1), and the display unit 20 displays the model parameter value θ that has been set and maintained. i The driving road boundary model is (N(k-1), k-1). Then, when it becomes recognition cycle k, similarly, the image processing device 100 uses the DNN output values ​​x'(1,k), y'(1,k), x'(2,k), y'(2,k), ..., x'(N(k),k), y'(N(k),k) to update the model parameter values ​​θ. i (n, k). At this time, the image processing device 100 will use the model parameter values ​​θ determined in the recognition loop k-1. i (N(k-1), k-1) is used to calculate the estimated value y_ hat The initial value of (1, k).

[0083] According to the embodiment described above, the image processing device divides the forward region in a coordinate system based on the vehicle at predetermined intervals. Based on the probability value representing the probability of the existence of road markings in each segmented region and the coordinates thereon, it generates a function that approximates the road markings in each region. The functions generated for each region are then combined to generate a function that approximates the road markings in the forward region. Therefore, road markings that can be stably used for driving control of a moving vehicle can be estimated.

[0084] The implementation methods described above can be performed as follows.

[0085] An image processing apparatus, configured as follows:

[0086] have:

[0087] A storage device containing a program; and

[0088] Hardware processor,

[0089] The image representing the area in front of the mobile body, captured by a camera mounted on the mobile body, is segmented at a predetermined interval. Based on the probability value of the existence probability of the driving road boundary in each segmented area and the coordinates therein, an element function is generated to approximate the driving road boundary in each area.

[0090] The element functions generated according to each region are synthesized, thereby generating a composite function that approximates the road boundary in the forward region; and

[0091] Driving control or driving support for the mobile body is performed based on the driving road boundary approximated by the generated synthetic function.

[0092] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. An image processing apparatus, wherein, The image processing device includes: The feature function generation unit divides an image representing the area in front of the mobile body, captured by a camera mounted on the mobile body, into segments at predetermined intervals, and generates a feature function that approximates the road boundary in each segment based on the probability value of the existence probability of candidate points representing the road boundary in each segmented area and the coordinates therein. The synthesis function generation unit synthesizes the element functions generated according to each region, thereby generating a synthesis function that approximates the road boundary in the forward region; as well as The control unit performs driving control or driving support for the moving body based on the driving path boundary approximated by the generated synthetic function. The element function is a quadratic function. The element function generation unit uses coordinates sequentially, starting from the coordinates with the largest probability values, to update the parameters of the quadratic function that approximates the boundary of the driving road for each candidate point, thereby generating the element function.

2. The image processing apparatus according to claim 1, wherein, The feature function generation unit uses a learned model that has been trained to output a probability value representing the probability of the existence of candidate points representing the road boundary according to the coordinates based on the input of the image to obtain the probability value, and generates the feature function based on the probability value and the coordinates.

3. The image processing apparatus according to claim 1, wherein, The synthetic function generation unit multiplies the element function generated according to each region with the weighting function defined according to each region and sums them to generate the synthetic function.

4. An image processing method, wherein, The image processing method causes the computer to perform the following processing: The image representing the area in front of the mobile body, captured by a camera mounted on the mobile body, is segmented at a predetermined interval. Based on the probability value of the existence probability of candidate points representing the road boundary in each segmented area and the coordinates therein, an element function is generated to approximate the road boundary in each area. The element functions generated according to each region are synthesized to generate a composite function that approximates the driving road boundary in the forward region. The driving control or driving support of the mobile body is performed based on the driving road boundary approximated by the generated synthesis function. as well as The element function is a quadratic function. Starting from the coordinates with the largest probability values, the parameters of the quadratic function that approximates the boundary of the driving road are updated sequentially for each candidate point, thereby generating the element function.

5. A storage medium storing a program, wherein, The program causes the computer to perform the following processing: The image representing the area in front of the mobile body, captured by a camera mounted on the mobile body, is segmented at a predetermined interval. Based on the probability value of the existence probability of candidate points representing the road boundary in each segmented area and the coordinates therein, an element function is generated to approximate the road boundary in each area. The element functions generated according to each region are synthesized to generate a composite function that approximates the driving road boundary in the forward region. The driving control or driving support of the mobile body is performed based on the driving road boundary approximated by the generated synthesis function. as well as The element function is a quadratic function. Starting from the coordinates with the largest probability values, the parameters of the quadratic function that approximates the boundary of the driving road are updated sequentially for each candidate point, thereby generating the element function.

Images