Training method of road information estimation model, road information extraction method and device

Abstract

The application provides a road information estimation model training method, a road information extraction method and device, and relates to the technical field of image processing. The road information estimation model training method comprises the following steps: acquiring continuous frame images containing road plane areas and non-road plane areas; using an initial road information estimation model to generate depth images and height images corresponding to the continuous frame images; calculating photometric error values of the non-road plane areas after image alignment processing according to the depth images and the height images; taking the photometric error values as a supervised loss term to perform pre-estimation training of road information in the road plane areas on the initial road information estimation model, and determining the trained initial road information estimation model as a target road information estimation model. The application can make the pre-estimated road information accurately reflect the real road conditions, thereby effectively ensuring the safety of driving.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and in particular to a training method of a road information estimation model, a road information extraction method and device. BACKGROUND

[0002] In the actual driving environment of a vehicle, not only the distance between the vehicle and each target object needs to be determined, but also the surrounding road conditions need to be recognized while ensuring driving safety. By collecting road information, an intelligent vehicle can adaptively adjust its control strategy, thereby improving its driving efficiency, stability and comfort.

[0003] At present, road depth information can be estimated based on deep learning. Existing deep learning estimation tasks can be summarized as follows: the depth values of the road plane regions in sample images are labeled, and the labeled sample images are used to pre-estimate and train a network estimation model for road depth values.

[0004] However, under this model training method, the network estimation model can only estimate road depth values, resulting in a relatively single feature dimension of the estimated road information, which cannot accurately reflect the real road conditions, and thus cannot guarantee the safety of driving. SUMMARY

[0005] The present application provides a training method of a road information estimation model, a road information extraction method and device, which can calculate photometric error values according to depth images and height images, perform supervised loss training of the model based on the photometric error values, so that the model can simultaneously estimate depth values and height values for road conditions, and multiple feature dimensions of road information can accurately reflect the real road conditions, thereby effectively guaranteeing the safety of driving.

[0006] In a first aspect, a training method of a road information estimation model is provided, comprising:

[0007] obtaining continuous frame images containing road plane regions and non-road plane regions;

[0008] generating depth images and height images corresponding to the continuous frame images by using an initial road information estimation model;

[0009] calculating photometric error values of the non-road plane regions after image alignment processing according to the depth images and the height images;

[0010] using the photometric error values as a supervised loss term to perform pre-estimation training of road information in the road plane regions on the initial road information estimation model, and determining the trained initial road information estimation model as a target road information estimation model.

[0011] In a second aspect, a road information extraction method is provided, comprising:

[0012] determining a target continuous frame image to be extracted for road information;

[0013] inputting the target continuous frame image into the target road information estimation model to obtain target road information, the target road information at least including target depth values and target height values of road images.

[0014] In a third aspect, a training device for a road information estimation model is provided, comprising:

[0015] an acquisition module configured to acquire a continuous frame image containing a road plane area and a non-road plane area;

[0016] a generation module configured to generate a depth image and a height image corresponding to the continuous frame image by using an initial road information estimation model;

[0017] a calculation module configured to calculate a photometric error value of the non-road plane area after image alignment processing according to the depth image and the height image;

[0018] a training module configured to perform pre-estimation training of the initial road information estimation model in the road plane area by taking the photometric error value as a supervised loss term, and determine the trained initial road information estimation model as a target road information estimation model.

[0019] In a fourth aspect, a road information extraction device is provided, comprising:

[0020] a determination module configured to determine a target continuous frame image to be extracted for road information;

[0021] an input module configured to input the target continuous frame image into the target road information estimation model to obtain target road information, the target road information at least including target depth values and target height values of road images.

[0022] In a fifth aspect, an electronic device is provided, comprising a processor and a memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program stored in the memory to execute the method in the first aspect or any implementation manner thereof.

[0023] In a sixth aspect, a computer readable storage medium is provided, configured to store a computer program, the computer program enabling a computer to execute the method in the first aspect or any implementation manner thereof.

[0024] By the technical solution provided in the present application, after a continuous frame image containing a road plane area is acquired, an initial road information estimation model can be used to generate a depth image and a height image corresponding to the continuous frame image, and then, according to the depth image and the height image, a photometric error value of a non-road plane area after image alignment processing is calculated, the photometric error value is taken as a supervision loss item for model training, the initial road information estimation model is pre-estimated and trained for road information in the road plane area, so as to obtain a trained target road information estimation model. The technical solution in the present application can calculate the photometric error value according to the depth image and the height image, and increase the supervision loss item of the photometric error value in the model training process, which can guide the model to comprehensively learn the features in the depth feature dimension and the height feature dimension of the image area when the model is trained for road information, so that the model can simultaneously estimate the depth value and the height value for the road condition, and multiple feature dimensions of the road information can accurately reflect the real road condition, thereby effectively ensuring the safety of driving.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Other features and advantages of the present application will be described in detail in the subsequent specific embodiments section. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0027] Figure 1 An application scenario graph provided for the embodiments of the present application;

[0028] Figure 2 A flowchart of a training method of a road information estimation model provided for the embodiments of the present application;

[0029] Figure 3 A principle flowchart of road information estimation provided for the embodiments of the present application;

[0030] Figure 4 A flowchart of a training method of a road information estimation model provided for the embodiments of the present application;

[0031] Figure 5 A flowchart of a road information extraction method provided for the embodiments of the present application;

[0032] Figure 6 A structural diagram of a training device of a road information estimation model provided for the embodiments of the present application;

[0033] Figure 7 A structural schematic diagram of a road information extraction device provided by an embodiment of the present application is shown in the figure.

[0034] Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0036] It should be noted that the terms “first”, “second”, and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0037] In the field of assisted driving and automatic driving technology, road information estimation plays a very important role. At present, the depth information of the road can be estimated based on deep learning. The existing deep learning estimation task can be summarized as follows: the depth values of the road plane region in the sample image are labeled, and the labeled sample image is used to pre-estimate and train the network estimation model of the road depth value.

[0038] As described above, under this model training method, the network estimation model can only estimate the road depth value, resulting in a relatively single feature dimension of the estimated road information, which cannot accurately reflect the real road conditions, and thus cannot guarantee the safety of driving.

[0039] To address the aforementioned technical problems, the inventive concept of this application is as follows: An electronic device can calculate the photometric error value of non-road planar regions after image alignment processing based on the depth and height images output by the initial road information estimation model. This photometric error value is then used as a supervised loss term for model training, enabling the initial road information estimation model to predict road information within road planar regions. By adding this photometric error value as a supervised loss term during model training, the model can be guided to comprehensively learn features under both the depth and height dimensions of the image region during road information training, allowing the model to simultaneously predict both road depth and height values.

[0040] It should be understood that the technical solution of this application can be applied to the following scenarios, but is not limited to:

[0041] In some possible ways, Figure 1 An application scenario diagram provided for an embodiment of this application, such as... Figure 1 As shown, this application scenario may include electronic device 110 and network device 120. Electronic device 110 can establish a connection with network device 120 through a wired network or a wireless network.

[0042] For example, electronic device 110 may be a desktop computer, laptop computer, tablet computer, etc., but is not limited thereto. Network device 120 may be a terminal device or a server, but is not limited thereto. In one embodiment of this application, electronic device 110 may send a request message to network device 120, which may be used to request the acquisition of continuous frame images. Further, electronic device 110 may receive a response message sent by network device 120, which includes continuous frame images.

[0043] also, Figure 1 An electronic device 110 and a network device 120 are provided as examples, but other numbers of electronic devices and network devices may be included in practice, and this application does not limit this.

[0044] In other possible implementations, the technical solution of this application may also be executed by the aforementioned electronic device 110, or by the aforementioned network device 120, and this application does not impose any restrictions on this.

[0045] After introducing the application scenarios of the embodiments of this application, the technical solution of this application will be described in detail below:

[0046] Figure 2 A flowchart illustrating a training method for a road information estimation model provided in this application embodiment is shown. This method can be performed by, for example... Figure 1 The electronic device 110 shown performs, but is not limited to, its functions. For example... Figure 2 As shown, the method may include the following steps:

[0047] Step 210, acquiring continuous frame images containing road plane areas and non-road plane areas.

[0048] The continuous frame images can be two frames of images obtained after continuous shooting by using a front-view camera. The continuous frame images can contain road plane areas and non-road plane areas. The road plane areas can contain at least one road object, such as road marking lines, cracks, and ruts. The non-road plane areas can contain at least one three-dimensional object, such as obstacles, pedestrians, vehicles, potholes, speed bumps, clouds, flying objects, trees, buildings, signs, and indicator lights.

[0049] In a specific application scenario, to ensure the training accuracy of the road information estimation model and avoid training errors caused by image distortion, the continuous frame images can be two frames of images obtained after image correction after being collected by the front-view camera. Image correction refers to a recovery process for distorted images. The causes of image distortion include image distortion caused by aberration, distortion, and limited bandwidth of the imaging system, image geometric distortion caused by the imaging device shooting posture and scanning nonlinearity, and image distortion caused by motion blur, radiation distortion, and noise introduction. The basic idea of image correction is to establish a corresponding mathematical model according to the causes of image distortion, extract the required information from the contaminated or distorted image signals, and restore the original appearance of the image along the inverse process of image distortion.

[0050] In the task learning of the road information estimation model in the prior art, only the road depth information is estimated, and the influence of the three-dimensional object on the road information estimation in the non-road plane area is ignored. According to the embodiment of the present application, a plurality of continuous frame image data collected by the front-view camera can be acquired first, and the plurality of continuous frame image data can be divided into a training data set and a verification data set. The training data set is used for training the model, and the verification data set is used for evaluating the performance of the model and adjusting the parameters. Further, the road plane areas and the non-road plane areas can be comprehensively analyzed based on the continuous frame images in the training data set. The three-dimensional road information of the road environment can be estimated and trained based on the new geometric constraint of the road plane disparity and in combination with the deep neural network. The model can further estimate the road depth value and the height value simultaneously. The feature dimensions of the plurality of road information can accurately reflect the real road conditions, thereby effectively ensuring the safety of driving.

[0051] Step 220, generating depth images and height images corresponding to the continuous frame images by using an initial road information estimation model.

[0052] The initial road information estimation model can be any deep convolutional neural network model, such as a residual network (ResNet), a lightweight model (MobileNet), or a network model with some modifications to the network layers based on these models. No specific limitations are imposed here. The depth image can contain the depth value of each pixel, and the height image can contain the height value of each pixel.

[0053] For the embodiments of this application, such as Figure 3 As shown, consecutive frames (i.e., two consecutive frames) from the training dataset can be used as input, with the road plane as the homography plane. A simple homography network generates a homography matrix relative to the road plane. The homography matrix is ​​then used to align the road plane regions in the two frames, resulting in a homography-aligned image. This aligned image is then input into an initial road information estimation model, which outputs two pixel-level maps: a pixel-level depth map (depth image) and a pixel-level elevation map (height image). The homography network used to generate the homography matrix can be a deep learning network (such as ResNet34 or ResNet50), or a convolutional neural network with modified layers (such as adding convolutional layers), etc. No specific limitations are imposed here.

[0054] Step 230: Calculate the photometric error value of the non-road plane area after image alignment processing based on the depth image and height image.

[0055] In one possible implementation of this disclosure, the photometric error value of consecutive frame images after image alignment processing can be calculated based on the depth and height images. This photometric error value reflects the alignment effect between the road plane area and the non-road plane area. This method eliminates the need for dividing the road plane area into road plane and non-road plane areas, reducing the workload of area division. Furthermore, calculating the photometric error for the road plane area can correct the homography alignment error of the road plane area; calculating the photometric error for the non-road plane area can eliminate the influence of the non-road plane area on road information prediction in consecutive frame images, eliminate alignment residues between consecutive frame images, and ensure good alignment of the non-road plane area between two frames.

[0056] For the embodiment of the present application, as a possible implementation manner, the non-road plane region can be determined in the continuous frame images. Further, according to the depth image and the height image, the photometric error value of the non-road plane region after image alignment processing is calculated, and the photometric error value at this time is used to reflect the alignment effect of the two frames of images on the non-road plane region. In this way, the calculation amount of the photometric error for the road plane region can be reduced, and the training efficiency of the model can be improved. Since the depth image and the height image are determined based on the road plane region in the continuous frame images, in order to eliminate the influence of the non-road plane region in the continuous frame images on the road information estimation, after the depth image and the height image are determined, the non-road plane region of the continuous frame images can be subjected to image alignment processing based on the depth image and the height image, so as to eliminate the alignment residual between the continuous frame images and make the non-road plane regions of the two frames of images well aligned.

[0057] The photometric error value can be a value between 0 and 1. The closer the photometric error value is to 0, the better the alignment effect of the two frames of images, and the higher the estimation accuracy of the model for the road depth value and the road height value. The closer the photometric error value is to 1, the worse the alignment effect of the two frames of images, and the lower the estimation accuracy of the model for the road depth value and the road height value.

[0058] It should be noted that in the following embodiment steps in the present disclosure, the photometric error value of the non-road plane region after image alignment processing is taken as an example to explain the technical solutions in the present disclosure, but it does not constitute a specific limitation.

[0059] In step 240, the photometric error value is taken as a supervised loss term to perform estimation training of the road information in the road plane region of the initial road information estimation model, and the trained initial road information estimation model is determined as the target road information estimation model.

[0060] For the embodiment of the present application, when the initial road information estimation model is trained, in addition to the real depth value and the real height value corresponding to the continuous frame images as the supervised loss term, the photometric error value can also be added as the supervised loss term. By taking the photometric error value as the supervised loss term to perform estimation training of the road information in the road plane region of the initial road information estimation model, the estimated depth value and the estimated height value output by the initial road information estimation model can gradually approach the real depth value and the real height value, and the photometric error value of the non-road plane region can reach the minimum value.

[0061] In summary, according to the training method of the road information estimation model provided in the present application, after the continuous frame images containing the road plane area are obtained, the initial road information estimation model can be used to generate the depth image and the height image corresponding to the continuous frame images, and then the photometric error value of the non-road plane area after the image alignment processing is calculated according to the depth image and the height image. The photometric error value is used as the supervision loss term of the model training, and the initial road information estimation model is pre-estimated and trained for the road information in the road plane area to obtain the target road information estimation model after the training. The technical solution in the present application can calculate the photometric error value according to the depth image and the height image, and increase the supervision loss term of the photometric error value in the model training process, which can guide the model to comprehensively learn the features in the depth feature dimension and the height feature dimension of the image area when the road information is trained, so that the model can simultaneously estimate the depth value and the height value for the road condition. The multiple feature dimensions of the road information can accurately reflect the real road condition, thereby effectively ensuring the safety of driving.

[0062] Based on Figure 2 As a refinement and extension of the above-mentioned embodiments, in order to fully describe the specific implementation process of the method of the present embodiment, the present embodiment provides a specific method as shown in Figure 4 Figure 4 Based on Figure 2 The steps 220 and 230 are further defined in the embodiment shown in Figure 2 In the embodiment shown in

[0063] As Figure 4 The method comprises the following steps:

[0064] Step 310, acquiring continuous frame images containing road plane areas and non-road plane areas.

[0065] For the specific implementation process of the present embodiment, reference can be made to the related description in the embodiment step 210, which will not be repeated here.

[0066] Step 320, determining a reference frame image and an alignment frame image in the continuous frame images, taking the road plane as the homography plane, and generating a homography matrix between the reference frame image and the alignment frame image.

[0067] The reference frame image is a frame of reference image used to provide the position of the aligned pixel points, and the pixel points in the reference frame image do not need to be subjected to homographic alignment processing. The alignment frame image is a frame of image that needs to be subjected to homographic alignment with reference to the reference frame image, that is, the road plane pixel points in the alignment frame image need to be aligned with the corresponding road plane pixel points in the reference frame image. ​

[0068] For the embodiments of the present application, when determining the reference frame image and the alignment frame image in the continuous frame images, as a possible implementation manner, any one of the continuous frame images can be determined as the reference frame image, and the remaining one of the continuous frame images can be determined as the alignment frame image; as a possible implementation manner, in order to ensure the training accuracy of the initial road information estimation model, one of the continuous frame images with a larger degree of clarity or a degree of clarity greater than a preset threshold can be determined as the reference frame image, and the preset threshold can be preset according to actual needs, and the purpose is to select one of the frame images with the largest degree of clarity as the reference frame image. Correspondingly, the alignment frame image is one of the continuous frame images with a poor degree of clarity or a degree of clarity less than or equal to the preset threshold.

[0069] For the embodiments of the present application, after the reference frame image and the alignment frame image are determined, the homography matrix between the reference frame image and the alignment frame image can be generated by a simple homography network taking the road plane as the homography plane. The homography network used to generate the homography matrix can be a deep learning network (such as ResNet34 or ResNet50, etc.), or a convolutional neural network with network layer modification (such as adding several convolutional layers) to the existing deep learning network, which is not specifically limited here.

[0070] Step 330, based on the homography matrix and the reference frame image, performing homography alignment processing on the road plane region of the alignment frame image to obtain a first image.

[0071] For the embodiments of the present application, a plurality of road plane pixel points in the alignment frame image corresponding to the road plane region can be determined, and then the pixel coordinate position of each road plane pixel point in the alignment frame image is transformed based on the homography matrix, so that the transformed pixel position is the same as the pixel position of the same road plane pixel point in the reference frame image, that is, the homography alignment of the road plane pixel point is completed. After the alignment of the plurality of road plane pixel points is completed, the first image of the alignment frame image after the homography alignment can be obtained.

[0072] For example, in the road plane region of the aligned frame image, there are multiple road plane pixels: road plane pixel 1, road plane pixel 2, road plane pixel 3, road plane pixel 4, and road plane pixel 5, and the pixel positions corresponding to the multiple road plane pixels are a, b, c, d, and e, respectively. The pixel positions corresponding to the multiple road plane pixels in the reference frame image are A, B, C, D, and E, respectively. When performing alignment transformation on the pixel coordinate positions of each road plane pixel in the aligned frame image based on the homography matrix, the pixel position corresponding to each road plane pixel can be adjusted to the pixel position corresponding to the same road plane pixel in the reference frame image. For example, the pixel position of road plane pixel 1 is adjusted from a to A; the pixel position of road plane pixel 2 is adjusted from b to B; the pixel position of road plane pixel 3 is adjusted from c to C; the pixel position of road plane pixel 4 is adjusted from d to D; and the pixel position of road plane pixel 5 is adjusted from e to E. After the alignment adjustment of all road plane pixels in the road plane region is completed, the first image of the aligned frame image after homographic alignment can be obtained. In the first image, the pixels in the non-road plane region remain unchanged, and the pixel positions of the road plane pixels in the road plane region are the pixel positions after homographic alignment.

[0073] In step 340, the reference frame image and the first image are input into an initial road information estimation model, and the initial road information estimation model is used to generate a depth image and an elevation image corresponding to the continuous frame image.

[0074] In a specific application scenario, the input features of the initial road information estimation model are two continuous frames of images, one of which is an original image (i.e., the reference frame image), and the other is an image deformed by the homographic network (i.e., the first image). The output features of the initial road information estimation model are a depth map (i.e., the depth image) and an elevation map (i.e., the height image). The role is to learn the mapping relationship between the input data (two continuous frames of images) and the output results (two maps), so as to automatically learn and extract the depth features and elevation features in the data, thereby estimating the three-dimensional structure of the road environment and obtaining the road information.

[0075] Specifically, the initial road information estimation model can include a feature extraction module, a cross-attention module, and a feature fusion module. The feature extraction module extracts features from two input images with shared weights, then inputs the extracted features into the cross-attention module, which can more effectively establish key clues between features and more easily find matching relationships between two input images, and finally applies a feature fusion module to predict the depth map and the elevation map.

[0076] Correspondingly, for the embodiments of the present application, the embodiment steps can comprise: inputting the reference frame image and the first image into a feature extraction module to extract first image features of the reference frame image and second image features of the first image; inputting the first image features and the second image features into a cross attention module to obtain a geometric matching relationship between image pairs contained in the reference frame image and the first image; and inputting the geometric matching relationship into a feature fusion module to obtain a depth image and an altitude image corresponding to the continuous frame image.

[0077] Step 350, based on the depth image and the altitude image, calculating a ratio of a depth value and a height value corresponding to each pixel point.

[0078] For the embodiments of the present application, the purpose of calculating the ratio of the depth value and the height value corresponding to each pixel point is to calculate the residual value of the reference frame image and the first image after the homography alignment processing, so as to facilitate the alignment of other static regions (i.e. non-road plane regions) in the first image except the road plane region based on the residual value.

[0079] For the embodiments of the present application, for example, if the size of the depth map and the height map are both 640*320, then for each pixel point on the depth map, the ratio of the depth value and the height value of the corresponding pixel point on the height map needs to be calculated. The ratio of the depth value and the height value corresponding to a single pixel point is calculated as follows: R(x, y) = d(x, y) / h(x, y), where (x, y) is the pixel position coordinate of a certain pixel point, R(x, y) is the ratio of the depth value and the height value corresponding to the pixel point, d(x, y) is the depth value of the pixel point in the depth image, and h(x, y) is the height value of the pixel point in the altitude image.

[0080] Step 360, performing residual alignment processing on the non-road plane region of the first image according to the ratio of the depth value and the height value to obtain a second image.

[0081] In a specific application scenario, after calculating the ratio of the depth value and the height value corresponding to each pixel point, the residual value can be calculated through traditional 3D geometric transformation, and this residual value represents the displacement of the corresponding pixels between the pair of images (the reference frame image and the first image) that have been aligned by the road homography. Further, the non-road plane region except the road plane can be aligned based on the residual value to obtain a second image, and the second image is a reconstructed image well aligned with the reference frame image with respect to the non-road plane region. The residual value can be a positive value or a negative value.

[0082] For the embodiments of the present application, when aligning the non-road plane area other than the road plane based on the residual value, each non-road plane pixel point on the non-road plane area can be accumulated with the residual value corresponding to the non-road plane pixel point at the corresponding pixel position, to obtain the pixel position of the non-road plane pixel point after the residual alignment processing. After completing the residual alignment processing of all non-road plane pixel points on the non-road plane area, the second image after the residual alignment processing of the first image on the non-road plane area can be obtained.

[0083] Correspondingly, the embodiment steps can include: calculating a residual value based on the ratio of the depth value and the height value, the residual value being used to represent the displacement deviation of the corresponding pixel points between the reference frame image and the first image; and accumulating the residual value at the corresponding pixel position for each non-road plane pixel point on the non-road plane area of the first image, to obtain the second image.

[0084] Step 370, calculating the photometric error value after the residual alignment processing based on the non-road plane pixel points in the second image and the reference frame image.

[0085] In a specific application scenario, the reconstructed second image and the reference frame image are well aligned, and then the pixel intensity of the corresponding pixels can be used to calculate the photometric error. For example, assuming that there are two images, one is a predicted output image (i.e., the second image reconstructed by combining the residual and the first image mentioned above), denoted as I_pred, and the other is a real observation image (i.e., the real input reference frame image), denoted as I_gt. When calculating the photometric error value, the difference between the corresponding pixel point information of the predicted output image I_pred and the real observation image I_gt can be calculated. Different measurement methods can be used, such as Mean Squared Error (MSE) or Mean Absolute Error (MAE).

[0086] Correspondingly, for the embodiments of the present application, the embodiment steps can include: determining two non-road plane pixel points corresponding to the same pixel position in the second image and the reference frame image; extracting the pixel point information of the two non-road plane pixel points; and calculating the pixel feature difference value between the two non-road plane pixel points based on the pixel point information, and determining the pixel feature difference value as the photometric error value. The pixel point information can include but is not limited to pixel value, position, color, brightness, etc., and is not limited here.

[0087] Step 380, using the photometric error value as a supervised loss term to perform pre-estimation training of the initial road information estimation model on the road information in the road plane area, and determining the trained initial road information estimation model as the target road information estimation model.

[0088] In specific application scenarios, the training process of the initial road information estimation model usually includes the following key steps:

[0089] 1. Data preparation of consecutive frame images collected by the front-view camera: divided into training dataset and validation dataset. The training dataset is used to train the model, and the validation dataset is used to evaluate the model performance and perform parameter tuning.

[0090] 2. Network construction: define the model network structure as above, such as the initial road information estimation model can include feature extraction module, cross attention module and feature fusion module.

[0091] 3. Loss function definition: select an appropriate loss function to measure the difference between the model's predicted output and the true value.

[0092] 4. Parameter initialization: randomly initialize the parameters of the model for subsequent optimization.

[0093] 5. Forward propagation: input data (consecutive two frames of images) is passed through the network for forward propagation to obtain the model's prediction results.

[0094] 6. Backpropagation: according to the value of the loss function, calculate the gradient (derivative of the loss function with respect to the model parameters) through the backpropagation algorithm.

[0095] 7. Parameter optimization: use optimization algorithms (such as stochastic gradient descent, Adam, etc.) to update the model's parameters according to the gradient, so that the loss function gradually decreases.

[0096] 8. Iterative training: repeat the process of forward propagation, backpropagation and parameter optimization until the predefined stopping condition is reached (such as reaching a certain number of training rounds or the loss function converges).

[0097] 9. Model evaluation: use the validation dataset to evaluate the performance of the trained model and see how well it generalizes to new data.

[0098] 10. Prediction and application: use the trained model for practical application and make predictions on new input data.

[0099] 11. Through continuous iterative training process, the network model can gradually optimize and improve performance, so that it can better adapt to input data and make accurate predictions and judgments. It is worth noting that the training process of the model needs to be carried out on appropriate hardware devices and sufficient computing resources to ensure the effect and speed of training.

[0100] Correspondingly, for the embodiments of the present application, the embodiment steps can comprise: taking the continuous frame images as input features, taking the real depth values, real height values and photometric error values corresponding to the continuous frame images as supervised loss terms, repeatedly performing the training process of the initial road information estimation model until the initial road information estimation model reaches a preset condition, and determining the initial road information estimation model as the target road information estimation model. The preset condition can be that the training repetition number reaches a certain training round or the loss function reaches a convergence state, and the like, which is not specifically limited here.

[0101] The repeatedly performed training process of the initial road information estimation model can comprise: inputting the continuous frame images into the initial road information estimation model through forward propagation, performing pre-estimation training of the road information in the road plane region of the initial road information estimation model, obtaining the estimated depth values and estimated height values; calculating the gradient error values of the initial road information estimation model according to the estimated depth values, the estimated height values, and the real depth values and real height values corresponding to the continuous frame images; calculating the loss function values of the initial road information estimation model based on the photometric error values and the gradient error values; calculating the model gradient corresponding to the loss function values based on the back propagation algorithm; and updating the model parameters of the initial road information estimation model according to the model gradient.

[0102] In summary, the technical solution in the present application adds the photometric error value of the non-road plane region as a supervised loss term in the model training process, which can guide the model to comprehensively learn the features in the depth feature dimension and the height feature dimension of the image region when performing road information training, so that the model can simultaneously estimate the depth values and height values of the road. The multiple feature dimensions of the road information can accurately reflect the real road conditions, thereby effectively ensuring the safety of driving. In addition, by fusing the depth information estimation and the height information estimation into the same model training task in the model training process, the target road information estimation model can output road information containing depth values and height values, thereby enriching the estimated feature dimensions of the road information. The height values can better distinguish and identify the drivable region and distinguish the obstacles above the road, such as pedestrians, to meet the safety driving needs of intelligent vehicles. In addition, fusing the depth and height information estimation into the same deep neural network can save memory resources and improve the computing performance.

[0103] Figure 5 A flowchart of a road information extraction method according to an embodiment of the present application is provided, which can be executed by the electronic device 110 as shown in Figure 1 but is not limited thereto. As shown in Figure 5 , the method can comprise the following steps:

[0104] Step 410: determining a target continuous frame image of road information to be extracted.

[0105] The target continuous frame image is captured in real time by a forward-looking camera, resulting in two consecutive frames. The target continuous frame image may include road plane areas and non-road plane areas. The road plane area may include road objects such as obstacles, pedestrians, vehicles, road markings, cracks, potholes, ruts, and speed bumps; the non-road plane area may include three-dimensional objects such as clouds, flying objects, trees, buildings, signs, and indicator lights.

[0106] In specific application scenarios, to ensure the accuracy of the target road information estimation model's prediction of target road information and avoid prediction errors caused by image distortion, the target continuous frame images can also be corrected images of two consecutive frames acquired in real time by the forward-looking camera. Image correction refers to the restorative processing of distorted images. Causes of image distortion include: image distortion caused by aberrations, distortions, and limited bandwidth of the imaging system; geometric distortion caused by the imaging device's shooting posture and scanning nonlinearity; and image distortion caused by motion blur, radiation distortion, and introduced noise. The basic idea of ​​image correction is to establish a corresponding mathematical model based on the cause of image distortion, extract the necessary information from the contaminated or distorted image signal, and restore the original image by following the reverse process that caused the image distortion.

[0107] Step 420: Input the target continuous frame images into the target road information estimation model to obtain the target road information, which includes at least the target depth value and target height value of the road object.

[0108] The target road information estimation model is trained by using the photometric error value as a supervised loss term and performing multiple predictions of road information within the road plane area on the initial road information estimation model. The specific training process can be found in the relevant descriptions in steps 210 to 240 or 310 to 380 of the embodiment, and will not be repeated here.

[0109] The target road information estimation model may include a feature extraction module, a cross-attention module, and a feature fusion module. In this embodiment, after inputting consecutive target frame images into the target road information estimation model, the feature extraction module extracts features from the consecutive target frame images. These extracted features are then input into the cross-attention module, which more effectively establishes key clues between features and finds matching relationships between two input images. Finally, the feature fusion module is applied to predict the depth map and elevation map, or directly outputs the predicted target road information. The depth map may contain the target depth value of the road object, and the elevation map may contain the target height value of the road object.

[0110] In summary, the target road information estimation model is input with the target continuous frame image, and the target road information including at least the target depth value and the target height value is output by the target road information estimation model. Since the target road information estimation model is trained by the photometric error value of the non-road plane area as the supervised loss term, the target road information estimation model can eliminate the influence of the three-dimensional geometric information outside the road plane area on the road information estimation when estimating the target road information, and thus can ensure the accuracy of the target road information.

[0111] Based on the above Figure 2 、 Figure 3 The specific description of the training method of the road information estimation model provided is as shown in Figure 6 , Figure 6 is a block diagram of a training device of a road information estimation model according to an example embodiment. As shown in Figure 6 , the device comprises:

[0112] The acquisition module 51 can be used to acquire continuous frame images containing road plane areas and non-road plane areas;

[0113] The generation module 52 can be used to generate depth images and height images corresponding to the continuous frame images by using the initial road information estimation model;

[0114] The calculation module 53 can be used to calculate the photometric error value of the non-road plane area after the image alignment processing according to the depth images and the height images;

[0115] The training module 54 can be used to take the photometric error value as a supervised loss term to perform the estimation training of the road information in the road plane area on the initial road information estimation model, and determine the trained initial road information estimation model as the target road information estimation model.

[0116] In some embodiments of the present application, the generation module 52 can be used to determine a reference frame image and an alignment frame image in the continuous frame image; take the road plane as a homography plane to generate a homography matrix between the reference frame image and the alignment frame image; perform homography alignment processing of the road plane area on the alignment frame image based on the homography matrix and the reference frame image to obtain a first image; input the reference frame image and the first image into the initial road information estimation model to generate the depth images and the height images corresponding to the continuous frame images by using the initial road information estimation model.

[0117] In some embodiments of the present application, the initial road information estimation model comprises a feature extraction module, a cross-attention module and a feature fusion module. The generation module 52 can be configured to input the reference frame image and the first image into the feature extraction module to extract first image features of the reference frame image and second image features of the first image; input the first image features and the second image features into the cross-attention module to obtain a geometric matching relationship between image pairs contained in the reference frame image and the first image; and input the geometric matching relationship into the feature fusion module to obtain a depth image and an altitude image corresponding to the continuous frame image.

[0118] In some embodiments of the present application, the depth image contains a depth value of each pixel point, and the altitude image contains an altitude value of each pixel point. The calculation module 53 can be configured to calculate a ratio of the depth value to the altitude value of each pixel point based on the depth image and the altitude image; perform residual alignment processing on a non-road plane region of the first image according to the ratio of the depth value to the altitude value to obtain a second image; and calculate a photometric error value after the residual alignment processing based on non-road plane pixel points in the second image and the reference frame image.

[0119] In some embodiments of the present application, the calculation module 53 can be configured to calculate a residual value based on the ratio of the depth value to the altitude value, and the residual value is used to represent a displacement deviation of corresponding pixel points between the reference frame image and the first image; and for each non-road plane pixel point on the non-road plane region of the first image, accumulate the residual value at the corresponding pixel position to obtain the second image.

[0120] In some embodiments of the present application, the calculation module 53 can be configured to determine two non-road plane pixel points corresponding to the same pixel position of the second image and the reference frame image; extract pixel point information of the two non-road plane pixel points; and calculate a pixel feature difference value between the two non-road plane pixel points based on the pixel point information, and determine the pixel feature difference value as the photometric error value.

[0121] In some embodiments of the present application, the training module 54 can be configured to repeatedly execute a training process of the initial road information estimation model by taking the continuous frame image as an input feature and taking a real depth value, a real altitude value and a photometric error value corresponding to the continuous frame image as a supervised loss term until the initial road information estimation model reaches a preset condition, and determine the initial road information estimation model as a target road information estimation model.

[0122] In some embodiments of the present application, the training process of the initial road information estimation model comprises: inputting the continuous frame images into the initial road information estimation model through forward propagation, performing pre-estimation training of the initial road information estimation model on road information in the road plane area, to obtain pre-estimated depth values and pre-estimated height values; calculating gradient error values of the initial road information estimation model according to the pre-estimated depth values, the pre-estimated height values, and real depth values and real height values corresponding to the continuous frame images; calculating loss function values of the initial road information estimation model based on the photometric error values and the gradient error values; calculating model gradients corresponding to the loss function values based on a back propagation algorithm; and updating model parameters of the initial road information estimation model according to the model gradients.

[0123] As to the device in the above-mentioned embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments relating to the method, and thus will not be described in detail here.

[0124] In the embodiments of the present application, the addition of the photometric error value of the non-road plane area as a supervision loss term in the model training process can guide the model to comprehensively learn the features of various image areas when performing road information pre-estimation, eliminate the influence of three-dimensional geometric information outside the road plane area on road information estimation, and improve the pre-estimation accuracy of road information.

[0125] Based on the above Figure 5 The specific description of the road information extraction method provided herein is as follows: after the target road information estimation model is trained, the use of the target road information estimation model can refer to the use of a normal deep learning model, and specifically, a road information extraction device is provided, as shown in Figure 7 The device comprises:

[0126] The determining module 61 can be configured to determine a target continuous frame image of road information to be extracted.

[0127] The input module 62 can be configured to input the target continuous frame image into the target road information estimation model to obtain target road information, the target road information at least comprising target depth values and target height values of road images.

[0128] As to the device in the above-mentioned embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments relating to the method, and thus will not be described in detail here.

[0129] The embodiment of the present application inputs a target continuous frame image into a target road information estimation model, and uses the target road information estimation model to output target road information including at least a target depth value and a target height value. Since the target road information estimation model is trained by using the photometric error value of the non-road plane area as a supervision loss term, when estimating the target road information, the target road information estimation model can eliminate the influence of the three-dimensional geometric information outside the road plane area on the road information estimation, so that the model can simultaneously estimate the road depth value and the height value, and the feature dimension of multiple road information can accurately reflect the real road conditions, thereby effectively ensuring the safety of driving.

[0130] The training device of the road information estimation model and the road information extraction device of the embodiment of the present application are described above from the perspective of functional modules in combination with the drawings. It should be understood that the functional modules can be realized in the form of hardware, or realized in the form of instructions of software, or realized in the form of a combination of hardware and software modules. Specifically, each step of the training method embodiment of the road information estimation model in the embodiment of the present application can be completed by the integrated logic circuit of hardware and / or the instruction of software in the processor. The steps of the training method of the road information estimation model in the embodiment of the present application can be directly embodied as hardware decoding processor execution completion, or executed by a combination of hardware and software modules in the decoding processor. Alternatively, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps in the above training method embodiment of the road information estimation model.

[0131] Figure 8 is a schematic block diagram of an electronic device 800 according to an embodiment of the present application.

[0132] As shown in Figure 8 , the electronic device 800 can include:

[0133] The memory 810 is configured to store a computer program and transmit the program code to the processor 820. In other words, the processor 820 can call and run the computer program from the memory 810 to implement the method in the embodiment of the present application.

[0134] For example, the processor 820 can be configured to execute the above method embodiments according to the instructions in the computer program.

[0135] In some embodiments of the present application, the electronic device 820 can include but is not limited to:

[0136] A general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

[0137] In some embodiments of the present application, the memory 810 includes, but is not limited to:

[0138] volatile memory and / or non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), which is used as the external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).

[0139] In some embodiments of the present application, the computer program can be divided into one or more modules, which are stored in the memory 810 and executed by the processor 820 to complete the method provided by the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the controller.

[0140] As shown in Figure 8 The electronic device 800 can further include:

[0141] A transceiver 830 can be connected to the processor 820 or the memory 810.

[0142] The processor 820 can control the transceiver 830 to communicate with other devices, specifically, can send information or data to other devices, or receive information or data sent by other devices. The transceiver 930 can include a transmitter and a receiver. The transceiver 830 can further include an antenna, and the number of antennas can be one or more.

[0143] It should be understood that various components in the electronic device are connected through a bus system, which includes a data bus, a power supply bus, a control bus, and a status signal bus, in addition to a data bus.

[0144] The present application also provides a computer storage medium, which stores a computer program, and the computer program enables a computer to execute the method of the above-mentioned method embodiment when executed by the computer. Alternatively, one embodiment of the present application also provides a computer program product containing instructions, which enables a computer to execute the method of the above-mentioned method embodiment when executed by the computer.

[0145] When implemented by using software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When loaded and executed by a computer, the computer program instructions produce the flow or function of the embodiment of the present application in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, digital video disc (DVD)), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

[0146] Those skilled in the art can understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0147] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0148] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiments according to actual needs. For example, the functional modules in each embodiment of the present application can be integrated in one processing module, or each module can be physically present separately, or two or more modules can be integrated in one module.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A training method for a road information estimation model, characterized in that, include: Acquire consecutive frame images containing both road surface areas and non-road surface areas; Using the initial road information estimation model, depth and height images corresponding to the consecutive frame images are generated; Based on the depth image and the height image, calculate the photometric error value of the non-road plane region after image alignment processing; The photometric error value is used as a supervised loss term to train the initial road information estimation model to predict road information within the road plane area, and the trained initial road information estimation model is determined as the target road information estimation model. The step of generating depth and height images corresponding to the consecutive frame images using the initial road information estimation model includes: A reference frame image and an alignment frame image are determined from the consecutive frame images; Using the road plane as the homography plane, a homography matrix is ​​generated between the reference frame image and the aligned frame image; Based on the homography matrix and the reference frame image, the alignment frame image is subjected to homography alignment processing of the road plane region to obtain the first image; The reference frame image and the first image are input into the initial road information estimation model, and the initial road information estimation model is used to generate the depth image and height image corresponding to the consecutive frame images; The depth image contains the depth value of each pixel, and the height image contains the height value of each pixel. The step of calculating the photometric error value of the non-road plane region after image alignment processing based on the depth image and the height image includes: Based on the depth image and the height image, calculate the ratio of the depth value to the height value for each pixel; Based on the ratio of the depth value to the height value, the first image is subjected to residual alignment processing for non-road plane areas to obtain the second image; Based on the non-road plane pixels in the second image and the reference frame image, the photometric error value after the residual alignment process is calculated.

2. The method according to claim 1, characterized in that, The initial road information estimation model includes a feature extraction module, a cross-attention module, and a feature fusion module. The step of inputting the reference frame image and the first image into the initial road information estimation model, and using the initial road information estimation model to generate depth and height images corresponding to the consecutive frame images, includes: The reference frame image and the first image are input to the feature extraction module to extract the first image feature of the reference frame image and the second image feature of the first image; The first image features and the second image features are input into the cross-attention module to obtain the geometric matching relationship between the reference frame image and the image objects contained in the first image; The geometric matching relationship is input into the feature fusion module to obtain the depth image and height image corresponding to the consecutive frame images.

3. The method according to claim 1, characterized in that, The step of performing residual alignment processing on the first image for non-road plane regions based on the ratio of the depth value to the height value to obtain the second image includes: The residual value is calculated based on the ratio of the depth value to the height value, and the residual value is used to represent the displacement deviation of corresponding pixels between the reference frame image and the first image. For each non-road plane pixel in the non-road plane region of the first image, the residual value is accumulated at the corresponding pixel position to obtain the second image.

4. The method according to claim 1, characterized in that, The step of calculating the photometric error value after residual alignment processing based on the non-road plane pixels in the second image and the reference frame image includes: Identify two non-road plane pixels at the same pixel location in the second image and the reference frame image; Extract the pixel information of the two non-road plane pixels; Based on the pixel information, the pixel feature difference value between the two non-road plane pixels is calculated, and the pixel feature difference value is determined as the photometric error value.

5. The method according to claim 1, characterized in that, The step of using the photometric error value as a supervised loss term to train the initial road information estimation model for predicting road information within the road plane area, and determining the trained initial road information estimation model as the target road information estimation model, includes: Using the continuous frame images as input features, and the true depth value, true height value, and photometric error value corresponding to the continuous frame images as supervised loss terms, the training process of the initial road information estimation model is repeatedly executed until the initial road information estimation model meets the preset conditions, and the initial road information estimation model is determined as the target road information estimation model.

6. The method according to claim 5, characterized in that, The training process of the initial road information estimation model includes: The continuous frame images are input into the initial road information estimation model through forward propagation. The initial road information estimation model is trained to predict road information within the road plane area to obtain the predicted depth and predicted height values. The gradient error of the initial road information estimation model is calculated based on the estimated depth value, the estimated height value, and the actual depth value and actual height value corresponding to the consecutive frame images. Based on the photometric error value and the gradient error value, calculate the loss function value of the initial road information estimation model; Based on the backpropagation algorithm, the model gradient corresponding to the loss function value is calculated; The model parameters of the initial road information estimation model are updated based on the model gradient.

7. A method for extracting road information, characterized in that, include: Identify the target consecutive frame images from which road information is to be extracted; The target continuous frame image is input into the target road information estimation model according to any one of claims 1 to 6 to obtain target road information, wherein the target road information includes at least the target depth value and the target height value of the road object.

8. A training device for a road information estimation model, characterized in that, include: The acquisition module is used to acquire continuous frame images that include both road surface areas and non-road surface areas; The generation module is used to generate depth and height images corresponding to the consecutive frame images using the initial road information estimation model; The calculation module is used to calculate the photometric error value of the non-road plane area after image alignment processing based on the depth image and the height image; The training module is used to use the photometric error value as a supervised loss term to train the initial road information estimation model to predict road information in the road plane area, and to determine the trained initial road information estimation model as the target road information estimation model. The generation module is configured to: determine a reference frame image and an aligned frame image in the continuous frame images; generate a homography matrix between the reference frame image and the aligned frame image, using the road plane as the homography plane; perform homography alignment processing on the aligned frame image for the road plane region based on the homography matrix and the reference frame image to obtain a first image; input the reference frame image and the first image into an initial road information estimation model, and use the initial road information estimation model to generate a depth image and a height image corresponding to the continuous frame images; The depth image contains the depth value of each pixel, the height image contains the height value of each pixel, and the calculation module includes: Based on the depth image and the height image, calculate the ratio of the depth value to the height value for each pixel; Based on the ratio of the depth value to the height value, the first image is subjected to residual alignment processing for non-road plane areas to obtain the second image; Based on the non-road plane pixels in the second image and the reference frame image, the photometric error value after the residual alignment process is calculated.

9. A road information extraction device, characterized in that, include: The determination module is used to determine the target continuous frame images from which road information is to be extracted; An input module is used to input the target continuous frame image into the target road information estimation model according to any one of claims 1 to 6 to obtain target road information, wherein the target road information includes at least the target depth value and the target height value of the road object.

10. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to perform a training method for a road information estimation model as described in any one of claims 1-6 or a road information extraction method as described in claim 7.

11. A computer-readable storage medium, characterized in that, Used to store computer programs that cause a computer to perform a training method for a road information estimation model as described in any one of claims 1-6 or a road information extraction method as described in claim 7.

Images