Accumulated water depth determination method and device, electronic equipment and storage medium

Abstract

The invention discloses a ponding depth determination method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target image of a to-be-detected ponding position; respectively determining the matching degree between the target image and each sample image in the sample set, and taking the sample image with the highest matching degree as a target sample; according to the feature matrix of the target image, the feature matrix of the target sample and a preset depth prediction model, the target ponding depth of the ponding position to be measured is determined, and the depth prediction model is a model used for determining the ponding depth and obtained after a sample set is used for training a convolutional neural network model. According to the method provided by the invention, the problem that a detection result has a relatively large error due to different shooting conditions or environmental factors in an actual shooting process can be reduced, and the accuracy and reliability of determining the depth of the accumulated water are improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a method, device, electronic device and storage medium for determining water depth. Background Art

[0002] During the driving process, the actual conditions of the road surface may cause accidents to the driver. For example, when the vehicle is passing, the road surface is flooded too deeply, and the driver does not know the depth of the water in time, thus causing a traffic accident.

[0003] Existing water accumulation detection usually relies on the monitoring screen of a fixed camera deployed above the road surface or the monitoring screen of the camera currently configured on the vehicle itself. However, if the monitoring near the water accumulation on the road is in a blind spot, or the camera currently configured on the vehicle itself cannot fully capture the entire water accumulation area, it will lead to estimation errors in the water accumulation depth detection. Therefore, it is urgent to propose a new method to solve the above problems. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for determining the depth of accumulated water. The present invention can reduce the problem of large errors in detection results caused by different shooting conditions or environmental factors during actual shooting, and improve the accuracy and reliability of determining the depth of accumulated water.

[0005] According to one aspect of the present invention, a method for determining water depth is provided, the method comprising:

[0006] Acquire a target image of the water accumulation position to be measured;

[0007] Determine the matching degree between the target image and each sample image in the sample set respectively, and take the sample image with the highest matching degree as the target sample;

[0008] The target water depth of the water accumulation position to be measured is determined according to the feature matrix of the target image, the feature matrix of the target sample and a preset depth prediction model, wherein the depth prediction model is a model for determining the water accumulation depth obtained by training a convolutional neural network model using a sample set.

[0009] According to another aspect of the present invention, there is provided a device for determining water depth, the device comprising:

[0010] An acquisition module is used to acquire a target image of a water accumulation position to be measured;

[0011] A selection module is used to determine the matching degree between the target image and each sample image in the sample set, and take the sample image with the highest matching degree as the target sample;

[0012] A determination module is used to determine the target water depth of the water accumulation position to be measured based on the feature matrix of the target image, the feature matrix of the target sample and a pre-set depth prediction model, wherein the depth prediction model is a model for determining the water depth obtained by training a convolutional neural network model using a sample set.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining the water accumulation depth described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining the water depth described in any embodiment of the present invention when executed.

[0018] The method for determining the water accumulation depth provided by the embodiment of the present invention obtains a target image of the water accumulation position to be measured; determines the matching degree between the target image and each sample image in the sample set respectively, and uses the sample image with the highest matching degree as the target sample; determines the target water accumulation depth of the water accumulation position to be measured according to the feature matrix of the target image, the feature matrix of the target sample and the pre-set depth prediction model. In the above technical scheme, on the one hand, by calculating the matching degree between the target image and each sample image in the sample set, the sample image with the highest matching degree with the target image is determined as the target sample, which provides an accurate image basis for determining the water accumulation depth of the water accumulation position to be measured according to the target sample and the target image. On the other hand, the target water accumulation depth is determined not only based on the feature matrix of the target image, but also based on the feature matrix of the target image and the feature matrix of the target sample with the highest matching degree with the target image, so that the target water accumulation depth of the water accumulation position to be measured is determined, and the image with the highest matching degree in the sample set is used as a reference image to determine the target water accumulation depth, although the shooting environment is different from that of the target image, so as to reduce the large calculation error caused by different shooting conditions or environmental factors in the actual shooting process when determining the target water accumulation depth only according to the target image, thereby improving the accuracy and reliability of the water accumulation depth determination.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of a flow chart of a method for determining water depth provided by an embodiment of the present invention;

[0022] Figure 2 Another schematic diagram of a flow chart of a method for determining water depth provided by an embodiment of the present invention;

[0023] Figure 3 An example flow chart of a method for training a depth prediction model provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the structure of a device for determining water depth provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variation of the terms "including" and "having" is intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Figure 1 A flow chart of a method for determining water depth provided in an embodiment of the present invention. This embodiment is applicable to situations where the depth of water on a road surface needs to be determined. The method can be executed by a water depth determination device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. In this embodiment, the electronic device can be a computer or a terminal device, and the electronic device can be set in a vehicle. Figure 1 As shown, the method includes:

[0029] S101, obtaining a target image of a water accumulation position to be measured.

[0030] The target image includes an image of a waterlogged area at a waterlogged location to be measured.

[0031] Specifically, the target image of the water accumulation position to be measured may be acquired by: taking the image with a camera carried by the vehicle itself during the driving process or by a surveillance camera installed on the roadside.

[0032] For example, if there are surveillance cameras installed on the roadside, the regulatory authorities with the relevant video acquisition rights can directly obtain the target image of the water accumulation location to be measured recorded by the surveillance camera. If there are few roadside cameras or the water accumulation location to be measured is in the blind spot of the surveillance camera, the on-board surveillance images of vehicles traveling near the water accumulation location to be measured can be obtained, and the target image of the water accumulation location to be measured can be obtained based on the on-board surveillance images.

[0033] Optionally, in this embodiment, the target images of the water accumulation position to be measured can be multiple or one. When multiple target images are obtained, the multiple target images can be processed to obtain a more complete or clearer target image containing the water accumulation area at the water accumulation position to be measured, so that the subsequent determination of the water accumulation depth is more accurate.

[0034] S102: Determine the matching degree between the target image and each sample image in the sample set respectively, and use the sample image with the highest matching degree as the target sample.

[0035] The sample set includes multiple sample images, each of which includes an image of a waterlogged area at a measured waterlogged location. The matching degree is the matching degree or similarity between the sample image and the target image.

[0036] For example, the sample images in the sample set are a large number of road surface water pictures and their corresponding real water depth data collected under different road and environmental conditions. These pictures are taken at similar angles, usually from the front. The water depth can be measured manually or using professional equipment.

[0037] Specifically, since the sample set includes multiple sample images, the sample image with the highest matching degree with the target image is determined from the sample set as the target sample. Therefore, for each sample image in the sample set, it is necessary to calculate the matching degree with the target image, and determine the matching degree of each sample image with the target image. And, after determining the matching degree of each sample image with the target image, select the sample image with the highest matching degree with the target image as the target sample.

[0038] Exemplarily, the calculation of the degree of matching can be determined by difference calculation, mean square error calculation or standard deviation calculation. For example, the mean square error between each sample image and the target image is calculated, and the sample image with the smallest mean square error value with the target image is selected as the target sample. The smallest mean square error value means the highest degree of matching.

[0039] In this embodiment, by calculating the matching degree between the target image and each sample image in the sample set, the sample image with the highest matching degree with the target image is determined as the target sample, which provides an accurate image basis for subsequently determining the water depth at the water accumulation location to be measured based on the target sample and the target image.

[0040] S103. Determine the target water depth of the water accumulation position to be measured according to the feature matrix of the target image, the feature matrix of the target sample and a pre-set depth prediction model, wherein the depth prediction model is a model for determining the water accumulation depth obtained by training a convolutional neural network model using a sample set.

[0041] The feature matrix of the image is determined by extracting features from pixels of the image using a feature extraction method. In this embodiment, the feature matrix of the image can be determined using a feature point detection and matching algorithm, such as scale-invariant feature transform (SIFT), ORB algorithm (Oriented FAST and rotated BRIEF) or local binary patterns (LBP). The target water depth is the depth of the water to be measured in the water position to be measured corresponding to the target image.

[0042] Specifically, in one implementation, the feature matrix of the target image and the feature matrix of the target sample are respectively input into the depth prediction model, and the depth prediction model determines the water depth of the water accumulation position to be measured in the target image according to the feature matrix of the target image, and determines the water depth of the water accumulation position to be measured in the target sample according to the feature matrix of the target sample, and then determines the target water accumulation depth of the water accumulation position to be measured based on the water accumulation depth of the water accumulation position to be measured in the target image and the water accumulation depth of the water accumulation position to be measured in the target sample. In another implementation, the feature matrix difference value between the feature matrix of the target image and the feature matrix of the target sample is first determined, and the feature matrix difference value is input into the depth prediction model to obtain the water accumulation depth difference value, and then the target water accumulation depth is determined based on the water accumulation depth difference value and the actual water accumulation depth of the water accumulation position in the target sample.

[0043] In this embodiment, the target water depth is determined not only based on the feature matrix of the target image, but also based on the feature matrix of the target image and the feature matrix of the target sample with the highest matching degree with the target image. This achieves the goal of determining the target water depth at the water accumulation position to be measured while using the image in the sample set that has the highest matching degree as a reference image although the shooting environment is different from that of the target image. This reduces the large calculation error caused by different shooting conditions or environmental factors during the actual shooting process when determining the target water depth based only on the target image, thereby improving the accuracy and reliability of the water depth determination.

[0044] The method for determining the water accumulation depth provided by the embodiment of the present invention obtains a target image of the water accumulation position to be measured; determines the matching degree between the target image and each sample image in the sample set respectively, and uses the sample image with the highest matching degree as the target sample; determines the target water accumulation depth of the water accumulation position to be measured according to the feature matrix of the target image, the feature matrix of the target sample and the pre-set depth prediction model. In the above technical scheme, on the one hand, by calculating the matching degree between the target image and each sample image in the sample set, the sample image with the highest matching degree with the target image is determined as the target sample, which provides an accurate image basis for determining the water accumulation depth of the water accumulation position to be measured according to the target sample and the target image. On the other hand, the target water accumulation depth is determined not only based on the feature matrix of the target image, but also based on the feature matrix of the target image and the feature matrix of the target sample with the highest matching degree with the target image, so that the target water accumulation depth of the water accumulation position to be measured is determined, and the image with the highest matching degree in the sample set is used as a reference image to determine the target water accumulation depth, although the shooting environment is different from that of the target image, so as to reduce the large calculation error caused by different shooting conditions or environmental factors in the actual shooting process when determining the target water accumulation depth only according to the target image, thereby improving the accuracy and reliability of the water accumulation depth determination.

[0045] Figure 2 Another flow chart of the method for determining the depth of water accumulation provided in an embodiment of the present invention. Based on the above embodiments, this embodiment describes in detail the steps of obtaining a target image of the water accumulation position to be measured, the steps after obtaining the target image of the water accumulation position to be measured, the step of determining the degree of matching between the target image and the sample image, and the step of determining the target water accumulation depth of the water accumulation position to be measured based on the feature matrix of the target image, the feature matrix of the target sample and a pre-set depth prediction model.

[0046] Before describing in detail the method for determining the water depth, a training method for the depth prediction model provided in an embodiment of the present invention is described in detail. Figure 3 FIG. 1 is a flowchart of a method for training a depth prediction model provided by an embodiment of the present invention. Figure 3 As shown, the method includes:

[0047] S301: Obtain a sample set.

[0048] Specifically, road surface waterlogging images collected from different roads and in different environments, as well as the actual water depth value corresponding to each road surface waterlogging image, are collected and used as samples in the sample set.

[0049] Exemplarily, collect pictures of road waterlogging taken by surveillance cameras installed by relevant departments on the road, pictures of road waterlogging taken by merchants, or pictures of road waterlogging taken by cameras carried by traveling vehicles themselves. Alternatively, the experimenters take pictures of road waterlogging by themselves, and ensure that the shooting angles are consistent during shooting. For example, take pictures of road waterlogging from the front. Screen the collected pictures, and select pictures of road waterlogging with shooting angles that are as consistent as possible. After the road waterlogging pictures are screened, use manual measurement or professional equipment to measure the water depth at the waterlogging location corresponding to each road waterlogging picture, and determine the actual water depth value corresponding to each road waterlogging picture. Use the road waterlogging pictures and their corresponding actual water depth values ​​as samples in the sample set.

[0050] For example, in addition to using the road surface waterlogging images and their corresponding real waterlogging depth values ​​directly as samples in the sample set, the road surface waterlogging images can also be processed for feature construction. For example, taking the pixel value of each road surface waterlogging image as a reference, taking into account the features such as the clarity, brightness and edge density of each road surface waterlogging image, the pixel value of each road surface waterlogging image is reconstructed. The feature construction method is as follows:

[0051] (1) For each road surface waterlogging image, the feature construction method of its clarity is to solve the variance of Laplace transform and transform the pixels of the image, so as to improve the clarity of the road surface waterlogging image.

[0052] (2) For each road surface waterlogging image, the characteristic construction method of its brightness distribution is to process the pixels of the image by using the method of calculating the brightness standard deviation, so as to average the brightness of each area in the road surface waterlogging image. For example, the brightness standard deviation is calculated as follows:

[0053]

[0054] Among them, brightness mean is the average brightness, N is the total number of pixels in image I, i is the pixel of the i-th pixel in image I, brightness std is the brightness standard deviation.

[0055] (3) For each road surface waterlogging image, the feature construction method of its edge density is to process the pixels of the image by dividing the sum of the edge pixels by the area of ​​the area, so as to quantify the edges of important areas in the road surface waterlogging image, for example, to make the edges of the waterlogged area more obvious. The feature construction method of edge density can be calculated as follows:

[0056]

[0057] Among them, densityedge is the edge density, ∑Canny(I) is the value obtained by summing the edge pixels in image I using the Canny edge detection algorithm, width is the pixel width of image I, and height is the pixel height / length of image I.

[0058] (4) For each road surface waterlogging picture, at least one of the above three is selected to process the image pixel value according to the situation of the picture itself and the factors required by the user to obtain the pixel after the image pixel is reconstructed. The pixel value reconstruction formula can be:

[0059] point=αpoint_pre+βlaplacian_var+γbrightness std +δdensity edge ;

[0060] Among them, point is the image pixel after pixel value reconstruction, point_pre is the original pixel value of image I before pixel value reconstruction, laplacian_var is the change in pixel value of the image after solving the variance of Laplace transform, α, β, γ and δ are hyperparameters used to fine-tune the corresponding clarity, brightness distribution, edge density and original image pixels. Optionally, its value can be 0.5, 0.1, etc. For example, if the shooting environment is at night and under backlight conditions, γ can be increased.

[0061] After the above-mentioned feature construction is performed on the road surface water image, it can be used as a sample in the sample set.

[0062] In this embodiment, a large number of road surface water images collected under different roads and different environmental conditions are obtained so that the samples in the obtained sample set have diversity, thereby ensuring the generalization ability of the model trained based on these samples.

[0063] S302: Input the feature matrix of the current sample image into the convolutional neural network model to determine the predicted water depth corresponding to the current sample image.

[0064] The convolutional neural network model is an original model trained by a feedforward neural network with a deep structure that includes convolution calculations. The predicted water depth is the water depth data output by the convolutional neural network model. The feature matrix is ​​the pixel features corresponding to the sample image, expressed in matrix form.

[0065] Specifically, the feature matrix corresponding to the sample image in the acquired sample image set is input into the convolutional neural network model, and the predicted water depth corresponding to the sample image is output through the learning of the convolutional neural network model.

[0066] S303, determining whether the difference between the predicted water depth corresponding to the current sample image and the standard water depth corresponding to the current sample image is less than or equal to a preset threshold; if so, executing S305; if not, executing S304.

[0067] The standard water depth is the actual water depth value corresponding to the sample image. The preset threshold is a training end mark of the model training set by the user, such as a loss function. Optionally, the loss function can be a mean square error, and the loss function is minimized by an optimization algorithm.

[0068] Specifically, for each sample image, it is necessary to determine whether the difference between the predicted water depth output by the model for the sample image and the standard water depth of the sample image is less than or equal to the preset threshold. If the difference between the predicted water depth and the standard water depth is less than or equal to the preset threshold, it means that the current model training results meet the user's needs and the model training can be ended. At this point, the convolutional neural network model can be determined as a depth prediction model. If the difference between the predicted water depth and the standard water depth is greater than the preset threshold, it means that the training of the convolutional neural network model has not met the user's needs. Therefore, it is necessary to continue to adjust the parameters of the convolutional neural network model, and continue to use the next sample image as the current sample image, and continue to input the current sample image into the model and train the model.

[0069] S304, adjusting the parameters of the convolutional neural network model, and taking the next sample image as the current sample image; returning to execute S302.

[0070] Specifically, when it is determined that the difference between the predicted water accumulation depth and the standard water accumulation depth is greater than a preset threshold, the parameters of the convolutional neural network model can be adjusted. After the parameters are adjusted, the next sample image is used as the current sample image, and the step of "inputting the feature matrix of the current sample image into the convolutional neural network model to determine the predicted water accumulation depth corresponding to the current sample image" is returned.

[0071] Exemplarily, the convolutional neural network model parameters that can be adjusted include: learning rate, sample batch capacity, weight decay, and number of training times.

[0072] S305: Determine the convolutional neural network model as a depth prediction model.

[0073] Specifically, after determining that the difference between the predicted waterlogging depth and the standard waterlogging depth is less than or equal to a preset threshold, the convolutional neural network model can be determined as a depth prediction model.

[0074] In this embodiment, a convolutional neural network model is trained using the acquired sample images to obtain a depth prediction model, which provides support for the subsequent use of the depth prediction model to determine the water depth at the water accumulation location to be measured corresponding to the target image, thereby achieving rapid and accurate determination of the water depth.

[0075] After introducing the training method of the depth prediction model provided by the embodiment of the present invention, the method for determining the water accumulation depth provided by the embodiment of the present invention is further introduced.

[0076] like Figure 2 As shown, the method includes:

[0077] S201, receiving original images of a water accumulation location to be measured taken by at least two vehicles.

[0078] The water accumulation to be measured at the water accumulation to be measured position in the original image may be an incomplete image of the water accumulation to be measured.

[0079] Specifically, during the driving process of the vehicle, due to the influence of various factors such as shooting angle, shooting environment or driving conditions, the original image of the water accumulation position to be measured taken by the vehicle may not include a complete image of the water accumulation position to be measured. Therefore, it is possible to obtain original images of the water accumulation position to be measured taken by at least two vehicles, and by obtaining more than one original image of the water accumulation position to be measured, more accurate data can be provided for determining the water depth of the water accumulation position to be measured.

[0080] S202: splice original images into at least one target image.

[0081] Specifically, after receiving the original images of the water accumulation position to be measured taken by at least two vehicles, the several original images can be spliced ​​into a target image containing the complete water accumulation to be measured according to the coverage position of the water accumulation to be measured at the water accumulation position to be measured.

[0082] Exemplarily, if 6 original images of the water accumulation location to be measured are received, 3 of the original images can be spliced ​​into a target image containing the complete water accumulation to be measured, and the other three original images can also be spliced ​​into a target image containing the complete water accumulation to be measured, then two target images can be obtained.

[0083] Exemplarily, the method of stitching original images into a target image is: using feature point detection and matching algorithms to detect and match feature points in the image; using feature points to calculate a transformation matrix to align images at different angles; and using a stitching tool of an image processing tool to stitch the images into a complete image.

[0084] In this embodiment, multiple original images of the water accumulation position to be measured taken by different vehicles are obtained, and the multiple original images are spliced ​​into at least one target image, so as to obtain a target image containing the complete water accumulation to be measured, eliminating the error caused by the single perspective of image shooting, and providing an accurate basic image for improving the accuracy of water accumulation depth detection later. At the same time, by splicing several incomplete original images into a relatively complete target image, comprehensive road surface information can be quickly obtained, which improves the comprehensiveness and accuracy of detection.

[0085] S203: Obtain pixel information of each pixel of the target image.

[0086] The pixel information is the numerical information of the pixel point.

[0087] Specifically, for each target image, pixel information of each pixel is obtained and presented in the form of a matrix, so that a pixel matrix corresponding to each target image can be obtained.

[0088] S204: construct features for the pixel information of each pixel according to preset demand factors to obtain a feature matrix of the target image.

[0089] The preset requirement factor is at least one of clarity, brightness division and edge density.

[0090] Specifically, according to the preset demand factors of the user, feature construction is performed on the pixel information corresponding to each target image, so as to reconstruct the original pixel matrix to obtain the feature matrix.

[0091] Exemplarily, feature construction of the feature matrix of the target image may be performed in the manner of constructing the feature matrix in S301.

[0092] In this embodiment, by constructing features of the pixel information of the target image, a feature matrix of the target image is obtained, and further processing of the target image is achieved so that the pixel information in the target image can more accurately and comprehensively display the information of the water accumulation location to be measured, thereby providing more accurate basic information for subsequent detection of the water accumulation depth at the water accumulation location to be measured.

[0093] S205 . Determine a mean square error between the target image and the sample image according to the feature matrix of the target image and the feature matrix of the sample image.

[0094] Specifically, after the characteristic matrix of the target image is determined, the mean square error value between the target image and each sample image is determined according to the characteristic matrix of the target image and the characteristic matrix of the sample image.

[0095] Exemplarily, for a target image and a sample image, the mean square difference value between the target image and the sample image may be determined by calculating the mean square difference pixel by pixel.

[0096] S206: Determine the matching degree between the target image and the sample image according to the mean square error.

[0097] The matching degree is the matching degree between the target image and the sample image. In this embodiment, the matching degree is determined by using the mean square error.

[0098] Specifically, a target image and a sample image can determine a mean square error, and therefore, the mean square errors between the target image and different sample images can be determined, and based on these mean square errors, the matching degree between the target image and each sample image can be determined.

[0099] For example, if the mean square error value between the target image and the sample image is small, it means that the two images have a high degree of match.

[0100] S207: Use the sample image with the highest matching degree as the target sample.

[0101] Specifically, after determining the matching degree between the target image and each sample image, the sample image with the highest matching degree is selected as the target sample, that is, the sample image with the smallest mean square error between the target image and the sample image is selected as the target sample.

[0102] For example, if there are multiple target images, the sample image with the highest matching degree with each target image can be determined as the target sample. Therefore, each target image will correspond to a target sample.

[0103] S208 . Determine a depth difference according to a feature matrix of the target image, a feature matrix of the target sample, and a preset depth prediction model.

[0104] Among them, a sample image in the sample set corresponds to a standard water depth.

[0105] Specifically, the feature matrix of the target image and the feature matrix of the target sample are input into a preset depth prediction model to obtain a depth difference.

[0106] Exemplarily, in one implementation, a matrix difference is determined based on a feature matrix of a target image and a feature matrix of a target sample, the matrix difference is used as an input to a depth prediction model, and the depth difference is determined using the depth prediction model. In another implementation, the feature matrix of the target image and the feature matrix of the target sample are respectively input into the depth prediction model to obtain a depth value corresponding to the feature matrix of the target image and a depth value corresponding to the feature matrix of the target sample, and the two depth values ​​are subtracted to obtain a depth difference.

[0107] Optionally, the depth difference may be determined as follows:

[0108] (1) Determine the difference feature matrix based on the feature matrix of the target image and the feature matrix of the sample image.

[0109] Specifically, the difference between the feature matrix of the target image and the feature matrix of the sample image is calculated to obtain a difference feature matrix.

[0110] Exemplarily, the difference feature matrix may be determined according to the following formula.

[0111] F diff =F A -F match ;

[0112] Among them, F diff is the difference feature matrix, F A is the feature matrix of the target image, F match is the feature matrix of the sample image.

[0113] (2) Perform threshold processing on the difference feature matrix.

[0114] Specifically, after the difference feature matrix is ​​determined, threshold processing is performed on the difference feature matrix to reduce the impact of environmental interference on the final calculation result.

[0115] Exemplarily, the difference feature matrix may be thresholded according to the following formula.

[0116]

[0117] Among them, F diff ' is the difference feature matrix, "if | F diff |<T” and “Others” are threshold limits. That is, determine F diff The absolute value of each difference pixel in , if its absolute value satisfies "absolute value is less than T", then the difference pixel is set to zero, if not, then the difference pixel value remains unchanged.

[0118] (3) The difference feature matrix after threshold processing is input into the depth prediction model to obtain the depth difference.

[0119] Specifically, the difference feature matrix after threshold processing is input into the depth prediction model, which can be predicted by the depth prediction model, and the output result is the depth difference.

[0120] Exemplarily, in this embodiment, if there are multiple target images. In one implementation, the target samples corresponding to each target image are determined separately. The average matrix of the feature matrices of all target images is calculated, that is, the feature matrices of all target images are added and averaged to obtain the average matrix of the target image. Similarly, the average matrix of the feature matrices of all target samples is calculated. The difference feature matrix between the average matrix of the target image and the average matrix of the target sample is calculated, and the difference feature matrix is ​​input into the depth prediction model to obtain the depth difference. In another implementation, the difference feature matrix of each target image and each target sample is calculated separately to obtain multiple difference feature matrices, and they are respectively input into the depth prediction model to obtain multiple depth differences. The depth differences are averaged, and finally the average depth difference is used as the final depth difference.

[0121] Exemplarily, if the method of "inputting the feature matrix of the target image and the feature matrix of the target sample into the depth prediction model respectively, obtaining the depth value corresponding to the feature matrix of the target image and the depth value corresponding to the feature matrix of the target sample, and subtracting the two depth values ​​to obtain the depth difference" is selected, then similarly, the depth difference can be determined according to the methods in one of the above-mentioned implementation methods and another implementation method, which will not be repeated here.

[0122] In this embodiment, by determining the difference feature matrix, the interference of changes in the shooting environment (such as light intensity, shadow size, and weather changes, etc.) on the water depth detection results is effectively reduced. And the threshold processing of the difference feature matrix emphasizes the change of water depth and reduces the influence of environmental factors on the detection accuracy.

[0123] S209: Determine the target water depth of the water accumulation position to be measured according to the standard water accumulation depth and the depth difference corresponding to the target sample.

[0124] Specifically, since the target sample is the image with the highest matching degree with the target image, that is, the feature matrix of the target sample has the highest matching degree with the feature matrix of the target image, and the depth difference is obtained by inputting the difference feature matrix between the feature matrix of the target sample and the feature matrix of the target image into the depth prediction model, the target water depth of the water accumulation position to be measured can be directly determined based on the standard water accumulation depth corresponding to the target sample and the depth difference.

[0125] For example, the target water depth can be determined according to the following formula:

[0126] D real =D match -D diff ;

[0127] Among them, D diff is the depth difference, D matchis the standard water depth corresponding to the target sample, D real is the target water depth.

[0128] Optionally, after determining the target water depth, the following steps may be performed:

[0129] When the target water depth exceeds the preset safety threshold, an alarm is triggered and the water depth information is fed back to the relevant regulatory authorities for processing.

[0130] In this embodiment, the target water depth of the water sample with the highest matching degree with the target image and the depth difference are directly used to determine the target water depth of the water position to be measured, which reduces the problem that the target image is easily affected by the actual shooting angle, environment or other influencing factors in the actual process of shooting the target image, and reduces the impact of environmental factors on the detection accuracy. At the same time, after obtaining the depth difference determined by the depth prediction model, the target water depth of the water position to be measured is determined by using the standard water depth and the depth difference, instead of directly using the depth prediction model to predict the feature matrix of the target image, which increases the robustness and stability of the detection.

[0131] The method for determining the depth of water accumulation provided by the embodiment of the present invention receives original images of the water accumulation position to be measured taken by at least two vehicles; splices the original images into at least one target image; obtains the pixel information of each pixel of the target image; constructs the features of the pixel information of each pixel according to the preset demand factors, and obtains the feature matrix of the target image; determines the mean square error between the target image and the sample image according to the feature matrix of the target image and the feature matrix of the sample image; determines the matching degree between the target image and the sample image according to the mean square error; uses the sample image with the highest matching degree as the target sample; determines the depth difference according to the feature matrix of the target image, the feature matrix of the target sample and the pre-set depth prediction model; determines the target water accumulation depth of the water accumulation position to be measured according to the standard water accumulation depth and the depth difference corresponding to the target sample. In the above technical scheme, on the one hand, by acquiring multiple original images of the water accumulation position to be measured taken by different vehicles, the multiple original images are spliced ​​into at least one target image, thereby obtaining a target image containing the complete water accumulation to be measured, eliminating the error caused by the single perspective of image shooting, and providing an accurate basic image for improving the detection accuracy of water accumulation depth later. At the same time, by splicing several incomplete original images into a relatively complete target image, comprehensive road surface information can be quickly obtained, which improves the comprehensiveness and accuracy of the detection. On the other hand, by constructing the features of the pixel information of the target image, the feature matrix of the target image is obtained, and the target image is further processed so that the pixel information in the target image can more accurately and comprehensively show the information of the water accumulation position to be measured, and provide more accurate basic information for the subsequent detection of the water accumulation depth of the water accumulation position to be measured. On the other hand, by determining the difference feature matrix, the interference of the water accumulation depth detection result caused by changes in the shooting environment (such as light intensity, shadow size and weather changes, etc.) is effectively reduced. And the threshold processing of the difference feature matrix emphasizes the change of water accumulation depth and reduces the influence of environmental factors on the detection accuracy. Finally, the standard water accumulation depth and depth difference of the target sample with the highest matching degree with the target image are directly used to determine the target water accumulation depth of the water accumulation position to be measured, which reduces the problem that the target image is easy to affect the detection accuracy due to the actual shooting angle, environment or other influencing factors in the actual shooting process of the target image, and reduces the influence of environmental factors on the detection accuracy. At the same time, after obtaining the depth difference determined by the depth prediction model, the target water depth of the water location to be measured is determined using the standard water depth and the depth difference, instead of directly using the depth prediction model to predict the feature matrix of the target image, which increases the robustness and stability of the detection.

[0132] Figure 4 Schematic diagram of the structure of the water depth determination device provided by the embodiment of the present invention. Figure 4 As shown, the device comprises:

[0133] An acquisition module 401 is used to acquire a target image of a water accumulation position to be measured;

[0134] A selection module 402 is used to determine the matching degree between the target image and each sample image in the sample set, and select the sample image with the highest matching degree as the target sample;

[0135] Determination module 403 is used to determine the target water depth of the water accumulation position to be measured based on the feature matrix of the target image, the feature matrix of the target sample and a pre-set depth prediction model, wherein the depth prediction model is a model for determining the water accumulation depth obtained by training a convolutional neural network model using a sample set.

[0136] Optionally, the acquisition module 401 is specifically used for:

[0137] Receive original images of the water accumulation location to be measured taken by at least two vehicles; and splice the original images into at least one target image.

[0138] Optionally, after acquiring the target image of the water accumulation position to be measured, the acquisition module 401 is further used to:

[0139] Obtain pixel information of each pixel of the target image; construct features for the pixel information of each pixel according to preset demand elements to obtain a feature matrix of the target image, wherein the preset demand elements are at least one of clarity, brightness division and edge density.

[0140] Optionally, for any sample image in the sample set, the selection module 402 is specifically configured to:

[0141] According to the feature matrix of the target image and the feature matrix of the sample image, the mean square error between the target image and the sample image is determined; according to the mean square error, the matching degree between the target image and the sample image is determined.

[0142] Optionally, a sample image in the sample set corresponds to a standard water depth, and the determination module 403 is specifically used to:

[0143] The depth difference is determined according to the feature matrix of the target image, the feature matrix of the target sample and the preset depth prediction model; the target water depth of the water accumulation position to be measured is determined according to the standard water accumulation depth corresponding to the target sample and the depth difference.

[0144] Optionally, the determination module 403 is specifically configured to:

[0145] According to the feature matrix of the target image and the feature matrix of the sample image, a difference feature matrix is ​​determined; a threshold process is performed on the difference feature matrix; and the difference feature matrix after the threshold process is input into a depth prediction model to obtain a depth difference.

[0146] Optionally, the device further includes a depth prediction model training module, and the depth prediction model training module is specifically used for:

[0147] A sample set is obtained, wherein the sample set includes at least two sample images, and one sample image corresponds to a standard water depth; a feature matrix of a current sample image is input into a convolutional neural network model to determine a predicted water depth corresponding to the current sample image; if the difference between the predicted water depth corresponding to the current sample image and the standard water depth corresponding to the current sample image is less than or equal to a preset threshold, the convolutional neural network model is determined as a depth prediction model; if the difference between the predicted water depth corresponding to the current sample image and the standard water depth corresponding to the current sample image is greater than a preset threshold, the parameters of the convolutional neural network model are adjusted, the next sample image is used as the current sample image, and the step of inputting the feature matrix of the current sample image into the convolutional neural network model to determine the predicted water depth corresponding to the current sample image is returned.

[0148] The device for determining the water depth provided in the embodiment of the present invention can execute the method for determining the water depth provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0149] Figure 5 A schematic diagram of the structure of an electronic device 10 provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0150] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0151] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0152] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for determining the depth of water accumulation.

[0153] In some embodiments, the method for determining the depth of water accumulation may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for determining the depth of water accumulation described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method for determining the depth of water accumulation in any other appropriate manner (e.g., by means of firmware).

[0154] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0156] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0158] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0159] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0160] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0161] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining water depth, characterized in that: The method comprises: Acquire a target image of the water accumulation position to be measured; Determine the matching degree between the target image and each sample image in the sample set respectively, and take the sample image with the highest matching degree as the target sample; The target water depth of the water accumulation position to be measured is determined according to the feature matrix of the target image, the feature matrix of the target sample and a pre-set depth prediction model, wherein the depth prediction model is a model for determining the water accumulation depth obtained by training a convolutional neural network model using the sample set.

2. The method for determining the water depth according to claim 1, characterized in that: The step of obtaining a target image of a water accumulation position to be measured comprises: Receiving original images of the water accumulation location to be measured taken by at least two vehicles; The original images are stitched into at least one target image.

3. The method for determining the water depth according to claim 1 or 2, characterized in that: After obtaining the target image of the water accumulation position to be measured, it also includes: Acquire pixel information of each pixel of the target image; According to preset requirement factors, feature construction is performed on pixel information of each pixel to obtain a feature matrix of the target image, wherein the preset requirement factor is at least one of clarity, brightness division and edge density.

4. The method for determining the water depth according to claim 1, characterized in that: For any sample image in the sample set, determining the matching degree between the target image and the sample image includes: Determining a mean square error between the target image and the sample image according to a feature matrix of the target image and a feature matrix of the sample image; The matching degree between the target image and the sample image is determined according to the mean square error.

5. The method for determining the water depth according to claim 1, characterized in that: A sample image in the sample set corresponds to a standard water depth; The step of determining the target water depth of the water accumulation position to be measured according to the feature matrix of the target image, the feature matrix of the target sample and a preset depth prediction model includes: Determining a depth difference according to a feature matrix of the target image, a feature matrix of the target sample, and a preset depth prediction model; The target water depth of the water accumulation position to be measured is determined according to the standard water accumulation depth corresponding to the target sample and the depth difference.

6. The method for determining the water depth according to claim 5, characterized in that: The determining of the depth difference according to the feature matrix of the target image, the feature matrix of the target sample and a preset depth prediction model includes: Determining a difference feature matrix according to the feature matrix of the target image and the feature matrix of the sample image; Performing threshold processing on the difference feature matrix; The difference feature matrix after threshold processing is input into the depth prediction model to obtain the depth difference.

7. The method for determining water depth according to claim 1, characterized in that: The method for training the depth prediction model comprises: Acquire the sample set, wherein the sample set includes at least two sample images, and one sample image corresponds to a standard water depth; Inputting the feature matrix of the current sample image into the convolutional neural network model to determine the predicted water depth corresponding to the current sample image; If the difference between the predicted water depth corresponding to the current sample image and the standard water depth corresponding to the current sample image is less than or equal to a preset threshold, the convolutional neural network model is determined as the depth prediction model; If the difference between the predicted water depth corresponding to the current sample image and the standard water depth corresponding to the current sample image is greater than a preset threshold, the parameters of the convolutional neural network model are adjusted, and the next sample image is used as the current sample image, and the step of inputting the feature matrix of the current sample image into the convolutional neural network model to determine the predicted water depth corresponding to the current sample image is returned.

8. A device for determining water depth, characterized in that: include: An acquisition module is used to acquire a target image of a water accumulation position to be measured; A selection module is used to determine the matching degree between the target image and each sample image in the sample set, and take the sample image with the highest matching degree as the target sample; A determination module is used to determine the target water depth of the water accumulation position to be measured based on the feature matrix of the target image, the feature matrix of the target sample and a pre-set depth prediction model, wherein the depth prediction model is a model for determining the water accumulation depth obtained by training a convolutional neural network model using the sample set.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the water depth according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining the water depth according to any one of claims 1 to 7 when executed.

Images