Region detection model training method, region detection method and device
By introducing an edge loss function into the area detection model of autonomous driving vehicles, the problem of insufficient edge detection accuracy in the feasible area is solved, more accurate edge detection is achieved, and the safety of the vehicle is improved.
Patent Information
- Application Number
- CN202310981395.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-04
- Publication Date
- 2025-02-14
AI Technical Summary
Among existing autonomous driving vehicles, the edge detection accuracy of the driving area detection is insufficient, resulting in the vehicle that may drive to an unmovable area, posing a safety hazard.
By introducing an edge loss function during the training of the area detection model, the model parameters are adjusted to improve the fineness of edge detection, thereby detecting more accurate driving area edges.
The detection accuracy of the area detection model on the edge of the travelable area is improved, so as to avoid the vehicle from driving to the unmovable area, and enhance the safety of the vehicle.
Smart Images
Figure CN119445136A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of machine learning technology, and in particular, relates to a training method for a region detection model, a region detection method and a device. Background Art
[0002] With the development of social science and technology, autonomous driving vehicles have begun to be used in more and more scenarios. The detection of drivable areas is a particularly important part of autonomous driving. The drivable area detection mainly participates in the overall decision-making of autonomous driving by detecting the areas where the vehicle can pass.
[0003] Most existing methods use artificial intelligence models to segment the drivable area in the image. During the model training process, the cross-entropy loss function is mainly used to measure the difference between the true value and the model's predicted value, thereby standardizing the learning direction of the model.
[0004] However, since there are usually large areas in an image that belong to the drivable area, the proportion is too high, which often results in a very low cross entropy loss function, a high IOU between the true value and the model prediction value, but in fact, the predicted drivable area edge is uneven with the true value edge, and the effect is very poor. This may cause the vehicle to travel to a place that does not belong to the drivable area, which is very likely to cause a safety accident. Summary of the invention
[0005] The embodiments of the present application provide a training method, a region detection method and a device for a region detection model, which can enhance the precision of the region detection model for edge detection of a drivable area through an edge loss function, so that the region detection model can detect a drivable area with a more precise edge, thereby preventing the vehicle from driving into a non-drivable area and improving the safety of the vehicle.
[0006] In a first aspect, an embodiment of the present application provides a method for training a region detection model, the method comprising:
[0007] Obtain multiple training samples, the training samples include a target image and its annotated image, the target image contains a drivable area, and the annotated image is obtained by annotating the drivable area in the target image.
[0008] The target image in the training sample is input into the preset area detection model. The drivable area in the target image is detected by the area detection model, and the predicted image is output. The predicted image is an image with the predicted drivable area marked.
[0009] Extract the edge of the predicted drivable area in the predicted image to obtain a first edge image, and extract the edge of the drivable area in the labeled image to obtain a second edge image.
[0010] According to the first edge image and the second edge image, an edge loss function value is determined,
[0011] When the edge loss function value does not meet the training stop condition, the model parameters of the region detection model are adjusted, and the region detection model after the parameter adjustment is trained using training samples until the training stop condition is met, thereby obtaining a trained region detection model.
[0012] In a first aspect, an embodiment of the present application provides a region detection method, the method comprising:
[0013] Acquire a first image, the first image including a drivable area,
[0014] The first image is input into the region detection model, and the drivable area in the first image is detected by the region detection model, and a second image is output, where the second image is obtained by marking the drivable area in the first image, and the region detection model is trained based on the training method of the region detection model according to any one of claims 1 to 7.
[0015] In a third aspect, an embodiment of the present application provides a training device for a region detection model, the device comprising:
[0016] The first acquisition module is used to acquire multiple training samples, the training samples include a target image and an annotated image thereof, the target image includes a drivable area, and the annotated image is obtained by annotating the drivable area in the target image.
[0017] The input module is used to input the target image in the training sample into the preset area detection model, detect the drivable area in the target image through the area detection model, and output a predicted image, which is an image with the predicted drivable area marked.
[0018] The extraction module is used to extract the edge of the predicted drivable area in the predicted image to obtain a first edge image, and to extract the edge of the drivable area in the annotated image to obtain a second edge image.
[0019] A determination module, used to determine an edge loss function value according to the first edge image and the second edge image,
[0020] The adjustment module is used to adjust the model parameters of the region detection model when the edge loss function value does not meet the training stop condition, and use the training samples to train the region detection model with the adjusted parameters until the training stop condition is met, thereby obtaining the trained region detection model.
[0021] In a fourth aspect, an embodiment of the present application provides a region detection device, the device comprising:
[0022] A first acquisition module is used to acquire a first image, wherein the first image includes a drivable area.
[0023] A detection module is used to input a first image into a region detection model, detect a drivable area in the first image through the region detection model, and output a second image, wherein the second image is obtained by marking the drivable area in the first image, and the region detection model is trained based on the training method of the region detection model shown in any embodiment of the first aspect.
[0024] In a fifth aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions,
[0025] When the processor executes the computer program instructions, it implements the training method of the region detection model as shown in any one of the embodiments of the first aspect and / or the region detection method as shown in any one of the embodiments of the second aspect.
[0026] In a sixth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the training method for the region detection model shown in any one of the embodiments of the first aspect and / or the region detection method shown in any one of the embodiments of the second aspect are implemented.
[0027] In the seventh aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the training method of the region detection model shown in any one of the embodiments of the first aspect and / or the region detection method shown in any one of the embodiments of the second aspect.
[0028] The training method, method and device of the region detection model of the embodiment of the present application can input the target image in the training sample into the preset region detection model, detect the drivable area in the target image through the region detection model, output a predicted image with the predicted drivable area marked, then extract the edge of the predicted drivable area in the predicted image to obtain a first edge image, and extract the edge of the drivable area in the marked image in the training sample to obtain a second edge image, and then determine the edge loss function value based on the first edge image and the second edge image, and adjust the model parameters of the region detection model when the edge loss function value does not meet the training stop condition, and use the training sample to train the region detection model with the adjusted parameters until the training stop condition is met to obtain the trained region detection model. In this way, the edge loss function can be used to enhance the precision of the region detection model's edge detection of the drivable area, so that the region detection model can detect the drivable area with more accurate edges, thereby avoiding the vehicle from driving into the non-drivable area and improving the safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0030] Figure 1 is one of the flow charts of a method for training a region detection model provided by an embodiment of the present application,
[0031] Figure 2 is a schematic diagram of pixel values of a second edge image provided by an embodiment of the present application,
[0032] Figure 3 is a schematic diagram of grayscale values of a grayscale image provided by an embodiment of the present application,
[0033] Figure 4 This is a second flowchart of a method for training a region detection model provided by an embodiment of the present application.
[0034] Figure 5 is a flow chart of a region detection method provided by an embodiment of the present application.
[0035] Figure 6 is a structural diagram of a training device for a region detection model provided by an embodiment of the present application,
[0036] Figure 7 is a schematic diagram of the structure of a region detection device provided by an embodiment of the present application,
[0037] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0038] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0039] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0040] As mentioned in the background technology, in the existing technical solutions, the model will pay more attention to the size of the overlapping area of the segmented area and ignore the refinement of the edge. Therefore, although most of the drivable areas that can be passed can be segmented, there is no good constraint on the segmented edges. The edge of the drivable area is the dividing line between safety and unsafety. Therefore, it is particularly important to add an edge constraint on the basis of the existing method to refine the segmented edges.
[0041] In order to solve the above problems, an embodiment of the present application provides a training method for a region detection model, which is described in detail below.
[0042] Figure 1 A flow chart of a method for training a region detection model provided by an embodiment of the present application is shown. It should be noted that the method for training a region detection model can be applied to a training device for a region detection model, such as Figure 1 As shown, the training method of the region detection model may include the following steps:
[0043] S110, obtaining multiple training samples,
[0044] S120, inputting the target image in the training sample into a preset area detection model, detecting the drivable area in the target image through the area detection model, and outputting a predicted image,
[0045] S130, extracting the edge of the predicted drivable area in the predicted image to obtain a first edge image, and extracting the edge of the drivable area in the annotated image to obtain a second edge image,
[0046] S140, determining an edge loss function value according to the first edge image and the second edge image,
[0047] S150, when the edge loss function value does not meet the training stop condition, adjust the model parameters of the region detection model, and use the training samples to train the region detection model with the adjusted parameters until the training stop condition is met, thereby obtaining the trained region detection model.
[0048] Thus, the target image in the training sample can be input into the preset region detection model, and the drivable region in the target image can be detected by the region detection model, and the predicted image with the predicted drivable region marked is outputted, and then the edge of the predicted drivable region in the predicted image is extracted to obtain the first edge image, and the edge of the drivable region in the marked image in the training sample is extracted to obtain the second edge image, and then the edge loss function value is determined according to the first edge image and the second edge image, and when the edge loss function value does not meet the training stop condition, the model parameters of the region detection model are adjusted, and the region detection model with the adjusted parameters is trained using the training sample until the training stop condition is met, and the trained region detection model is obtained. In this way, the edge loss function can be used to enhance the precision of the region detection model in detecting the edge of the drivable region, so that the region detection model can detect the drivable region with more accurate edges, thereby preventing the vehicle from driving into the non-drivable region and improving the safety of the vehicle.
[0049] In S110, the drivable area may be an area where a vehicle can pass. The training sample may include a target image and its annotated image. The target image may include a drivable area, and the annotated image may be obtained by annotating the drivable area in the target image.
[0050] The annotated image may be a mask image. For example, in the annotated image, the pixel value of the drivable area may be 1, and the pixel value of other areas may be 0, or the pixel value of the drivable area may be 0, and the pixel value of other areas may be 1.
[0051] For example, the target image may be obtained by photographing the target environment, and the target environment may include a drivable area. The annotated image may be annotated with the drivable area in the target image in advance and stored in an annotation file, and obtained from the annotation file when used, or may be obtained by annotating the drivable area in the target image when used.
[0052] In S120 , the predicted image may be an image with the predicted drivable area annotated. The predicted image may be a model prediction value, and the annotated image may be a true value.
[0053] In some implementations, the region detection model may be a convolutional neural network, so that the drivable region in the target image can be detected more accurately.
[0054] Regarding S130, the first edge image may include an edge of the predicted drivable area in the predicted image, and the second edge image may include an edge of the drivable area in the labeled image.
[0055] In some implementations, in order to obtain more accurate first edge images and second edge images, the above S130 may include:
[0056] Convolution operations are performed on the predicted image and the annotated image respectively to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image.
[0057] In this way, the edge extraction of the predicted image and the annotated image can be performed through the convolution operation to obtain a more accurate first edge image and a second edge image.
[0058] Most existing edge detection algorithms can only be implemented in a central processing unit (CPU), and when training a model, the data is usually placed in a graphics processing unit (GPU) for operation. If edge extraction is performed during the training process, it will inevitably cause frequent data interaction between the CPU and the GPU, seriously slowing down the training efficiency of the model. In order to improve the training efficiency of the model, in some embodiments, the above-mentioned convolution operation is performed on the predicted image and the annotated image respectively to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image, which may include:
[0059] Get the Laplacian operator template,
[0060] The Laplacian operator template is used as the weight in the convolution kernel, and convolution operations are performed on the predicted image and the annotated image respectively to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image.
[0061] Here, in order to realize edge extraction in GPU, convolution operation can be used to extract edges. Since there will be obvious step changes at the edges in the image (that is, the first-order derivative reaches a peak at the edge, and the second-order derivative reaches zero at the edge) and roof changes (that is, the first-order derivative reaches zero at the edge, and the second-order derivative reaches a peak at the edge), in view of this characteristic, the Laplacian operator template can be used as the weight in the convolution kernel, and the Laplacian operator can be used for edge extraction.
[0062] In this way, by using the Laplacian operator template as the weight in the convolution kernel, convolution operations can be performed in the GPU, and edges in the image can be extracted in the GPU. As a result, both model training and edge extraction can be performed in the GPU without the need for frequent data interaction between the CPU and GPU, thereby improving the training efficiency of the model.
[0063] In some implementations, in order to obtain an accurate Laplacian operator template, the above-mentioned obtaining of the Laplacian operator template may include:
[0064] Determine the second-order derivative formulas of x and y in the pixel coordinates (x, y), respectively,
[0065] Substitute the second-order derivative formulas of x and y into the Laplace operator formula to obtain the Laplace operator template.
[0066] Exemplarily, the Cartesian coordinates may be represented by a Laplace operator in a two-dimensional space.
[0067] The Laplace operator formula can be:
[0068]
[0069] Wherein, Δf may be a Laplace operator, and (x, y) may be a pixel coordinate.
[0070] Consider an image as a two-dimensional space and each pixel as the value of the corresponding coordinate. According to the above formula, we can first find the second-order derivatives corresponding to x and y respectively, and then use the derivative formula to find the first-order derivatives on the left and right sides of the current coordinate (x, y).
[0071] The first-order derivative formulas for the left and right sides of x and y can be:
[0072]
[0073]
[0074] After obtaining the first-order derivative, the second-order derivative in the x and y directions can be obtained from the first-order derivatives on both sides of the coordinate point.
[0075] The second-order derivative formulas for x and y can be:
[0076] f(x+1,y)+f(x-1,y)-2f(x,y)
[0077] f(x,y+1)+f(x,y-1)-2f(x,y)
[0078] Substituting the second-order derivative formulas of x and y into the Laplace operator formula, we can get the following formula:
[0079] Δf=f(x,y+1)+f(x,y-1)+f(x+1,y)+f(x-1,y)-4f(x,y)
[0080] Then the Laplacian operator template can be obtained.
[0081] The Laplacian operator template can be:
[0082]
[0083] in, Can be a Laplacian operator template.
[0084] Thus, through the above process, an accurate Laplace operator template can be obtained.
[0085] In S140, an edge loss function value may be determined according to the first edge image and the second edge image. By applying the edge loss function to the area detection model, the precision of detecting the edge of the drivable area may be enhanced.
[0086] The existing methods for calculating the loss between two images are all region-based loss calculation methods, that is, calculating the difference values of the pixels at corresponding positions of the two images and taking the average. However, in the embodiment of the present application, what needs to be calculated is the difference between two binary images with edges. Since the number of edges accounts for a very small proportion and the samples are extremely unbalanced, if the existing region-based loss calculation method is used to calculate the edge difference, the model cannot learn the edge information well. In order to enable the region detection model to better learn the edge information, in some embodiments, the above S140 may include:
[0087] Convert the second edge image to a grayscale image,
[0088] Multiply the grayscale values of multiple pixels in the grayscale image by the pixel values of the pixels at corresponding positions in the first edge image to obtain multiple products.
[0089] The average of multiple products is determined as the marginal loss function value.
[0090] Here, both the first edge image and the second edge image may be mask images. Exemplarily, the pixel value of the edge in the first edge image and the second edge image may be 1, and the pixel value of other areas may be 0. The gray value of each pixel in the gray image may be the closest Euclidean distance from the pixel to the edge line included in the second edge image.
[0091] For example, the process of converting the second edge image into a grayscale image can be implemented using the distance_transform_edt() method in the scipy library (scipy is an open source Python algorithm library and mathematical toolkit).
[0092] Exemplarily, the pixel values of the second edge image may be as follows: Figure 2 As shown, the grayscale value of the grayscale image can be expressed as Figure 3 shown.
[0093] Then, the grayscale image is multiplied by the corresponding pixels of the first edge image, the sum is calculated, and the mean is finally obtained to obtain the edge loss function value.
[0094] In this way, by calculating the edge loss function value through the above process, the region detection model can better learn the edge information.
[0095] In some examples, Figure 4 As shown, the drivable area in the target image 410 is annotated to obtain the annotated image 420, the target image 410 is input into the area detection model 430 to obtain the predicted image 440, and then the annotated image 420 and the predicted image 440 are input into the edge extraction module 450 to obtain the first edge image 460 and the second edge image 470, and then the second edge image 470 is input into the conversion module 480, and the second edge image 470 is converted into a grayscale image by the conversion module 480, and then the grayscale image and the first edge image 460 are input into the loss calculation module 490 to calculate the loss function value.
[0096] Involving S150, if the edge loss function value does not meet the training stop condition, the model parameters of the region detection model can be adjusted, and then the region detection model with adjusted training parameters can be trained using the training samples (return to execute S120 to S150) until the edge loss function value meets the training stop condition, thereby obtaining the trained region detection model.
[0097] Exemplarily, the training stop condition may be a loss function threshold.
[0098] In an embodiment of the present application, a method for training a region detection model is proposed to address the problem of poor edge distinction capability when performing drivable area detection tasks. Specifically, the edge extraction module can quickly extract edges in the mask image through convolution operations in the GPU, and the distance transformation module converts the edges in the annotation image into a grayscale image through the Euclidean distance transformation, and multiplies it with the first edge image to obtain the position of each edge pixel deviation, which can further enhance the ability of the deep model to learn edges. The edge of the drivable area is the dividing line between safety and unsafety, so improving the edge detection capability of the deep model can greatly improve the safety of the autonomous driving system.
[0099] Combine the following Figure 5 The area detection method provided in the embodiment of the present application is described in detail.
[0100] Figure 5 A schematic flow chart of a region detection method provided by an embodiment of the present application is shown. It should be noted that the region detection method can be applied to a region detection device, such as Figure 5 As shown, the area detection method may include the following steps:
[0101] S510, acquiring a first image,
[0102] S520: Input the first image to the area detection model, detect the drivable area in the first image through the area detection model, and output a second image.
[0103] Therefore, the first image containing the drivable area is input into the region detection model, and the drivable area in the first image is detected by the region detection model, and a second image with the drivable area marked in the first image can be output. Since the edge loss function is used to measure the difference between the true value and the predicted value of the model during the training process of the region detection model, the precision of the region detection model in detecting the edge of the drivable area is enhanced, so that the region detection model can detect the drivable area with more precise edges, thereby avoiding the vehicle from driving into the non-drivable area and improving the safety of the vehicle.
[0104] In S510, the first image may include a drivable area, the first image may be obtained by photographing a first environment, and the first environment may include a drivable area. The area detection model may be obtained by training based on the area detection model training method of any implementation of the above embodiments.
[0105] In S520, the second image may be obtained by marking the drivable area in the first image. For example, the second image may be a mask image, in which the pixel values of the drivable area and other areas are different.
[0106] Based on the same inventive concept, the present application embodiment also provides a training device for a region detection model. Figure 6 The training device of the region detection model provided in the embodiment of the present application is described in detail.
[0107] Figure 6 A schematic structural diagram of a training device for a region detection model provided in one embodiment of the present application is shown.
[0108] like Figure 6 As shown, the training device of the region detection model may include:
[0109] The first acquisition module 601 is used to acquire multiple training samples, where the training samples include a target image and an annotated image thereof, where the target image includes a drivable area, and the annotated image is obtained by annotating the drivable area in the target image.
[0110] The input module 602 is used to input the target image in the training sample into a preset area detection model, detect the drivable area in the target image through the area detection model, and output a predicted image, which is an image with the predicted drivable area marked.
[0111] The extraction module 603 is used to extract the edge of the predicted drivable area in the predicted image to obtain a first edge image, and to extract the edge of the drivable area in the annotated image to obtain a second edge image.
[0112] A determination module 604 is used to determine an edge loss function value according to the first edge image and the second edge image.
[0113] The adjustment module 605 is used to adjust the model parameters of the region detection model when the edge loss function value does not meet the training stop condition, and use the training sample to train the region detection model with adjusted parameters until the training stop condition is met to obtain the trained region detection model.
[0114] Thus, the target image in the training sample can be input into the preset region detection model, and the drivable region in the target image can be detected by the region detection model, and the predicted image with the predicted drivable region marked is outputted, and then the edge of the predicted drivable region in the predicted image is extracted to obtain the first edge image, and the edge of the drivable region in the marked image in the training sample is extracted to obtain the second edge image, and then the edge loss function value is determined according to the first edge image and the second edge image, and when the edge loss function value does not meet the training stop condition, the model parameters of the region detection model are adjusted, and the region detection model with the adjusted parameters is trained using the training sample until the training stop condition is met, and the trained region detection model is obtained. In this way, the edge loss function can be used to enhance the precision of the region detection model in detecting the edge of the drivable region, so that the region detection model can detect the drivable region with more accurate edges, thereby preventing the vehicle from driving into the non-drivable region and improving the safety of the vehicle.
[0115] In some implementations, in order to obtain more accurate first edge images and second edge images, the extraction module 603 may include:
[0116] The convolution submodule is used to perform convolution operations on the predicted image and the annotated image respectively to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image.
[0117] In some implementations, in order to improve the training efficiency of the model, the convolution submodule may include:
[0118] The acquisition unit is used to obtain the Laplace operator template.
[0119] The convolution unit is used to perform convolution operations on the predicted image and the annotated image respectively using the Laplacian operator template as the weight in the convolution kernel to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image.
[0120] In some implementations, in order to obtain an accurate Laplacian operator template, the acquisition unit may include:
[0121] Determine a subunit for determining the second-order derivative formulas of x and y in the pixel coordinates (x, y), respectively,
[0122] The substitution subunit is used to substitute the second-order derivative formulas of x and y into the Laplace operator formula to obtain the Laplace operator template.
[0123] In some embodiments, the Laplace operator formula is:
[0124]
[0125] The formulas for the second derivatives of x and y are:
[0126] f(x+1,y)+f(x-1,y)-2f(x,y)
[0127] f(x,y+1)+f(x,y-1)-2f(x,y)
[0128] The Laplacian operator template is:
[0129]
[0130] Where Δf is the Laplace operator, (x, y) is the pixel coordinate, is the Laplacian operator template.
[0131] In some implementations, in order to enable the region detection model to better learn edge information, the determination module 604 may include:
[0132] The conversion submodule is used to convert the second edge image into a grayscale image, wherein the grayscale value of each pixel in the grayscale image is the closest Euclidean distance from the pixel to the edge line contained in the second edge image.
[0133] A calculation submodule is used to multiply the grayscale values of multiple pixels in the grayscale image with the pixel values of the pixels at corresponding positions in the first edge image to obtain multiple products.
[0134] A determination submodule is used to determine the average value of multiple products as the marginal loss function value.
[0135] In some embodiments, the region detection model is a convolutional neural network.
[0136] Based on the same inventive concept, the present application embodiment also provides a region detection device. Figure 7 The area detection device provided in the embodiment of the present application is described in detail.
[0137] Figure 7 A schematic structural diagram of a region detection device provided in one embodiment of the present application is shown.
[0138] like Figure 7 As shown, the area detection device may include:
[0139] The first acquisition module 701 is used to acquire a first image, where the first image contains a drivable area.
[0140] The detection module 702 is used to input the first image into the area detection model, detect the drivable area in the first image through the area detection model, and output a second image, where the second image is obtained by marking the drivable area in the first image, and the area detection model is trained based on the training method of the area detection model of any implementation method in the above embodiments.
[0141] Therefore, the first image containing the drivable area is input into the region detection model, and the drivable area in the first image is detected by the region detection model, and a second image with the drivable area marked in the first image can be output. Since the edge loss function is used to measure the difference between the true value and the predicted value of the model during the training process of the region detection model, the precision of the region detection model in detecting the edge of the drivable area is enhanced, so that the region detection model can detect the drivable area with more precise edges, thereby avoiding the vehicle from driving into the non-drivable area and improving the safety of the vehicle.
[0142] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present application is shown.
[0143] like Figure 8 As shown, the electronic device 8 is a structural diagram of an exemplary hardware architecture of an electronic device that can implement the training method of the region detection model, the region detection method, the training device of the region detection model, and the region detection device in the embodiments of the present application. The electronic device may refer to the electronic device in the embodiments of the present application.
[0144] The electronic device 8 may include a processor 801 and a memory 802 storing computer program instructions.
[0145] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0146] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 802 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory. In a specific embodiment, the memory 802 may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Therefore, typically, the memory 802 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.
[0147] The processor 801 implements any one of the region detection model training methods or region detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 802 .
[0148] In one example, the electronic device may further include a communication interface 803 and a bus 804. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 804 and communicate with each other.
[0149] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0150] Bus 804 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 804 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0151] The electronic device can execute the training method or the area detection method of the area detection model in the embodiment of the present application, thereby realizing the combination of Figures 1 to 7 Described are a training method for a region detection model, a region detection method, a training device for a region detection model, and a region detection device.
[0152] In addition, in combination with the training method of the region detection model or the region detection method in the above embodiments, the embodiments of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, any of the training methods or region detection methods in the above embodiments is implemented.
[0153] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0154] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0155] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0156] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.
[0157] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A method for training a region detection model, characterized in that: The method comprises: Acquire multiple training samples, wherein the training samples include a target image and an annotated image thereof, wherein the target image includes a drivable area, and the annotated image is obtained by annotating the drivable area in the target image. The target image in the training sample is input into a preset area detection model, the drivable area in the target image is detected by the area detection model, and a predicted image is output, wherein the predicted image is an image with the predicted drivable area marked. extracting the edge of the predicted drivable area in the predicted image to obtain a first edge image, and extracting the edge of the drivable area in the annotated image to obtain a second edge image, determining an edge loss function value according to the first edge image and the second edge image, When the edge loss function value does not meet the training stop condition, the model parameters of the region detection model are adjusted, and the region detection model after the parameter adjustment is trained using the training samples until the training stop condition is met, thereby obtaining the trained region detection model.
2. The method according to claim 1, characterized in that The step of extracting the edge of the predicted drivable area in the predicted image to obtain a first edge image, and extracting the edge of the drivable area in the annotated image to obtain a second edge image, comprises: Convolution operations are performed on the predicted image and the annotated image respectively to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image.
3. The method according to claim 2, characterized in that The performing convolution operations on the predicted image and the annotated image respectively to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image includes: Get the Laplacian operator template, The Laplacian operator template is used as the weight in the convolution kernel to perform convolution operations on the predicted image and the annotated image respectively, so as to obtain a first edge image corresponding to the predicted image and a second edge image corresponding to the annotated image.
4. The method according to claim 3, characterized in that: The obtaining of the Laplace operator template comprises: Determine the second-order derivative formulas of x and y in the pixel coordinates (x, y), respectively, Substitute the second-order derivative formulas of x and y into the Laplace operator formula to obtain a Laplace operator template.
5. The method according to claim 4, characterized in that The Laplace operator formula is: The second-order derivative formulas for x and y are: f(x+1,y)+f(x-1,y)-2f(x,y) f(x,y+1)+f(x,y-1)-2f(x,y) The Laplace operator template is: Where Δf is the Laplace operator, (x, y) is the pixel coordinate, is the Laplacian operator template.
6. The method according to claim 1, characterized in that The first edge image and the second edge image are both mask images, and determining the edge loss function value according to the first edge image and the second edge image includes: The second edge image is converted into a grayscale image, wherein the grayscale value of each pixel in the grayscale image is the closest Euclidean distance from the pixel to the edge line included in the second edge image, Multiplying the grayscale values of a plurality of pixels in the grayscale image with the pixel values of the pixels at corresponding positions in the first edge image to obtain a plurality of products, An average value of the multiple products is determined as the edge loss function value.
7. The method according to claim 1, characterized in that The region detection model is a convolutional neural network.
8. A region detection method, characterized in that: The method comprises: Acquire a first image, wherein the first image includes a drivable area, The first image is input into a region detection model, and the drivable area in the first image is detected by the region detection model, and a second image is output, wherein the second image is obtained by marking the drivable area in the first image, and the region detection model is trained based on the training method of the region detection model described in any one of claims 1-7.
9. A training device for a region detection model, characterized in that: The device comprises: The first acquisition module is used to acquire multiple training samples, wherein the training samples include a target image and an annotated image thereof, wherein the target image includes a drivable area, and the annotated image is obtained by annotating the drivable area in the target image. An input module is used to input the target image in the training sample into a preset area detection model, detect the drivable area in the target image through the area detection model, and output a predicted image, wherein the predicted image is an image with the predicted drivable area marked. an extraction module, configured to extract the edge of the predicted drivable area in the predicted image to obtain a first edge image, and to extract the edge of the drivable area in the annotated image to obtain a second edge image, a determination module, configured to determine an edge loss function value according to the first edge image and the second edge image, The adjustment module is used to adjust the model parameters of the region detection model when the edge loss function value does not meet the training stop condition, and use the training sample to train the region detection model with adjusted parameters until the training stop condition is met, so as to obtain the trained region detection model.
10. A region detection device, characterized in that: The device comprises: A first acquisition module is used to acquire a first image, wherein the first image contains a drivable area. A detection module is used to input the first image into a region detection model, detect the drivable area in the first image through the region detection model, and output a second image, wherein the second image is obtained by marking the drivable area in the first image, and the region detection model is trained based on the training method of the region detection model described in any one of claims 1-7.