Image processing method and device
By detecting the grounding segments on the vehicle peripheral images and determining their direction characteristics, a target feature map marked with the direction of the segments is generated, which solves the problem of low detection accuracy of grounding segments in the prior art, and improves the accuracy of target object detection.
Patent Information
- Application Number
- CN202411831147.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-11
AI Technical Summary
When the prior art detects the grounding segment on an image, it can only obtain the feature map of the grounding segment, but cannot obtain its directional features, resulting in low detection accuracy of the target object.
By detecting the grounding segments on the vehicle peripheral images, a feature area is obtained, and the line segment direction is determined based on the inclination angle of the grounding segment in the image coordinate system, and a target feature map marked with the line segment direction is generated.
By including feature maps that include ground segment direction features, the accuracy of target object detection is improved, so that target object detection can be performed more accurately.
Smart Images
Figure CN119810791A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image detection technology, and in particular to an image processing method and device. Background Art
[0002] When detecting targets around a vehicle, ground line segment detection is usually performed on the captured surrounding image first, so that subsequent target detection can be performed through the ground line segment of the target in the image, thereby improving the detection accuracy of the target.
[0003] In the related art, grounding line segment detection on an image is to perform image recognition on the image to determine the grounding line segment of the target object in the image and mark it, so as to use the feature map of the grounding line segment for subsequent target detection. However, this method can only obtain the feature map of the grounding line segment of the target object in the image, and cannot obtain other feature information of the grounding line segment, which affects the detection accuracy of the target object detection using the feature map of the grounding line segment. Summary of the invention
[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes an image processing method that can include the directional features of the grounding line segment in the feature map of the grounding line segment, thereby improving the detection accuracy of the target object detection using the feature map of the grounding line segment.
[0005] According to the first aspect of the present application, the image processing method includes:
[0006] Performing grounding line segment detection on the acquired surrounding image of the vehicle to obtain a characteristic area of the grounding line segment of any target object in the surrounding image;
[0007] Obtaining a line segment direction of any grounding line segment in the feature area according to an inclination angle of the grounding line segment in the image coordinate system of the feature area;
[0008] According to the line segment directions of the grounding line segments and the characteristic regions, a target characteristic map marked with the line segment directions of the grounding line segments is obtained.
[0009] By performing grounding line segment detection on the acquired peripheral images of the vehicle, a feature area of a grounding line segment of any target object in the peripheral images is obtained, and based on the inclination angle of any grounding line segment in the feature area in the image coordinate system of the feature area, the line segment direction of the grounding line segment is obtained, so as to obtain a target feature map marked with the line segment direction of each grounding line segment according to the line segment direction of each grounding line segment and the feature area, thereby obtaining a feature map marked with the line segment direction of the grounding line segment, so that the feature map of the grounding line segment includes the directional features of the grounding line segment, so that when using the feature map of the grounding line segment to detect the target object, the grounding line segment and its directional features can be combined to detect the target object, thereby improving the detection accuracy of the target object detection using the feature map of the grounding line segment.
[0010] According to an embodiment of the present application, obtaining the line segment direction of the grounding line segment according to the inclination angle of any grounding line segment in the feature area in the image coordinate system of the feature area includes:
[0011] Inputting the feature area into a first image detection model for detecting the inclination angle of the grounding line segment in the image coordinate system to obtain the line segment direction of the grounding line segment;
[0012] The first image detection model is obtained by training each first image sample marked with the inclination angle of the grounding line segment.
[0013] According to an embodiment of the present application, obtaining a target feature map marked with the line segment directions of each of the grounding line segments according to the line segment directions of each of the grounding line segments and the feature area includes:
[0014] According to the line segment directions of the grounding line segments and the characteristic regions, a marked characteristic graph marked with the line segment directions of the grounding line segments is obtained;
[0015] Obtaining the target feature map according to the marked feature map and the attribute information of each ground line segment in the feature area;
[0016] The attribute information is used to indicate the relative position of the grounding line segment and the vehicle.
[0017] According to one embodiment of the present application, it also includes:
[0018] Inputting the feature area into a second image detection model for detecting a relative position between the grounding line segment and the vehicle to obtain attribute information of the grounding line segment;
[0019] The second image detection model is obtained by training each second image sample marked with the relative position of the grounding line segment and the vehicle.
[0020] According to one embodiment of the present application, it also includes:
[0021] Each of the second image samples is sequentially input into the second image detection model for training. Each time, the second image detection model is adjusted according to the comparison result of the attribute information of each of the grounding wire segments output by the second image detection model and the relative position of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, until the attribute information of each of the grounding wire segments output by the second image detection model each time is consistent with the relative position of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, thereby obtaining a trained second image detection model.
[0022] According to one embodiment of the present application, it also includes:
[0023] According to the target feature map, the feature area in the surrounding image is replaced to obtain a target surrounding image.
[0024] According to an embodiment of the present application, replacing the feature area in the surrounding image according to the target feature map to obtain the target surrounding image includes:
[0025] Performing image compensation on the ground line segment in the target feature map to obtain a compensated feature map;
[0026] The characteristic region in the peripheral image is replaced according to the compensated characteristic map to obtain a target peripheral image.
[0027] An image processing device according to an embodiment of the second aspect of the present application includes:
[0028] A feature region acquisition module, used to perform grounding line segment detection on the acquired surrounding image of the vehicle, and acquire a feature region of the grounding line segment of any target object in the surrounding image;
[0029] A line segment direction determination module, used for obtaining the line segment direction of the grounding line segment according to the inclination angle of any grounding line segment in the feature area in the image coordinate system of the feature area;
[0030] The feature image processing module is used to obtain a target feature image marked with the line segment directions of each of the grounding line segments according to the line segment directions of each of the grounding line segments and the feature areas.
[0031] An electronic device according to an embodiment of the third aspect of the present application includes a processor and a memory storing a computer program, and when the processor executes the computer program, the image processing method described in any of the above embodiments is implemented.
[0032] According to the computer-readable storage medium of the fourth aspect of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, the image processing method described in any of the above embodiments is implemented.
[0033] The vehicle according to the fifth embodiment of the present application includes the electronic device described in the above embodiment. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0035] Figure 1 A schematic diagram of the process of an image processing method provided in an embodiment of the present application;
[0036] Figure 2 is a schematic diagram of a peripheral image provided by an embodiment of the present application;
[0037] Figure 3 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;
[0038] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0040] The image processing method and device provided in the embodiments of the present application will be introduced and explained in detail through several specific embodiments.
[0041] When detecting targets around a vehicle, ground line segment detection is usually performed on the captured surrounding image first, so that subsequent target detection can be performed through the ground line segment of the target in the image, thereby improving the detection accuracy of the target.
[0042] In the related art, the grounding line segment detection of an image is to perform image recognition on the image to determine the grounding line segment of the target object in the image and mark it, so as to use the image with the marked grounding line segment for subsequent target detection. However, this method can only determine the grounding line segment of the target object in the image, and cannot obtain other characteristic information of the grounding line segment, resulting in the inability to affect the detection accuracy of the target object detection using the grounding line segment.
[0043] To this end, in one embodiment, an image processing method is provided, which is applied to an electronic device for performing image processing. The electronic device may include a controller, which may be a controller mounted on a vehicle, such as a VCU (Vehicle Control Unit) or MCU (Microcontroller Unit) of the vehicle.
[0044] like Figure 1 As shown, an image processing method provided by this embodiment includes:
[0045] Step 101, performing grounding line segment detection on the acquired surrounding image of the vehicle, and acquiring a characteristic region of the grounding line segment of any target object in the surrounding image;
[0046] Step 102, obtaining a line segment direction of any grounding line segment in the feature region according to an inclination angle of the grounding line segment in the image coordinate system of the feature region;
[0047] Step 103: obtaining a target feature map marked with the line segment directions of the grounding line segments according to the line segment directions of the grounding line segments and the feature regions.
[0048] In some embodiments, the peripheral image of the vehicle may be an image captured by an image sensor mounted on the vehicle, such as a camera or a laser radar, which may be a two-dimensional image or a three-dimensional image. The number of peripheral images may be one or more. For example, if the acquisition area is a smaller angle range in front of the vehicle, then at the current position of the vehicle, an image sensor may be used to perform image acquisition processing on the front of the vehicle to obtain a peripheral image. Alternatively, if the acquisition area is a larger angle range around the vehicle, such as a full angle range, then at the current position of the vehicle, multiple image sensors facing different directions may be used to perform image acquisition processing to obtain multiple peripheral images.
[0049] After acquiring the surrounding image of the vehicle through the image sensor, the grounding line segment detection can be performed on the surrounding image. For example, the drivable area detection algorithm can be used to detect the grounding line segment of the surrounding image to obtain the grounding line segment of any target object in the surrounding image, so as to divide the area where all the grounding line segments of the target object are located into the characteristic area of the grounding line segment. If a target object in the surrounding image includes two grounding line segments, the area where these two grounding line segments are located is divided into the characteristic area of the grounding line segment. The grounding line segment refers to the line segment formed by the target object in the surrounding image and the ground of the surrounding image. For example, Figure 2 As shown, the line segment A formed by the vehicle and the ground in the two-dimensional image is the grounding line segment, and the area where all the grounding line segments of the vehicle are located is a feature area. If there are multiple targets in the surrounding image, multiple feature areas with grounding line segments can be obtained accordingly.
[0050] In some embodiments, for a certain feature region, an image coordinate system of the feature region may be constructed first, for example, the center point of the feature region may be used as the origin, the length of the feature region may be used as the X-axis direction, and the width may be used as the Y-axis direction to construct the image coordinate system of the feature region. After the image coordinate system of the feature region is constructed, the angle between any grounding line segment in the feature region and the X-axis of the image coordinate system may be detected as the inclination angle in the image coordinate system of the feature region, and the inclination angle may be determined as the line segment direction of the grounding line segment.
[0051] After determining the line segment directions of each grounding line segment in the feature area, the line segment directions of each grounding line segment can be marked one by one for each grounding line segment in the feature area, so as to obtain a target feature map based on the feature map marked with the line segment directions of each grounding line segment, such as determining the feature map marked with the line segment directions of each grounding line segment as the target feature map, so as to use the target feature map for subsequent target detection.
[0052] By performing grounding line segment detection on the acquired peripheral images of the vehicle, a feature area of a grounding line segment of any target object in the peripheral images is obtained, and based on the inclination angle of any grounding line segment in the feature area in the image coordinate system of the feature area, the line segment direction of the grounding line segment is obtained, so as to obtain a target feature map marked with the line segment direction of each grounding line segment according to the line segment direction of each grounding line segment and the feature area, thereby obtaining a feature map marked with the line segment direction of the grounding line segment, so that the feature map of the grounding line segment includes the directional features of the grounding line segment, so that when using the feature map of the grounding line segment to detect the target object, the grounding line segment and its directional features can be combined to detect the target object, thereby improving the detection accuracy of the target object detection using the feature map of the grounding line segment.
[0053] In order to improve the detection efficiency of the line segment direction of the grounding line segment, in some embodiments, according to the inclination angle of any of the grounding line segments in the feature area in the image coordinate system of the feature area, the line segment direction of the grounding line segment is obtained, including:
[0054] Inputting the feature area into a first image detection model for detecting the inclination angle of the grounding line segment in the image coordinate system to obtain the line segment direction of the grounding line segment;
[0055] The first image detection model is obtained by training each first image sample marked with the inclination angle of the grounding line segment.
[0056] In some embodiments, the first image detection model may be a Resnet network or an FPN pooling pyramid network. For the training of the first image detection model, the first image sample with the grounding line segment of the target object may be first constructed with the center point as the origin, the length as the X-axis direction, and the width as the Y-axis direction, and the angle between the grounding line segment and the X-axis in the first image sample is marked as the inclination angle of the grounding line segment to obtain the first image sample marked with the inclination angle of the grounding line segment.
[0057] After obtaining each first image sample, each first image sample can be input into the first image detection model for training. Each training is performed by inputting the first image sample into the first image detection model, obtaining the predicted inclination angle of each grounding line segment in the first image sample output by the first image detection model, and then matching the predicted inclination angle of each grounding line segment with the inclination angle marked on each grounding line segment. In the case where the two do not match, the gradient descent method is used to adjust the network parameters of the first image detection model through error back propagation, and then the next training is performed until the predicted inclination angle of each grounding line segment obtained by inputting the first image sample each time matches the inclination angle marked on each grounding line segment in the first image sample input this time, which means that the training of the first image detection model is completed and a trained first image detection model is obtained.
[0058] After obtaining the trained first image detection model, the feature area of the grounding line segment of any target object in the surrounding image can be input into the first image detection model to obtain the inclination angle of each grounding line segment in the feature area, and use the inclination angle of each grounding line segment as the segment direction of each grounding line segment.
[0059] In order to make the information marked in the feature map more comprehensive and to further improve the detection accuracy of the target object detection using the feature map of the grounding line segment, in some embodiments, according to the line segment direction of each of the grounding line segments and the feature area, a target feature map marked with the line segment direction of each of the grounding line segments is obtained, including:
[0060] According to the line segment directions of the grounding line segments and the characteristic regions, a marked characteristic graph marked with the line segment directions of the grounding line segments is obtained;
[0061] Obtaining the target feature map according to the marked feature map and the attribute information of each ground line segment in the feature area;
[0062] The attribute information is used to indicate the relative position of the grounding line segment and the vehicle.
[0063] In some embodiments, after determining the line segment directions of each grounding line segment in the feature area, the line segment directions of each grounding line segment can be marked one-to-one for each grounding line segment in the feature area to obtain a feature map marked with the line segment directions of each grounding line segment as a marked feature map.
[0064] At the same time, the relative position of any grounding wire segment and the vehicle can be obtained, such as the relative position of any grounding wire segment and the center point of the vehicle, and the relative position can be used as the attribute information of the grounding wire segment. The relative position refers to the position of the grounding wire segment of the target object relative to the vehicle. For example, if the grounding wire segment is located on the left front side of the vehicle, the attribute information of the grounding wire segment can be determined as "left front"; if the grounding wire segment is located on the right side of the vehicle, the attribute information of the grounding wire segment can be determined as "right"; if the grounding wire segment is located directly in front of the vehicle, the attribute information of the grounding wire segment can be determined as "front", and so on. For example, Figure 2 As shown, the grounding wire segment of the target object is located in front of the vehicle, and the attribute information of the grounding wire segment can be determined as "front".
[0065] After obtaining the attribute information of each grounding line segment in the feature area, the attribute information of each grounding line segment can be used to mark each grounding line segment in the marking feature map one by one, so as to obtain a feature map marked with the line segment direction and attribute information of each grounding line segment as the target feature map. In this way, the target feature map obtained can be marked with multiple features of the grounding line segment, thereby improving the detection accuracy of the subsequent target object detection using the target feature map.
[0066] In order to improve the efficiency of acquiring the attribute information of the grounding wire segment, in some embodiments, the characteristic area of the target object can be input into a second image detection model used to detect the relative position of the grounding wire segment and the vehicle to obtain the attribute information of the grounding wire segment; wherein the second image detection model is trained by each second image sample marked with the relative position of the grounding wire segment and the vehicle.
[0067] In some embodiments, the second image detection model may be a Resnet network or an FPN pooling pyramid network. For training the second image detection model, the relative position of the grounding wire segment in the second image sample with the grounding wire segment of the target object may be marked first to obtain the second image sample marked with the relative position of the grounding wire segment and the vehicle.
[0068] After obtaining each second image sample, each second image sample can be input into the second image detection model for training. Each training is performed by inputting the second image sample into the second image detection model, obtaining the attribute information of each grounding line segment in the second image sample output by the second image detection model, and then matching the attribute information of each grounding line segment obtained by inputting the second image sample with the relative position marked by each grounding line segment. In the case where the two do not match, the gradient descent method is used to adjust the network parameters of the second image detection model through error back propagation, and then the next training is performed until the attribute information of each grounding line segment obtained by inputting the second image sample each time matches the relative position marked by each grounding line segment in the second image sample input this time, which means that the training of the second image detection model is completed and a trained second image detection model is obtained.
[0069] After obtaining the trained second image detection model, the feature area of the grounding line segment of any target object in the surrounding image can be input into the second image detection model to obtain the relative positions of each grounding line segment in the feature area, so that the relative positions of each grounding line segment can be used as the attribute information of each grounding line segment.
[0070] In order to improve the detection accuracy of the attribute information of the grounding wire segment by the second image model, in some embodiments, each of the second image samples can be input into the second image detection model for training in turn, and the second image detection model is adjusted each time according to the comparison result of the attribute information of each of the grounding wire segments output by the second image detection model and the relative positions of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, until the attribute information of each of the grounding wire segments output by the second image detection model each time is consistent with the relative positions of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, thereby obtaining a trained second image detection model.
[0071] In some embodiments, the training of the second image detection model may be to first mark the relative positions of the endpoints of the grounding line segment in the second image sample with the grounding line segment of the target object, so as to obtain the second image sample with the relative positions of the endpoints of the grounding line segment and the vehicle marked. If the endpoints on both sides of the grounding line segment are located at the left front side and the left rear side of the vehicle, respectively, the relative positions of the endpoints on both sides of the grounding line segment may be marked as "left front" and "left rear", respectively.
[0072] After obtaining each second image sample, each second image sample can be input into the second image detection model for training. Each training is performed by inputting the second image sample into the second image detection model, and obtaining the attribute information of the two side endpoints of each grounding line segment in the second image sample output by the second image detection model, and then matching the attribute information of the two side endpoints of each grounding line segment with the relative positions marked on the two side endpoints of each grounding line segment. In the case where the two do not match, the gradient descent method is used to adjust the network parameters of the second image detection model through error back propagation, and then the next training is performed until the attribute information of the two side endpoints of each grounding line segment obtained by inputting the second image sample each time matches the relative positions marked on the two side endpoints of each grounding line segment in the second image sample input this time, which indicates that the training of the second image detection model is completed and a trained second image detection model is obtained. By training the second image detection model based on the relative position of the endpoint mark of the grounding line segment, there is no need to consider other pixel points of the grounding line segment, thereby avoiding the influence of other pixel points in the grounding line segment on the training results, and avoiding the influence of fluctuations of other pixel points in the grounding line segment on the training results and the detection results. At the same time, it reduces the amount of data that the model needs to process, thereby improving the training efficiency and subsequent detection efficiency.
[0073] After obtaining the trained second image detection model, the feature area of the grounding line segment of any target object in the surrounding image can be input into the second image detection model to obtain the relative positions of each grounding line segment in the feature area, so that the relative positions of each grounding line segment can be used as the attribute information of each grounding line segment.
[0074] After obtaining the attribute information of each grounding line segment in the feature area, the attribute information of each grounding line segment and the line segment direction of each grounding line segment can be marked one by one for each grounding line segment in the feature map to obtain a feature map marked with the line segment direction and attribute information of each grounding line segment as the target feature map.
[0075] Considering that there may be more than one target object in the surrounding image, and the environment in which the target object is located usually needs to be considered when detecting the target object, in order to facilitate subsequent target detection, in some embodiments, the feature area in the surrounding image can be replaced according to the target feature map to obtain a target surrounding image.
[0076] Exemplarily, after obtaining the target feature map, since the target feature map may have been cropped or scaled, the target feature map may be compared with the feature region, and the target feature map may be enlarged according to the ratio of the target feature map to the feature region to make it the same size as the feature region, and the target feature map of the same size as the feature region may be mapped to the feature region in the surrounding image to replace the feature region in the surrounding image. After completing the replacement of all feature regions in the surrounding image, a target surrounding image marked with the line segment direction and attribute information of each grounding line segment may be obtained, thereby facilitating the subsequent detection of the target object through the target surrounding image.
[0077] When mapping the target feature map back to its corresponding feature area, it is usually necessary to enlarge the target feature map. After the target feature map is enlarged, its ground line segment may be distorted. To this end, in some embodiments, according to the target feature map, the feature area in the surrounding image is replaced to obtain the target surrounding image, including:
[0078] Performing image compensation on the ground line segment in the target feature map to obtain a compensated feature map;
[0079] The characteristic region in the peripheral image is replaced according to the compensated characteristic map to obtain a target peripheral image.
[0080] In some embodiments, for the target feature map, the target feature map may be enlarged according to the ratio of the target feature map to the feature area to obtain an enlarged target feature map. However, the ground line segment of the enlarged target feature map may be distorted, such as changing into a sawtooth shape, so image compensation may be performed on the ground line segment in the enlarged target feature map to obtain a compensated feature map.
[0081] Exemplarily, the grounding line segment can be compensated based on the absolute difference between the position of each pixel point of the grounding line segment in the target feature map before amplification in the image coordinate system and the position of each pixel point of the grounding line segment in the target feature map after amplification in the image coordinate system, and the grounding line segment can be compensated as the compensation value of each pixel point to obtain a compensated feature map, so that the grounding line segment is a smooth line segment in the compensated feature map, thereby avoiding the subsequent influence of the grounding line segment distortion on the detection of the target object. Among them, the grounding line segment can be compensated by Gaussian Focal Loss. In order to prevent the compensation value of two adjacent pixel points in the grounding line segment from changing suddenly, the difference of the compensation value of the adjacent pixel points can be calculated by Gaussian Focal Loss again to ensure that the compensated grounding line is more accurate.
[0082] After obtaining the compensated feature map, the compensated feature map can be mapped to the corresponding feature area in the surrounding image, replacing the feature area in the surrounding image to obtain the target surrounding image.
[0083] The image processing device provided by the present application is described below. The image processing device described below and the image processing method described above can be referenced to each other.
[0084] In one embodiment, if Figure 3 As shown, an image processing device is provided, comprising:
[0085] The feature region acquisition module 210 is used to perform grounding line segment detection on the acquired surrounding image of the vehicle, and acquire the feature region of the grounding line segment of any target object in the surrounding image;
[0086] A line segment direction determination module 220, configured to obtain a line segment direction of any grounding line segment in the feature region according to an inclination angle of the grounding line segment in the image coordinate system of the feature region;
[0087] The feature image processing module 230 is used to obtain a target feature image marked with the line segment directions of each of the grounding line segments according to the line segment directions of each of the grounding line segments and the feature areas.
[0088] By performing grounding line segment detection on the acquired peripheral images of the vehicle, a feature area of a grounding line segment of any target object in the peripheral images is obtained, and based on the inclination angle of any grounding line segment in the feature area in the image coordinate system of the feature area, the line segment direction of the grounding line segment is obtained, so as to obtain a target feature map marked with the line segment direction of each grounding line segment according to the line segment direction of each grounding line segment and the feature area, thereby obtaining a feature map marked with the line segment direction of the grounding line segment, so that the feature map of the grounding line segment includes the directional features of the grounding line segment, so that when using the feature map of the grounding line segment to detect the target object, the grounding line segment and its directional features can be combined to detect the target object, thereby improving the detection accuracy of the target object detection using the feature map of the grounding line segment.
[0089] In one embodiment, the line segment direction determination module 220 is specifically used for:
[0090] Inputting the feature area into a first image detection model for detecting the inclination angle of the grounding line segment in the image coordinate system to obtain the line segment direction of the grounding line segment;
[0091] The first image detection model is obtained by training each first image sample marked with the inclination angle of the grounding line segment.
[0092] In one embodiment, the feature image processing module 230 is specifically used for:
[0093] According to the line segment directions of the grounding line segments and the characteristic regions, a marked characteristic graph marked with the line segment directions of the grounding line segments is obtained;
[0094] Obtaining the target feature map according to the marked feature map and the attribute information of each ground line segment in the feature area;
[0095] The attribute information is used to indicate the relative position of the grounding line segment and the vehicle.
[0096] In one embodiment, the feature image processing module 230 is further used for:
[0097] Inputting the feature area into a second image detection model for detecting a relative position between the grounding line segment and the vehicle to obtain attribute information of the grounding line segment;
[0098] The second image detection model is obtained by training each second image sample marked with the relative position of the grounding line segment and the vehicle.
[0099] In one embodiment, the feature image processing module 230 is further used for:
[0100] Each of the second image samples is sequentially input into the second image detection model for training. Each time, the second image detection model is adjusted according to the comparison result of the attribute information of each of the grounding wire segments output by the second image detection model and the relative position of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, until the attribute information of each of the grounding wire segments output by the second image detection model each time is consistent with the relative position of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, thereby obtaining a trained second image detection model.
[0101] In one embodiment, the feature image processing module 230 is further used for:
[0102] According to the target feature map, the feature area in the surrounding image is replaced to obtain a target surrounding image.
[0103] In one embodiment, the feature image processing module 230 is specifically used for:
[0104] Performing image compensation on the ground line segment in the target feature map to obtain a compensated feature map;
[0105] The characteristic region in the peripheral image is replaced according to the compensated characteristic map to obtain a target peripheral image.
[0106] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call a computer program in the memory 830 to execute an image processing method, for example, including:
[0107] Performing grounding line segment detection on the acquired surrounding image of the vehicle to obtain a characteristic area of the grounding line segment of any target object in the surrounding image;
[0108] Obtaining a line segment direction of any grounding line segment in the feature area according to an inclination angle of the grounding line segment in the image coordinate system of the feature area;
[0109] According to the line segment directions of the grounding line segments and the characteristic regions, a target characteristic map marked with the line segment directions of the grounding line segments is obtained.
[0110] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0111] On the other hand, an embodiment of the present application further provides a storage medium, the storage medium includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the image processing method provided by the above embodiments, for example, including:
[0112] Performing grounding line segment detection on the acquired surrounding image of the vehicle to obtain a characteristic area of the grounding line segment of any target object in the surrounding image;
[0113] Obtaining a line segment direction of any grounding line segment in the feature area according to an inclination angle of the grounding line segment in the image coordinate system of the feature area;
[0114] According to the line segment directions of the grounding line segments and the characteristic regions, a target characteristic map marked with the line segment directions of the grounding line segments is obtained.
[0115] On the other hand, an embodiment of the present application further provides a vehicle, which includes the electronic device as described in the above embodiment.
[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An image processing method, characterized in that: include: Performing grounding line segment detection on the acquired surrounding image of the vehicle to obtain a characteristic area of the grounding line segment of any target object in the surrounding image; Obtaining a line segment direction of any grounding line segment in the feature area according to an inclination angle of the grounding line segment in the image coordinate system of the feature area; According to the line segment directions of the grounding line segments and the characteristic regions, a target characteristic map marked with the line segment directions of the grounding line segments is obtained.
2. The image processing method according to claim 1, characterized in that: Obtaining the line segment direction of any grounding line segment in the feature area according to the inclination angle of the grounding line segment in the image coordinate system of the feature area includes: Inputting the feature area into a first image detection model for detecting the inclination angle of the grounding line segment in the image coordinate system to obtain the line segment direction of the grounding line segment; The first image detection model is obtained by training each first image sample marked with the inclination angle of the grounding line segment.
3. The image processing method according to claim 1, characterized in that: According to the line segment directions of the grounding line segments and the characteristic regions, a target characteristic map marked with the line segment directions of the grounding line segments is obtained, including: According to the line segment directions of the grounding line segments and the characteristic regions, a marked characteristic graph marked with the line segment directions of the grounding line segments is obtained; Obtaining the target feature map according to the marked feature map and the attribute information of each ground line segment in the feature area; The attribute information is used to indicate the relative position of the grounding line segment and the vehicle.
4. The image processing method according to claim 3, characterized in that: Also includes: Inputting the feature area into a second image detection model for detecting a relative position between the grounding line segment and the vehicle to obtain attribute information of the grounding line segment; The second image detection model is obtained by training each second image sample marked with the relative position of the grounding line segment and the vehicle.
5. The image processing method according to claim 4, characterized in that: Also includes: Each of the second image samples is sequentially input into the second image detection model for training. Each time, the second image detection model is adjusted according to the comparison result of the attribute information of each of the grounding wire segments output by the second image detection model and the relative position of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, until the attribute information of each of the grounding wire segments output by the second image detection model each time is consistent with the relative position of the endpoint marks on both sides of each of the grounding wire segments in the second image sample currently input into the second image detection model, thereby obtaining a trained second image detection model.
6. The image processing method according to any one of claims 1 to 5, characterized in that: Also includes: According to the target feature map, the feature area in the surrounding image is replaced to obtain a target surrounding image.
7. The image processing method according to claim 6, characterized in that: Replacing the feature region in the surrounding image according to the target feature map to obtain a target surrounding image includes: Performing image compensation on the ground line segment in the target feature map to obtain a compensated feature map; The characteristic region in the peripheral image is replaced according to the compensated characteristic map to obtain a target peripheral image.
8. An image processing device, characterized in that: include: A feature region acquisition module, used to perform grounding line segment detection on the acquired surrounding image of the vehicle, and acquire a feature region of the grounding line segment of any target object in the surrounding image; A line segment direction determination module, used for obtaining the line segment direction of the grounding line segment according to the inclination angle of any grounding line segment in the feature area in the image coordinate system of the feature area; The feature image processing module is used to obtain a target feature image marked with the line segment directions of each of the grounding line segments according to the line segment directions of each of the grounding line segments and the feature areas.
9. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the image processing method according to any one of claims 1 to 7 is implemented.
10. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 9.