A training method, model and storage medium for lower edge detection model based on target detection
By adopting a lower edge detection model based on object detection in the autonomous driving system, combining neural network and feature extraction module, the shortcomings of traditional edge detection algorithms in terms of anti-interference and noise processing are solved, and the accuracy of autonomous driving vehicles is achieved Edge detection improves the accuracy of driving trajectory prediction and body control.
Patent Information
- Application Number
- CN201910158356.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-03-01
- Publication Date
- 2024-11-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional edge detection algorithms have shortcomings in anti-interference and noise processing, resulting in blurred image edges and cannot be effectively extracted at multiple scales, affecting the driving trajectory prediction and body control of autonomous driving vehicles.
The lower edge detection model based on object detection is adopted, and the basic neural network, height graph neural network and lower edge neural network are synchronously segmented training, and the model is adjusted using appropriate loss functions to improve training speed and accuracy, combined with feature extraction and fusion modules, Achieve accurate detection of the target lower edge.
It improves the anti-interference ability and accuracy of edge detection, can effectively extract image edges at multiple scales, and enhances support for autonomous driving systems.
Smart Images

Figure CN110148148B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of automobile electronic technology, and in particular to a training method, a model and a storage medium of a lower edge detection model based on target detection. Background Art
[0002] Images are an important source of information for people to obtain from the objective world. Image information mainly comes from its edges and contours. The so-called edge refers to the collection of pixels whose surrounding pixel grayscale changes sharply. It exists between the target, background and area. Image edge detection is an important content and step in image processing.
[0003] In recent years, with the increasing demand for autonomous driving, autonomous parking systems have become a topic of in-depth research. However, during the parking process, obtaining the bottom edges of targets such as vehicles, pedestrians, and obstacles in the current driving area, and then calculating the bottom edge distance between the unmanned vehicle and other targets, plays an important role in vehicle trajectory prediction and body control.
[0004] Traditional edge detection algorithms such as differential operator method and optimal operator method use the maximum value of gradient or the zero-crossing value of second-order derivative to obtain the edge of the image. Although these algorithms have good real-time performance, they have poor anti-interference performance and cannot effectively overcome the influence of noise, resulting in blurred edges after image denoising. They cannot extract image edges at multiple scales, resulting in poor positioning. Summary of the invention
[0005] In order to solve the above-mentioned and other potential technical problems, the present invention provides a training method, model and storage medium for a lower edge detection model based on target detection. Each part is synchronously segmented and trained respectively through a basic neural network, a height map neural network and a lower edge neural network, which saves computational time and improves the training speed of the neural network. Different neural networks use loss functions suitable for the segmented training, which improves the accuracy of each segment of the neural network. The height map neural network part uses the mean square error loss function as a correction of the loss function.
[0006] A method for training a lower edge detection model for target detection comprises the following steps:
[0007] Acquire a sample image, wherein the sample image is annotated with an instance segmentation result, and the instance segmentation result includes a target detection box;
[0008] Inputting the sample image into a pre-established initial neural network model;
[0009] The sample image is used to train the initial neural network model, wherein the initial neural network model includes a basic neural network model, a height map neural network model and a lower edge network model, the sample image is first input into the basic neural network model to obtain an instance segmentation result of target detection, and then the instance segmentation result of target detection is input into the height map neural network model, and then the height map is converted to obtain a target lower edge detection result;
[0010] Compare the result output by the target lower edge detection model during the execution process with each of the target lower edge detection annotation lines to obtain a comparison result;
[0011] The loss function in the target lower edge detection model is adjusted according to the comparison result so that the loss function of the target lower edge detection model is minimized.
[0012] Furthermore, the loss function uses a regression problem loss function based on a distance metric.
[0013] Furthermore, the regression problem loss function based on distance metric uses one or more of a mean square error loss function, a mean absolute value error loss function, a smoothed mean absolute error loss function, a Log-Cosh loss function, and a quantile loss function.
[0014] Furthermore, the distance metric-based regression problem loss function uses a mean square error loss function:
[0015]
[0016] in is the predicted value of the lower edge of the target object, where yi is the calibrated value of the lower edge of the target object.
[0017] Furthermore, the predicted value shape of the lower edge of the target object is 1*W, and the calibrated value of the lower edge of the target object is 1*W.
[0018] Furthermore, the target lower edge detection frame includes the following elements:
[0019] 1) Segment and capture different semantic parts in the image to obtain the category elements of the semantic segmentation in the target detection frame;
[0020] 2) Instance segmentation results of specific categories after semantic segmentation;
[0021] 3) The lower edge contour of the target in each instance segmentation result;
[0022] 4) The contact points with the ground in the edge contour line of each instance segmentation result.
[0023] A lower edge detection model for target detection includes the following parts:
[0024] A target detection module, wherein the target detection module segments the parts of different semantics in the captured image by the initial neural network model to form an instance segmentation result, and then performs instance segmentation on the segmentation result to obtain each instance in different semantics. The target detection module also includes a feature extraction module, which is used to extract the features of the target image in each instance target detection frame to form a feature map;
[0025] A height map extraction module, wherein the bottom edge height map extraction module repeatedly performs convolution operations on the feature map with multiple branches through a height map neural network, and then merges the results of the multi-branch convolution operations to obtain a bottom edge height map with a height dimension of 1 corresponding to the feature map;
[0026] A lower edge extraction module is used to compare the feature information of each pixel point in the bottom edge height map with the feature information of the pixel point column in the original feature map, obtain the correspondence between the pixel points in a single column of the bottom edge height map and the pixel points in the column of the original feature map, obtain the corresponding position relationship between each pixel point in the bottom edge height map in the original feature map, and obtain the lower edge contour of the instance in the target detection frame with the corresponding position relationship.
[0027] Furthermore, the feature extraction module includes but is not limited to extracting features of a specific convolutional layer in the target detection frame.
[0028] Furthermore, the feature extraction module includes extracting features of multiple convolutional layers in the target detection frame respectively, and also includes a feature fusion module, which is used to fuse the features of the multiple convolutional layers extracted respectively, and obtain a feature map of the fused features, and use the feature map of the fused features to replace the original feature map.
[0029] Furthermore, when the bottom edge height map extraction module repeatedly performs convolution operations on the feature map with multiple branches, the dimension number of the convolution kernel of each convolution operation is not specific, but the convolution kernel size (hk, wk) is specific to hk equal to 1, wk equal to 1.
[0030] Furthermore, the targets of the target lower edge detection frame include movable objects and stationary objects. The movable objects include pedestrians, devices held by pedestrians that touch the ground, and mobile devices including but not limited to wheelchairs, balance bikes, bicycles, carts, motorcycles, and motor vehicles; the stationary objects include various ground obstacles including signs, roadblocks, trees, fences, and pillars.
[0031] Furthermore, the target lower edge detection frame includes the following elements:
[0032] A segmentation captures different semantic parts in the image and obtains the category elements of the semantic segmentation in the target detection box;
[0033] B. Instance segmentation results of specific categories after semantic segmentation;
[0034] C. The lower edge contour of the target in each instance segmentation result;
[0035] D. The contact points with the ground in the lower edge contour of each instance segmentation result.
[0036] Furthermore, the feature extraction module includes a preliminary feature extraction part and a complex feature extraction part. The preliminary feature extraction part and the complex feature extraction part serve as two branches and can work simultaneously. The two branches are used for feature extraction, that is, preliminary feature extraction is first performed through a basic network, and then complex feature extraction is performed through a special network. Image features at different stages in the preliminary feature extraction and complex feature extraction will be temporarily saved.
[0037] Furthermore, the preliminary feature extraction part branch content in the feature extraction module can call the preliminary feature extraction result of the feature extraction part in the target detection module.
[0038] Furthermore, the feature extraction module uses the Resnet network to perform preliminary feature extraction on the captured image, and the RRC network then performs more complex feature extraction. The image feature maps at different stages of the Resnet network and RRC network processing will be temporarily saved, and finally the result with the target detection box will be output.
[0039] Furthermore, the target detection module also includes a feature extraction part. The feature extraction part of the target detection module can also use two branches to perform feature extraction, that is, first perform preliminary feature extraction through the basic network, and then perform complex feature extraction through the special network. The image features at different stages of the preliminary feature extraction and complex feature extraction will be temporarily saved.
[0040] A relative position conversion method based on target height map extraction comprises the following steps:
[0041] S01: The target detection module captures the parts of different semantics in the image through the basic neural network segmentation to form a segmentation result, and then performs instance segmentation on the segmentation result to obtain each instance in different semantics, and annotates each instance in the form of a target detection frame, and extracts the features of the target image in each instance target detection frame to form a feature map; the height map neural network repeatedly performs convolution operations on the feature map with multiple branches, and then merges the results of the multi-branch convolution operations, and finally performs a fully connected expansion of the bottom edge height map of the target; the bottom edge extraction module takes the bottom edge height map as input, and inputs the bottom edge height map into the bottom edge neural network, so that the feature information of each pixel point is compared with the feature information of the column of pixels in the original feature map, and the corresponding relationship between the single column of pixels in the bottom edge height map and the column of pixels in the original feature map is obtained, and the corresponding position relationship of each pixel point in the bottom edge height map in the original feature map is obtained, and the bottom edge contour of the instance in the target detection frame is obtained by the corresponding position relationship;
[0042] S02: Extract the pixel point at the lowest position relative to the ground in the lower edge contour of the instance in the target detection frame, determine that the pixel point is the contact point between the object shown in the target detection frame and the ground, use the calibration parameters of the body camera to obtain the projection of the body camera on the ground, and obtain the relative position relationship between the body and the target detection frame through coordinate system conversion.
[0043] As described above, the present invention has the following beneficial effects:
[0044] (1) The target detection module uses the RRC network to enhance the capture of the global context information of the target object and can achieve good detection of objects of different sizes.
[0045] (2) The feature fusion module performs multi-scale feature fusion on the detection box of each detected object and its corresponding feature maps at multiple different stages, making the fused features more representative of the detected object.
[0046] (3) The bottom edge detection module makes extensive use of dual-branch 1x1 convolution operations to improve image accuracy without affecting the time consumption.
[0047] (4) The bottom edge detection image of the object in the top view output by the front fisheye to top view module can realize the accurate distance measurement of the detected target. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0049] Figure 1 Shown are sample images acquired for the present invention.
[0050] Figure 2 Shown is a schematic diagram of the target semantic segmentation result obtained by the present invention after passing through the basic neural network.
[0051] Figure 3 Shown is a schematic diagram of the target instance segmentation result obtained by the present invention through the basic neural network.
[0052] Figure 4 Shown is a schematic diagram of the output result of the lower edge neural network of the present invention.
[0053] Figure 5 Shown is a schematic diagram of the output results of the single-target lower edge neural network of the present invention.
[0054] Figure 6 It shows the feature value of each pixel in the feature map before the height map neural network converts the feature map into the bottom edge height map in one embodiment of the present invention.
[0055] Figure 7 Display as Figure 6 Schematic diagram of the bottom edge height map output by the mid-height map neural network.
[0056] Figure 8 Shown is a schematic diagram of the relative coordinate position relationship between the camera calibration origin and the target detection frame in the top view after matrix transformation. DETAILED DESCRIPTION
[0057] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0058] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention, so they have no substantial technical significance. Any modification of the structure, change of the proportion relationship or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the effects and purposes that can be achieved by the present invention. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" quoted in this specification are only for the convenience of description, and are not used to limit the scope of the implementation of the present invention. The change or adjustment of their relative relationship should also be regarded as the scope of the implementation of the present invention without substantially changing the technical content.
[0059] See also Figures 1 to 8 ,
[0060] A method for training a lower edge detection model for target detection comprises the following steps:
[0061] Acquire a sample image, wherein the sample image is annotated with an instance segmentation result, and the instance segmentation result includes a target detection box;
[0062] Inputting the sample image into a pre-established initial neural network model;
[0063] The sample image is used to train the initial neural network model, wherein the initial neural network model includes a basic neural network model, a height map neural network model and a lower edge network model, the sample image is first input into the basic neural network model to obtain an instance segmentation result of target detection, and then the instance segmentation result of target detection is input into the height map neural network model, and then the height map is converted to obtain a target lower edge detection result;
[0064] Compare the result output by the target lower edge detection model during the execution process with each of the target lower edge detection annotation lines to obtain a comparison result;
[0065] The loss function in the target lower edge detection model is adjusted according to the comparison result so that the loss function of the target lower edge detection model is minimized.
[0066] Furthermore, the loss function uses a regression problem loss function based on a distance metric.
[0067] Furthermore, the regression problem loss function based on distance metric uses one or more of a mean square error loss function, a mean absolute value error loss function, a smoothed mean absolute error loss function, a Log-Cosh loss function, and a quantile loss function.
[0068] Furthermore, the distance metric-based regression problem loss function uses a mean square error loss function:
[0069]
[0070] in is the predicted value of the lower edge of the target object, where yi is the calibrated value of the lower edge of the target object.
[0071] Furthermore, the predicted value shape of the lower edge of the target object is 1*W, and the calibrated value of the lower edge of the target object is 1*W.
[0072] Furthermore, the target lower edge detection frame includes the following elements:
[0073] 1) Segment and capture different semantic parts in the image to obtain the category elements of the semantic segmentation in the target detection frame;
[0074] 2) Instance segmentation results of specific categories after semantic segmentation;
[0075] 3) The lower edge contour of the target in each instance segmentation result;
[0076] 4) The contact points with the ground in the edge contour line of each instance segmentation result.
[0077] A lower edge detection model for target detection includes the following parts:
[0078] A target detection module, wherein the target detection module segments the parts of different semantics in the captured image by the initial neural network model to form an instance segmentation result, and then performs instance segmentation on the segmentation result to obtain each instance in different semantics. The target detection module also includes a feature extraction module, which is used to extract the features of the target image in each instance target detection frame to form a feature map;
[0079] A height map extraction module, wherein the bottom edge height map extraction module repeatedly performs convolution operations on the feature map with multiple branches through a height map neural network, and then merges the results of the multi-branch convolution operations to obtain a bottom edge height map with a height dimension of 1 corresponding to the feature map;
[0080] A lower edge extraction module is used to compare the feature information of each pixel point in the bottom edge height map with the feature information of the pixel point column in the original feature map, obtain the correspondence between the pixel points in a single column of the bottom edge height map and the pixel points in the column of the original feature map, obtain the corresponding position relationship between each pixel point in the bottom edge height map in the original feature map, and obtain the lower edge contour of the instance in the target detection frame with the corresponding position relationship.
[0081] Furthermore, the feature extraction module includes but is not limited to extracting features of a specific convolutional layer in the target detection frame.
[0082] Furthermore, the feature extraction module includes extracting features of multiple convolutional layers in the target detection frame respectively, and also includes a feature fusion module, which is used to fuse the features of the multiple convolutional layers extracted respectively, and obtain a feature map of the fused features, and use the feature map of the fused features to replace the original feature map.
[0083] Further, when the bottom edge height map extraction module repeatedly performs convolution operations on the feature map with multiple branches, the dimension number of the convolution kernel of each convolution operation is not specific, but the convolution kernel size (hk, wk) is specific that hk is equal to 1 and wk is equal to 1. Figure 6-Figure 7 , Figure 6 8*10 represents the size of the feature map output by the basic neural network. The size of the feature map is 8*10, and the value in each square represents the brightness contrast value of the pixel. When the feature map is input into the height neural network, the height neural network model first identifies the pixel points where the target object touches the ground in the feature map, marks the pixel points, and then obtains the points where the target object touches the ground in the feature map, and connects these pixel points one by one to form the lower edge line. The feature map marked with the lower edge line is further reduced in dimension to become a one-dimensional feature map, that is, Figure 7 As shown, it is the output of the height map neural network. The result of the height map neural network output is that the value of each pixel in the row represents the position information of the contact point between the target object and the ground in the corresponding column of the feature map.
[0084] Furthermore, the targets of the target lower edge detection frame include movable objects and stationary objects. The movable objects include pedestrians, devices held by pedestrians that touch the ground, and mobile devices including but not limited to wheelchairs, balance bikes, bicycles, carts, motorcycles, and motor vehicles; the stationary objects include various ground obstacles including signs, roadblocks, trees, fences, and pillars.
[0085] Furthermore, the target lower edge detection frame includes the following elements:
[0086] A segmentation captures different semantic parts in the image and obtains the category elements of the semantic segmentation in the target detection box;
[0087] B. Instance segmentation results of specific categories after semantic segmentation;
[0088] C. The lower edge contour of the target in each instance segmentation result;
[0089] D. The contact points with the ground in the lower edge contour of each instance segmentation result.
[0090] Furthermore, the feature extraction module includes a preliminary feature extraction part and a complex feature extraction part. The preliminary feature extraction part and the complex feature extraction part serve as two branches and can work simultaneously. The two branches are used for feature extraction, that is, preliminary feature extraction is first performed through a basic network, and then complex feature extraction is performed through a special network. Image features at different stages in the preliminary feature extraction and complex feature extraction will be temporarily saved.
[0091] Furthermore, the preliminary feature extraction part branch content in the feature extraction module can call the preliminary feature extraction result of the feature extraction part in the target detection module.
[0092] Furthermore, the feature extraction module uses the Resnet network to perform preliminary feature extraction on the captured image, and the RRC network then performs more complex feature extraction. The image feature maps at different stages of the Resnet network and RRC network processing will be temporarily saved, and finally the result with the target detection box will be output.
[0093] Furthermore, the target detection module also includes a feature extraction part. The feature extraction part of the target detection module can also use two branches to perform feature extraction, that is, first perform preliminary feature extraction through the basic network, and then perform complex feature extraction through the special network. The image features at different stages of the preliminary feature extraction and complex feature extraction will be temporarily saved.
[0094] A relative position conversion method based on target height map extraction comprises the following steps:
[0095] S01: The target detection module captures the parts of different semantics in the image through the basic neural network segmentation to form a segmentation result, and then performs instance segmentation on the segmentation result to obtain each instance in different semantics, and annotates each instance in the form of a target detection frame, and extracts the features of the target image in each instance target detection frame to form a feature map; the height map neural network repeatedly performs convolution operations on the feature map with multiple branches, and then merges the results of the multi-branch convolution operations, and finally performs a fully connected expansion of the bottom edge height map of the target; the bottom edge extraction module takes the bottom edge height map as input, and inputs the bottom edge height map into the bottom edge neural network, so that the feature information of each pixel point is compared with the feature information of the column of pixels in the original feature map, and the corresponding relationship between the single column of pixels in the bottom edge height map and the column of pixels in the original feature map is obtained, and the corresponding position relationship of each pixel point in the bottom edge height map in the original feature map is obtained, and the bottom edge contour of the instance in the target detection frame is obtained by the corresponding position relationship;
[0096] S02: Extract the pixel point at the lowest position relative to the ground in the lower edge contour of the instance in the target detection frame, determine that the pixel point is the contact point between the object shown in the target detection frame and the ground, use the calibration parameters of the body camera to obtain the projection of the body camera on the ground, and obtain the relative position relationship between the body and the target detection frame through coordinate system conversion.
[0097] As a preferred embodiment, this embodiment also provides a terminal device, such as a smart phone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server or cabinet server (including independent servers, or server clusters composed of multiple servers) that can execute programs. The terminal device of this embodiment includes at least but is not limited to: a memory and a processor that can communicate with each other through a system bus. It should be pointed out that the terminal device has component memory and processor, but it should be understood that it is not required to implement all the components shown, and the lower edge detection method of target detection that can be replaced can implement more or fewer components.
[0098] As a preferred embodiment, the memory (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device 20. In other embodiments, the memory can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is generally used to store the operating system and various application software installed on the computer device, such as the lower edge detection program code of the target detection in the embodiment, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or are to be output.
[0099] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chips in some embodiments. The processor is generally used to control the overall operation of a computer device. In the present embodiment, the processor is used to run the program code stored in the memory or process data, such as the detection program of operation, to realize the function of the lower edge detection model of target detection in the embodiment.
[0100] A computer-readable storage medium stores a computer program, wherein the program implements the steps in the above method when executed by a processor.
[0101] This embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, an App application store, etc., on which a computer program is stored, and the program realizes the corresponding function when executed by the processor. The computer-readable storage medium of this embodiment is used to store a lower edge detection program for target detection, and when executed by the processor, the lower edge detection method of target detection in the embodiment is realized.
[0102] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.
Claims
1. A training method for a lower edge detection model for target detection, characterized in that: The following steps are involved: Acquire a sample image, wherein the sample image is annotated with an instance segmentation result, and the instance segmentation result includes a target detection box; Inputting the sample image into a pre-established initial neural network model; The sample image is used to train the initial neural network model, wherein the initial neural network model includes a basic neural network model, a height map neural network model and a lower edge detection model. The sample image is first input into the basic neural network model to obtain an instance segmentation result of target detection. After the instance segmentation result of target detection is input into the height map neural network model, it is input into the lower edge detection model to obtain a target lower edge detection result by converting the height map. Compare the output of the lower edge detection model during the execution process with the corresponding lower edge detection annotation line; Obtaining comparison results; Adjusting the loss function in the lower edge detection model according to the comparison result so that the loss function of the lower edge detection model is minimized; Among them, the basic neural network model is a deep neural network that can extract instance segmentation results; the height map neural network is a deep neural network that can realize feature dimension conversion; the lower edge detection model is a deep neural network that can realize lower edge contour extraction; The height map conversion to obtain the target bottom edge detection result includes the following steps: repeatedly performing convolution operations with multiple branches on the original feature map through the height map neural network, and then merging the multi-branch convolution operation results to obtain a bottom edge height map with a height dimension of 1 corresponding to the feature map; the bottom edge detection model compares the feature information of each pixel point in the bottom edge height map with the feature information of the pixel point in the original feature map, obtains the correspondence between each pixel point in the bottom edge height map and the pixel point in the feature map, and obtains the bottom edge contour of the instance in the target detection frame based on the correspondence; The loss function uses a regression problem loss function based on a distance metric, including one or more of a mean square error loss function, a mean absolute value error loss function, a smoothed mean absolute error loss function, a Log-Cosh loss function, and a quantile loss function.
2. The training method of the lower edge detection model for target detection according to claim 1, characterized in that: The loss function for the regression problem based on distance metric uses the mean square error loss function: in is the predicted value of the lower edge of the target object, where yi is the calibrated value of the lower edge of the target object.
3. The training method of the lower edge detection model for target detection according to claim 2, characterized in that: The predicted value shape of the lower edge of the target object is 1*W, and the calibrated value of the lower edge of the target object is 1*W.
4. A lower edge detection device for target detection, characterized in that: Includes the following sections: A target detection module, wherein the target detection module segments and captures different semantic parts in the image by a basic neural network model to form a semantic segmentation result, and then performs instance segmentation on the segmentation result to obtain each instance in different semantics. The target detection module also includes a feature extraction module, which is used to extract the features of the target image in each instance target detection frame to form a feature map; A height map extraction module, which repeatedly performs convolution operations on the feature map with multiple branches through a height map neural network, and then merges the results of the multi-branch convolution operations to obtain a bottom edge height map with a height dimension of 1 corresponding to the feature map; A lower edge extraction module is used to compare the feature information of each pixel in the bottom edge height map with the feature information of the pixel column in the original feature map, obtain the corresponding relationship between the pixel columns in the bottom edge height map and the pixel columns in the original feature map, obtain the corresponding position relationship between each pixel in the bottom edge height map and the original feature map, and obtain the lower edge contour of the instance in the target detection frame by using the corresponding position relationship; Among them, the basic neural network model is a deep neural network that can extract features and instance segmentation results; the height graph neural network is a deep neural network that can realize feature dimension conversion.
5. The lower edge detection device for target detection according to claim 4, characterized in that: The feature extraction module includes but is not limited to extracting features of a specific convolutional layer in the target detection frame.
6. The lower edge detection device for target detection according to claim 5, characterized in that: The feature extraction module includes extracting features of multiple convolutional layers in the target detection frame respectively, and also includes a feature fusion module, which is used to fuse the multiple convolutional layer features extracted respectively, and obtain a feature map of the fused features, and use the feature map of the fused features to replace the original feature map.
7. A relative position conversion method based on target height map extraction, characterized in that: The following steps are involved: S01: The target detection module captures the parts of different semantics in the image through the basic neural network segmentation to form a segmentation result, and then performs instance segmentation on the segmentation result to obtain each instance in different semantics, and annotates each instance in the form of a target detection frame, and extracts the features of the target image in each instance target detection frame to form a feature map; the height map neural network repeatedly performs convolution operations on the feature map with multiple branches, and then merges the results of the multi-branch convolution operations, and finally performs a fully connected expansion of the bottom edge height map of the target; the bottom edge extraction module takes the bottom edge height map as input, and inputs the bottom edge height map into the bottom edge neural network, so that the feature information of each pixel point is compared with the feature information of the column of pixels in the original feature map, and the corresponding relationship between the single column of pixels in the bottom edge height map and the column of pixels in the original feature map is obtained, and the corresponding position relationship of each pixel point in the bottom edge height map in the original feature map is obtained, and the bottom edge contour of the instance in the target detection frame is obtained by the corresponding position relationship; S02: extracting the pixel point at the lowest position relative to the ground in the lower edge contour of the instance in the target detection frame, determining that the pixel point is the contact point between the object shown in the target detection frame and the ground, using the calibration parameters of the body camera to obtain the projection of the body camera on the ground, and obtaining the relative position relationship between the body and the target detection frame through coordinate system conversion; Among them, the height map neural network is a deep neural network that can realize feature dimension conversion; the lower edge neural network is a deep neural network that can realize lower edge contour extraction.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.
