Method, device, electronic equipment and storage medium for image processing

Abstract

The present disclosure relates to a kind of method, device, storage medium, electronic equipment and vehicle of image processing, relate to the automatic driving technical field.The method comprises: obtaining the multiple frames of environment image in the preset range around vehicle, by first object detection algorithm, the first object image corresponding to the test object is determined from each environment image, by second object detection algorithm, the second object image corresponding to the test object is determined from each environment image, the first object image and the second object image are matched, and the matching degree is obtained, according to the matching degree, the algorithm measurement parameter corresponding to the first object detection algorithm is calculated, and the algorithm measurement parameter represents the accuracy of the first object detection algorithm.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and more specifically, to an image processing method, apparatus, electronic device, storage medium, and vehicle. Background Technology

[0002] Image recognition based on Bird's Eye View (BEV) can transform the image to be detected from image space to BEV space through neural networks, thereby converting the image into projection information perpendicular to the height direction, which can improve the accuracy of object recognition. However, when there are few reference objects around the object, this method has certain errors in object distance recognition. It may identify relatively large objects as close to the vehicle, and relatively small objects as far away. Therefore, using this method for image recognition may also affect the accuracy of the image recognition results, and thus the accuracy of the image recognition results obtained by this method cannot be determined. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an image processing method, apparatus, electronic device, storage medium, and vehicle.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, applied to a vehicle, comprising:

[0005] Acquire multiple frames of environmental images within a preset range around the vehicle, the environmental images including the test object;

[0006] A first object image corresponding to the test object is determined from each frame of the environment image using a first object detection algorithm; the first object image includes object annotations based on the test object, and the object annotations represent the ground truth of the test object;

[0007] The second object image corresponding to the test object is determined from each frame of the environment image using a second object detection algorithm;

[0008] The first object image and the second object image are matched to obtain the matching degree;

[0009] Based on the true value and the matching degree, the algorithm measurement parameters corresponding to the first object detection algorithm are calculated, and the algorithm measurement parameters characterize the accuracy of the first object detection algorithm.

[0010] Optionally, determining the first object image corresponding to the test object from each frame of the environment image using the first object detection algorithm includes:

[0011] The multi-frame environmental images are converted into point cloud information;

[0012] The first object detection algorithm is used to determine the multi-frame test object corresponding to the test object from the point cloud information;

[0013] The point cloud information is transformed by coordinates to obtain the multi-frame first object image corresponding to the multi-frame test object.

[0014] Optionally, matching the first object image and the second object image to obtain a matching degree includes:

[0015] The first object image in each frame is matched with a specified image to obtain the matching degree; the specified image is the second object image corresponding to the first object image in the current frame.

[0016] Optionally, calculating the algorithm evaluation parameters corresponding to the first object detection algorithm based on the truth value and the matching degree includes:

[0017] Based on the matching degree of the first object image, determine the tracking object image from the first object image;

[0018] Based on the ground truth value corresponding to the tracked object image, calculate the algorithm evaluation parameters corresponding to the first object detection algorithm.

[0019] Optionally, the test object includes one, and determining the tracking object image from the first object image based on the matching degree of the first object image includes:

[0020] The image with a matching degree greater than or equal to a preset matching degree threshold among the multiple frames of the first object image is used as the tracking object image.

[0021] Optionally, the test objects include multiple objects, and determining the tracking object image from the first object image based on the matching degree of the first object image includes:

[0022] For each test object, calculate the matching degree and value of the first object image in each frame corresponding to the test object;

[0023] The image of the first object corresponding to the test object with the highest matching degree and value is used as the tracking object image.

[0024] Optionally, the method further includes:

[0025] Acquire the target image to be detected;

[0026] The target object in the target image is detected by the first object detection algorithm.

[0027] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0028] The first object detection module is configured to determine a first object image corresponding to a test object from the multi-frame environment images according to a first object detection algorithm; the first object image includes object annotations based on the test object, and the object annotations represent the ground truth of the test object.

[0029] The second object detection module is configured to determine the second object image corresponding to the test object from the multi-frame environmental images according to the second object detection algorithm;

[0030] The matching module is configured to match the first object image and the second object image to obtain a matching degree, and calculate the algorithm evaluation parameters corresponding to the first object detection algorithm based on the ground truth and the matching degree.

[0031] Optionally, the first object detection module is further configured to:

[0032] The multi-frame environmental images are converted into point cloud information;

[0033] The first object detection algorithm is used to determine the multi-frame test object corresponding to the test object from the point cloud information;

[0034] The point cloud information is transformed by coordinates to obtain the multi-frame first object image corresponding to the multi-frame test object.

[0035] Optionally, the matching module is further configured to:

[0036] The first object image in each frame is matched with a specified image to obtain the matching degree; the specified image is the second object image corresponding to the first object image in the current frame.

[0037] Optionally, the matching module is further configured to:

[0038] Based on the matching degree of the first object image, determine the tracking object image from the first object image;

[0039] Based on the ground truth value corresponding to the tracked object image, calculate the algorithm evaluation parameters corresponding to the first object detection algorithm.

[0040] Optionally, the test object includes one, and the matching module is further configured to:

[0041] The image with a matching degree greater than or equal to a preset matching degree threshold among the multiple frames of the first object image is used as the tracking object image.

[0042] Optionally, the test objects include multiple objects, and the matching module is further configured to:

[0043] For each test object, calculate the matching degree and value of the first object image in each frame corresponding to the test object;

[0044] The image of the first object corresponding to the test object with the highest matching degree and value is used as the tracking object image.

[0045] Optionally, the first object detection module is further configured to:

[0046] Acquire the target image to be detected;

[0047] The target object in the target image is detected by the first object detection algorithm.

[0048] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0049] A memory on which computer programs are stored;

[0050] A processor for executing the computer program in the memory to implement the steps of the method described in the first aspect of this disclosure.

[0051] According to a fourth aspect of the present disclosure, a vehicle is provided, including the electronic equipment described in the third aspect above.

[0052] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the method described in the first aspect of the present disclosure.

[0053] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0054] This disclosure improves the usability of the first object detection algorithm by introducing a second object detection algorithm, matching the detection results of the first object detection algorithm with those of the second object detection algorithm, obtaining the matching degree, and calculating the algorithm accuracy parameters based on the matching degree.

[0055] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0057] Figure 1This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0058] Figure 2 This is a flowchart illustrating another image processing method according to an exemplary embodiment.

[0059] Figure 3 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0060] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0062] First, the application scenario of this disclosure is explained. This disclosure is applied to the autonomous driving scenario of vehicles. Autonomous driving control systems are generally divided into three parts: environmental perception, decision planning, and control execution. Environmental perception collects information about the vehicle's surroundings through various sensors, commonly including cameras, LiDAR, millimeter-wave radar, and integrated navigation systems. Currently, there are two mainstream technical approaches in the industry: one is a multi-sensor fusion scheme dominated by cameras; the other is a scheme dominated by LiDAR, with other sensors as auxiliary sensors. Regardless of the approach, it is necessary to first collect information about the vehicle's surrounding environment, such as roads, pedestrians, traffic signals, traffic signs, traffic conditions, and surrounding vehicles. Then, intelligent recognition and control algorithms are used to calculate and plan driving commands and paths. Finally, the drive-by-wire chassis system executes the driving commands and controls the vehicle's operation. Therefore, in the research and development of autonomous driving technology, the visual recognition algorithm must first be tested and evaluated. Only visual recognition algorithms that have undergone complete testing and have received optimal evaluation can be used for autonomous driving of vehicles. Thus, how to test and accurately evaluate a visual recognition algorithm becomes the primary issue.

[0063] Visual recognition tasks can be performed in an image coordinate system, where the detected objects are all located in a perspective view space. However, BEV-based visual recognition algorithms can transform the spatial coordinate system of the detected objects to the BEV coordinate system. Since the BEV view has a different angle, the detected objects do not have the scale and occlusion problems of the image view, and can be easily recognized and processed.

[0064] However, the inventors discovered that BEV-based image recognition has an inherent disadvantage in recognizing object distances because it loses the three-dimensional information of objects in the image coordinate system, especially the height information. This is particularly pronounced in laboratory testing environments where there are few reference targets. This also makes it impossible to accurately evaluate the merits of the BEV algorithm.

[0065] To address the aforementioned issues, this disclosure provides an image recognition method, apparatus, storage medium, electronic device, and vehicle. By introducing a second object detection algorithm and matching the detection results of a first object detection algorithm with those of the second object detection algorithm to obtain a matching degree, and then calculating the algorithm's accuracy parameters based on the matching degree, the method obtains image frames where the first object detection algorithm accurately identifies and inaccurately identifies objects in object distance recognition. This improves the usability of the first object detection algorithm and simultaneously obtains algorithm evaluation parameters, providing a basis for accurately evaluating the first object detection algorithm.

[0066] The present disclosure will now be described in conjunction with specific embodiments.

[0067] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment, such as... Figure 1 As shown, this image processing method, when applied to a vehicle, may include the following steps.

[0068] In step S101, multiple frames of environmental images within a preset range around the vehicle are acquired, and the environmental images include the test object.

[0069] For example, a camera device can be used to capture environmental video within a preset range around the vehicle, and multiple frames of environmental images can be extracted from the environmental video.

[0070] In step S102, a first object image corresponding to the test object is determined from each frame of the environment image using a first object detection algorithm; the first object image includes object annotations based on the test object, and the object annotations represent the ground truth of the test object.

[0071] In some embodiments, a first object detection algorithm can be used to determine the recognition object image corresponding to the test object from each frame of the environment image. The recognition object image may include a bounding box (such as a rectangle) used to mark the test object in each frame of the environment image. After obtaining the object image to be determined, the recognition object image can be manually annotated to mark the ground truth of the recognition object image, thereby obtaining the first object image. The ground truth value is used to characterize whether the test object identified by the first object detection algorithm is correct. For example, the ground truth value may include "True Positive", "False Negative", "False Positive", and "True Negative". Here, "True Positive" indicates that for a test object in the environment image, the first object detection algorithm identifies the test object in the environment image; "False Negative" indicates that for a test object in the environment image, the first object detection algorithm does not identify the test object in the environment image; "False Positive" indicates that for a non-test object in the environment image, the first object detection algorithm identifies the non-test object as a test object; and "True Negative" indicates that for a non-test object in the environment image, the first object detection algorithm does not identify the non-test object.

[0072] For example, the first object detection algorithm may include the BEV algorithm.

[0073] In step S103, the second object image corresponding to the test object is determined from each frame of the environment image using the second object detection algorithm.

[0074] In this embodiment, the second object detection algorithm can be an algorithm with a high recognition rate for test objects in two-dimensional images. The second object detection algorithm may include a region recognition algorithm based on convolutional neural networks (R-CNN). The recognition of objects by the second object detection algorithm can refer to the algorithms in related technologies, which will not be elaborated here.

[0075] In step S104, the first object image and the second object image are matched to obtain the matching degree.

[0076] In step S105, based on the true value and the matching degree, the algorithm measurement parameter corresponding to the first object detection algorithm is calculated. The algorithm measurement parameter characterizes the accuracy of the first object detection algorithm.

[0077] In this step, the algorithm measurement parameters may include accuracy and / or recall. After obtaining the algorithm measurement parameters, the accuracy of the first object detection algorithm can be measured based on the algorithm measurement parameters. If the accuracy of the first object detection algorithm is high, the first object detection algorithm can be applied to actual vehicle driving scenarios to identify target objects in the input target image, measure the distance to the target object or measure the speed of the target object, thereby performing driving control.

[0078] For example, a target image to be detected can be obtained, and the target object in the target image can be detected by the first object detection algorithm.

[0079] By introducing a second object detection algorithm and matching the detection results of the first object detection algorithm with those of the second object detection algorithm to obtain the matching degree, and calculating the algorithm's accurate parameters based on the matching degree, the image frames in which the first object detection algorithm accurately identifies and inaccurately identifies objects in object distance recognition are obtained. This improves the usability of the first object detection algorithm and obtains algorithm evaluation parameters, providing a basis for accurately evaluating the first object detection algorithm.

[0080] In some embodiments, step S102 above may include the following steps:

[0081] S1. Convert the multi-frame environmental image into point cloud information.

[0082] S2. Using the first object detection algorithm, determine the multi-frame test object corresponding to the test object from the point cloud information.

[0083] S3. Perform coordinate transformation on the point cloud information to obtain the first image of the multi-frame test object corresponding to the multi-frame test object.

[0084] For example, in the first object detection algorithm, the multi-frame test image is first converted into point cloud information for three-dimensional modeling of the target object so that the machine can recognize and process it; secondly, the point cloud information is transformed into coordinates, specifically, from three-dimensional to two-dimensional, for matching with the two-dimensional image recognized by the second object detection algorithm in subsequent steps.

[0085] In some embodiments, step S104 above may include:

[0086] The matching degree is obtained by matching the first object image of each frame with a specified image; the specified image is the second object image corresponding to the first object image of the current frame.

[0087] The matching degree can be the Intersection Over Union (IOU), which is used to characterize the overlap rate between the candidate box and the original labeled box in the image to be recognized. That is, the ratio of their intersection to their union. The higher the ratio, the higher the recognition accuracy of the image to be recognized. The lower the ratio, the lower the recognition accuracy of the image to be recognized.

[0088] In some possible implementations, the tracking object image can be determined from the first object image based on the matching degree of the first object image. Furthermore, the first object detection algorithm can be used as the target object detection algorithm based on the tracking object image. This allows for the determination of tracking object images with high matching degrees, leading to a more accurate evaluation of the first object detection algorithm.

[0089] For example, the test object may include one or more. If the test object includes one, the above-mentioned determination of the tracking object image from the first object image based on the matching degree of the first object image may include: taking the image in the multiple frames of the first object image whose matching degree is greater than or equal to a preset matching degree threshold as the tracking object image.

[0090] For example, the preset matching threshold can be 0.5.

[0091] When the test object includes multiple objects, the above-mentioned determination of the tracking object image from the first object image based on the matching degree of the first object image may include:

[0092] For each test object, calculate the matching degree and value of the first object image in each frame corresponding to that test object; use the first object image corresponding to the test object with the largest matching degree and value as the tracking object image.

[0093] For example, taking the first object image as including 5 frames of images (namely image 1, image 2, image 3, image 4 and image 5 respectively), where image 1 may include test object A, test object B and test object C, image 2 may include test object A, test object B and test object C, image 3 may include test object A, test object B and test object C, image 4 may include test object A, test object B and test object C, and image 5 may include test object A, test object B and test object C;

[0094] Taking the second object image as an example, which also includes 5 frames (namely, image 6, image 7, image 8, image 9, and image 10 respectively), image 6 may include test object A, test object B, and test object C; image 7 may include test object A, test object B, and test object C; image 8 may include test object A, test object B, and test object C; image 9 may include test object A, test object B, and test object C; and image 10 may include test object A, test object B, and test object C.

[0095] Among them, image 1 in the first object image corresponds to image 6 in the second object image, image 2 in the first object image corresponds to image 7 in the second object image, image 3 in the first object image corresponds to image 8 in the second object image, image 4 in the first object image corresponds to image 9 in the second object image, and image 5 in the first object image corresponds to image 10 in the second object image.

[0096] Matching image 1 with image 6 yields the first set of matching scores for test subjects A, B, and C, denoted as A1, B1, and C1; matching image 2 with image 7 yields the second set of matching scores for test subjects A, B, and C, denoted as A2, B2, and C2; matching image 3 with image 8 yields the third set of matching scores for test subjects A, B, and C, denoted as A3, B3, and C3; matching image 4 with image 9 yields the fourth set of matching scores for test subjects A, B, and C, denoted as A4, B4, and C4; and matching image 5 with image 10 yields the fifth set of matching scores for test subjects A, B, and C, denoted as A5, B5, and C5.

[0097] Calculate the value of A1+A2+A3+A4+A5, and record the result as A0;

[0098] Calculate the value of B1+B2+B3+B4+B5, and record the result as B0;

[0099] Calculate the value of C1+C2+C3+C4+C5, and record the result as C0;

[0100] Arrange the values ​​of A0, B0, and C0 in descending order and find the maximum value. For example, we can assume that B0 is the maximum value, and then determine the test object B in the first object image as the tracking object image.

[0101] For example, if the accuracy parameter of the above algorithm includes the accuracy rate, the accuracy rate can be calculated using the following formula:

[0102]

[0103] Wherein, Accuracy represents the accuracy in the algorithm's accuracy parameters, TP is the "True Positive" described in step 102 above, TN is the "True Negative" described in step 102 above, FP is the "False Positive" described in step 102 above, and FN is the "False Negative" described in step 102 above. Based on the matching degree calculated in step 104, image frames that are greater than or equal to the preset matching degree threshold are selected. The value of TP can be determined from these image frames that are greater than or equal to the preset matching degree threshold, and the values ​​of TN, FP, and FN can be determined from the image frames that are less than the preset matching degree threshold, which are used to calculate the accuracy.

[0104] When the recall rate is included as an accurate parameter in the above algorithm, the recall rate can be calculated using the following formula:

[0105]

[0106] Wherein, Recall represents the recall rate in the accurate parameters of the algorithm, TP is the "True Positive" described in step 102 above, and FN is the "False Negative" described in step 102 above. Based on the matching degree calculated in step 104, image frames that are greater than or equal to the preset matching degree threshold are determined. The value of TP can be determined from these image frames that are greater than or equal to the preset matching degree threshold, and the value of FN can be determined from the image frames that are less than the preset matching degree threshold, which are used to calculate the recall rate.

[0107] Using the above method, multiple frames of environmental images of the test object are labeled, and image frames with matching scores greater than or equal to a preset matching score threshold are filtered to obtain data for calculating the algorithm's measurement parameters. This allows for a quantitative evaluation of the accuracy of the first object detection algorithm.

[0108] Figure 2 This is a flowchart illustrating another image processing method according to an exemplary embodiment, such as... Figure 2 As shown, the method may include the following steps.

[0109] In step S201, multiple frames of environmental images within a preset range around the vehicle are acquired, and the environmental images include the test object.

[0110] In step S202, the multi-frame environmental image is converted into point cloud information.

[0111] In step S203, the multi-frame test object corresponding to the test object is determined from the point cloud information using the first object detection algorithm.

[0112] In step S204, the point cloud information is transformed to obtain the first image of the multi-frame test object.

[0113] In step S205, the second object image corresponding to the test object is determined from each frame of the environment image using the second object detection algorithm.

[0114] In step S206, the first object image of each frame is matched with a specified image to obtain the matching degree; the specified image is the second object image corresponding to the first object image of the current frame.

[0115] In step S207, it is determined whether the test object is a single object.

[0116] If there is only one test object, execute steps S208 and S211;

[0117] If there are multiple test objects, execute steps S209, S210, and S211.

[0118] In step S208, the image in the multi-frame first object image whose matching degree is greater than or equal to a preset matching degree threshold is taken as the tracking object image.

[0119] In step S209, for each test object, the matching degree and value of the first object image in each frame corresponding to the test object are calculated;

[0120] In step S210, the first object image corresponding to the test object with the highest matching degree and value is used as the tracking object image.

[0121] In step S211, the algorithm evaluation parameters corresponding to the first object detection algorithm are calculated based on the ground truth value corresponding to the tracked object image.

[0122] By introducing a second object detection algorithm and matching the detection results of the first object detection algorithm with those of the second object detection algorithm to obtain the matching degree, and calculating the algorithm's accurate parameters based on the matching degree, the image frames in which the first object detection algorithm accurately identifies and inaccurately identifies objects in object distance recognition are obtained. This improves the usability of the first object detection algorithm and obtains algorithm evaluation parameters, providing a basis for accurately evaluating the first object detection algorithm.

[0123] It should be noted that, for the sake of simplicity, the above method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0124] Figure 3 This is a block diagram illustrating an image processing apparatus 300 according to an exemplary embodiment. (Refer to...) Figure 3 The device 300 includes an image acquisition module 301, a first object detection module 302, a second object detection module 303, and a matching module 304.

[0125] The image acquisition module 301 is configured to acquire multiple frames of environmental images within a preset range around the vehicle, the environmental images including the test object;

[0126] The first object detection module 302 is configured to determine a first object image corresponding to a test object from each frame of the environment image according to a first object detection algorithm; the first object image includes object annotations based on the test object, and the object annotations represent the ground truth of the test object.

[0127] The second object detection module 303 is configured to determine the second object image corresponding to the test object from each frame of the environment image according to the second object detection algorithm;

[0128] The matching module 304 is configured to match the first object image and the second object image to obtain a matching degree, and calculate the algorithm measurement parameter corresponding to the first object detection algorithm based on the ground truth and the matching degree. The algorithm measurement parameter characterizes the accuracy of the first object detection algorithm.

[0129] By using the above-mentioned device, by introducing a second object detection algorithm and matching the detection results of the first object detection algorithm with those of the second object detection algorithm to obtain the matching degree, and by calculating the algorithm accuracy parameters based on the matching degree, the device obtains image frames in which the first object detection algorithm accurately identifies objects in object distance recognition and frames in which it does not. This improves the usability of the first object detection algorithm and obtains algorithm evaluation parameters, providing a basis for accurately evaluating the first object detection algorithm.

[0130] Optionally, the first object detection module 302 is further configured to:

[0131] Convert the multi-frame environmental image into point cloud information;

[0132] The first object detection algorithm is used to determine the multi-frame test object corresponding to the test object from the point cloud information;

[0133] The point cloud information is transformed to obtain the first image of the multi-frame test object.

[0134] Optionally, the matching module 304 is also configured as follows:

[0135] The matching degree is obtained by matching the first object image of each frame with a specified image; the specified image is the second object image corresponding to the first object image of the current frame.

[0136] Optionally, the matching module 304 is also configured as follows:

[0137] Based on the matching degree of the first object image, determine the tracking object image from the first object image;

[0138] Based on the ground truth value corresponding to the image of the tracked object, calculate the algorithm evaluation parameters corresponding to the first object detection algorithm.

[0139] Optionally, the test object includes one, and the matching module 304 is further configured as follows:

[0140] The image with a matching degree greater than or equal to a preset matching degree threshold among the multiple frames of the first object image is taken as the tracking object image.

[0141] Optionally, the test object includes multiple objects, and the matching module 304 is further configured as follows:

[0142] For each test object, calculate the matching degree and value of the first object image in each frame corresponding to that test object;

[0143] The image of the first object corresponding to the test object with the highest matching degree and value is used as the tracking object image.

[0144] Optionally, the first object detection module 302 is further configured to:

[0145] Acquire the target image to be detected;

[0146] The first object detection algorithm is used to detect the target object in the target image.

[0147] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0148] In another exemplary embodiment, a computer-readable storage medium is also provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the image detection method provided in this disclosure.

[0149] Figure 4This is a block diagram illustrating an electronic device 400 according to an exemplary embodiment. Figure 4 As shown, the electronic device 400 may include a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output interface 404, and a communication component 405.

[0150] The processor 401 controls the overall operation of the electronic device 400 to complete all or part of the steps in the image processing method described above. The memory 402 stores various types of data to support the operation of the electronic device 400. This data may include, for example, instructions for any application or method operating on the electronic device 400, and application-related data such as sent and received messages, images, audio, video, etc. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 403 includes a screen and a camera, wherein the screen may be, for example, any one or a combination of a touchscreen and an LCD display; the camera may include one or more. Input / output interface 404 provides an interface between processor 401 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. Communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G, or one or more combinations thereof, is not limited here. Therefore, the corresponding communication component 405 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0151] This disclosure also provides a vehicle that may include the aforementioned electronic equipment.

[0152] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0153] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, Applied to vehicles, including: Acquire multiple frames of environmental images within a preset range around the vehicle, the environmental images including the test object; A first object image corresponding to the test object is determined from each frame of the environment image using a first object detection algorithm; the first object detection algorithm includes the BEV algorithm, and the first object image includes object annotations based on the test object, wherein the object annotations represent the ground truth of the test object; The second object detection algorithm is used to determine the second object image corresponding to the test object from each frame of the environmental image. The second object detection algorithm includes a region recognition algorithm based on a convolutional neural network. The first object image and the second object image are matched to obtain the matching degree; Based on the matching degree of the first object image, determine the tracking object image from the first object image; Based on the ground truth value corresponding to the tracked object image, the algorithm evaluation parameters corresponding to the first object detection algorithm are calculated, and the algorithm evaluation parameters characterize the accuracy of the first object detection algorithm.

2. The method according to claim 1, characterized in that, The step of determining the first object image corresponding to the test object from each frame of the environment image using the first object detection algorithm includes: The multi-frame environmental images are converted into point cloud information; The first object detection algorithm is used to determine the multi-frame test object corresponding to the test object from the point cloud information; The point cloud information is transformed by coordinates to obtain the multi-frame first object image corresponding to the multi-frame test object.

3. The method according to claim 1, characterized in that, The step of matching the first object image and the second object image to obtain the matching degree includes: The first object image in each frame is matched with a specified image to obtain the matching degree; the specified image is the second object image corresponding to the first object image in the current frame.

4. The method according to claim 1, characterized in that, The test object includes one, and determining the tracking object image from the first object image based on the matching degree of the first object image includes: The image with a matching degree greater than or equal to a preset matching degree threshold among the multiple frames of the first object image is used as the tracking object image.

5. The method according to claim 1, characterized in that, The test objects include multiple objects, and determining the tracking object image from the first object image based on the matching degree of the first object image includes: For each test object, calculate the matching degree and value of the first object image in each frame corresponding to the test object; The image of the first object corresponding to the test object with the highest matching degree and value is used as the tracking object image.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Acquire the target image to be detected; The target object in the target image is detected by the first object detection algorithm.

7. An image processing apparatus, characterized in that, include: The image acquisition module is configured to acquire multiple frames of environmental images within a preset range around the vehicle; The first object detection module is configured to determine a first object image corresponding to a test object from the multi-frame environment images according to a first object detection algorithm; the first object detection algorithm includes the BEV algorithm, and the first object image includes object annotations based on the test object, wherein the object annotations represent the ground truth of the test object. The second object detection module is configured to determine the second object image corresponding to the test object from the multi-frame environmental images according to the second object detection algorithm, wherein the second object detection algorithm includes a region recognition algorithm based on a convolutional neural network. The matching module is configured to match the first object image and the second object image to obtain a matching degree, and determine the tracking object image from the first object image based on the matching degree of the first object image; and calculate the algorithm evaluation parameters corresponding to the first object detection algorithm based on the ground truth value corresponding to the tracking object image, wherein the algorithm evaluation parameters characterize the accuracy of the first object detection algorithm.

8. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

9. A vehicle, characterized in that, Including the electronic device as described in claim 8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-6.

Images