Monocular three-dimensional target detection post-processing method, device and electronic equipment

Abstract

The disclosure provides a monocular three-dimensional target detection post-processing method, device and electronic equipment, the method comprising: obtaining detection output information of a candidate detection box and camera intrinsic parameters, wherein the detection output information comprises: a candidate class confidence corresponding to the candidate detection box, a pixel center coordinate, depth information, size information, and a rotation angle; generating a two-dimensional envelope box corresponding to the candidate detection box according to the camera intrinsic parameters, the pixel center coordinate, the depth information, and the size information; determining a three-dimensional overlap degree between a plurality of candidate detection boxes according to the two-dimensional envelope box, the depth information, and the rotation angle; determining a candidate quality value corresponding to the candidate detection box according to the candidate class confidence and the pixel center coordinate; and determining a target detection box from the plurality of candidate detection boxes according to the candidate quality value and the three-dimensional overlap degree. Thus, the post-processing calculation process can be decoupled to reduce the calculation overhead and effectively improve the accuracy and real-time performance of the obtained target detection box.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, specifically to a method, apparatus, and electronic device for post-processing monocular three-dimensional target detection. Background Technology

[0002] Object detection aims to identify objects of interest in images or videos, while simultaneously outputting the location and category of the detected objects. This is one of the core problems in the field of machine vision. With the rapid development of deep learning technology, object detection algorithms have shifted from traditional algorithms based on handcrafted features to detection techniques based on deep neural networks. Neural network-based detection algorithms mainly focus on two directions: two-stage algorithms and one-stage algorithms. Although they differ in the steps of generating detection boxes, both require further processing of the generated candidate boxes to filter out low-quality candidates and retain high-quality ones as the object detection output. However, overly complex post-processing methods can limit the model's hardware performance, hindering practical applications.

[0003] In related technologies, filtering out low-quality candidate boxes may incur significant computational overhead, affecting the real-time performance of the algorithm, and may result in redundant detection boxes, making it impossible to guarantee the accuracy of the target detection boxes. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the purpose of this disclosure is to propose a monocular 3D target detection post-processing method, device, electronic device and storage medium, which can decouple the post-processing calculation process to reduce computational overhead and effectively improve the accuracy and real-time performance of the obtained target detection box.

[0006] The monocular 3D target detection post-processing method proposed in the first aspect of this disclosure includes:

[0007] Obtain the detection output information and camera intrinsic parameters of the candidate detection box, wherein the detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box;

[0008] Based on the camera intrinsic parameters, the pixel center coordinates, the depth information, and the size information, a two-dimensional envelope box corresponding to the candidate detection box is generated.

[0009] Based on the two-dimensional envelope, the depth information, and the rotation angle, the three-dimensional overlap between the candidate detection boxes is determined:

[0010] Based on the candidate category confidence level and the pixel center coordinates, determine the candidate quality value corresponding to the candidate detection box;

[0011] Based on the candidate quality value and the three-dimensional overlap, a target detection box is determined from a plurality of candidate detection boxes.

[0012] The monocular 3D target detection post-processing method proposed in the first aspect of this disclosure obtains the detection output information of candidate detection boxes and camera intrinsic parameters. The detection output information includes: candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection boxes. Based on the camera intrinsic parameters, pixel center coordinates, depth information, and size information, a 2D envelope box corresponding to the candidate detection box is generated. Based on the 2D envelope box, depth information, and rotation angle, the 3D overlap between multiple candidate detection boxes is determined. Based on the candidate category confidence and pixel center coordinates, the candidate quality value corresponding to the candidate detection box is determined. Based on the candidate quality value and 3D overlap, the target detection box is determined from multiple candidate detection boxes. Thus, the post-processing calculation process can be decoupled to reduce computational overhead and effectively improve the accuracy and real-time performance of the obtained target detection boxes.

[0013] The monocular three-dimensional target detection post-processing apparatus proposed in the second aspect of this disclosure includes:

[0014] The first acquisition module is used to acquire the detection output information and camera intrinsic parameters of the candidate detection box, wherein the detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box;

[0015] The generation module is used to generate a two-dimensional envelope corresponding to the candidate detection box based on the camera intrinsic parameters, the pixel center coordinates, the depth information and the size information;

[0016] The first determining module is used to determine the three-dimensional overlap between multiple candidate detection boxes based on the two-dimensional envelope, the depth information, and the rotation angle.

[0017] The second determining module is used to determine the candidate quality value corresponding to the candidate detection box based on the candidate category confidence and the pixel center coordinates.

[0018] The third determining module is used to determine the target detection box from the multiple candidate detection boxes based on the candidate quality value and the three-dimensional overlap.

[0019] The monocular 3D target detection post-processing apparatus proposed in the second aspect of this disclosure acquires detection output information of candidate detection boxes and camera intrinsic parameters. The detection output information includes: candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection boxes. Based on the camera intrinsic parameters, pixel center coordinates, depth information, and size information, a 2D envelope box corresponding to the candidate detection boxes is generated. Based on the 2D envelope box, depth information, and rotation angle, the 3D overlap between multiple candidate detection boxes is determined. Based on the candidate category confidence and pixel center coordinates, the candidate quality value corresponding to the candidate detection boxes is determined. Based on the candidate quality value and 3D overlap, the target detection box is determined from multiple candidate detection boxes. Thus, the post-processing calculation process can be decoupled to reduce computational overhead and effectively improve the accuracy and real-time performance of the obtained target detection boxes.

[0020] The electronic device proposed in the third aspect of this disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the monocular three-dimensional target detection post-processing method proposed in the first aspect of this disclosure.

[0021] The fourth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the monocular three-dimensional target detection post-processing method as proposed in the first aspect of this disclosure.

[0022] A fifth aspect of this disclosure provides a computer program product that, when executed by a processor, performs a monocular 3D target detection post-processing method as described in a first aspect of this disclosure.

[0023] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0025] Figure 1 This is a schematic flowchart of a monocular three-dimensional target detection post-processing method proposed in an embodiment of this disclosure;

[0026] Figure 2 Here is a flowchart illustrating a monocular 3D target detection post-processing method according to another embodiment of this disclosure:

[0027] Figure 3 This is a schematic flowchart of a monocular three-dimensional target detection post-processing method proposed in another embodiment of this disclosure;

[0028] Figure 4 This is a schematic diagram of the structure of a monocular three-dimensional target detection post-processing device according to an embodiment of this disclosure;

[0029] Figure 5 This is a schematic diagram of the structure of a monocular three-dimensional target detection post-processing device according to another embodiment of this disclosure;

[0030] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0031] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0032] Figure 1 This is a schematic flowchart of a monocular three-dimensional target detection post-processing method proposed in an embodiment of this disclosure.

[0033] It should be noted that the execution subject of the monocular 3D target detection post-processing method in this embodiment is a monocular 3D target detection post-processing device. This device can be implemented by software and / or hardware. This device can be configured in an electronic device, which may include, but is not limited to, a terminal, a server, etc. For example, the terminal may be a mobile phone, a PDA, etc.

[0034] like Figure 1 As shown, the monocular 3D target detection post-processing method includes:

[0035] S101: Obtain the detection output information and camera intrinsic parameters of the candidate detection box. The detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box.

[0036] In this context, a bounding box refers to a box containing the object to be identified during the object detection process. A candidate bounding box, on the other hand, refers to a bounding box that may be used as the object's bounding box.

[0037] Among them, the detection output information refers to the information related to the candidate detection boxes output by the target detection model after executing the target detection process.

[0038] Among them, camera intrinsic parameters refer to parameters that characterize the transformation relationship between the imaging plane and the pixel plane.

[0039] Here, category confidence refers to the reliability of the object category indicated by the detection box. For example, category confidence can be used to indicate that the probability of the detection box corresponding to the detected object being a vehicle is 0.8. Candidate category confidence, on the other hand, refers to the category confidence corresponding to the candidate detection box.

[0040] The pixel center coordinates refer to the coordinates of the center point of the candidate detection box in the pixel coordinate system.

[0041] Depth information can be used to indicate the vertical distance from each pixel in the scene to the camera's imaging plane.

[0042] Among them, the size information can be, for example, the length, width, and height of the 3D detection box.

[0043] The rotation angle can refer to the deflection angle of the candidate detection box relative to the object to be detected on the horizontal plane.

[0044] In this embodiment of the disclosure, when obtaining the detection output information and camera intrinsic parameters of the candidate detection box, the image to be detected can be processed based on a monocular 3D detection model to obtain the detection output information and obtain the camera parameters corresponding to the image to be detected in order to determine the camera intrinsic parameters. There are no restrictions on this.

[0045] In this embodiment of the disclosure, when the detection output information of the candidate detection box and the camera intrinsic parameters are obtained, reliable data support can be provided for the subsequent post-processing.

[0046] S102: Generate a two-dimensional envelope corresponding to the candidate detection box based on the camera intrinsic parameters, pixel center coordinates, depth information, and size information.

[0047] Here, the envelope box refers to a rectangle that contains the target object and is parallel to the x-axis and y-axis. The 2D envelope box, on the other hand, refers to the envelope box in a 2D scene that corresponds to the aforementioned candidate detection boxes.

[0048] In this embodiment of the disclosure, when generating a two-dimensional envelope corresponding to a candidate detection box based on camera intrinsic parameters, pixel center coordinates, depth information, and size information, the camera intrinsic parameters, pixel center coordinates, depth information, and size information can be input into a pre-trained machine learning model to obtain a two-dimensional envelope corresponding to the candidate detection box. Alternatively, it can be generated using engineering methods based on camera intrinsic parameters, pixel center coordinates, depth information, and size information. No limitation is imposed on this method.

[0049] S103: Determine the 3D overlap between multiple candidate detection boxes based on the 2D envelope, depth information, and rotation angle.

[0050] Overlap, in this context, can refer to the ratio between the intersection and union of two bounding boxes. 3D overlap refers to the degree of overlap between candidate bounding boxes in 3D space.

[0051] It is understood that during the target detection process, multiple candidate detection boxes may be generated for the same object to be detected, and there may be a high degree of overlap between the candidate detection boxes of the same object. The three-dimensional overlap can accurately indicate the overlap information between different candidate detection boxes. Therefore, in this embodiment of the present disclosure, when the three-dimensional overlap between multiple candidate detection boxes is determined based on the two-dimensional envelope, depth information and rotation angle, it can provide a reliable basis for subsequent screening of redundant candidate detection boxes.

[0052] S104: Determine the candidate quality value corresponding to the candidate detection box based on the candidate category confidence and pixel center coordinates.

[0053] Among them, the candidate quality value can be used to indicate the reliability of the corresponding candidate detection box.

[0054] In this embodiment of the disclosure, when determining the candidate quality value corresponding to the candidate detection box based on the candidate category confidence level and pixel center coordinates, the relevant information of multiple dimensions of the candidate detection box can be fully considered in the process of determining the candidate quality value, thereby effectively improving the accuracy of the obtained candidate quality value indication.

[0055] S105: Determine the target detection box from multiple candidate detection boxes based on the candidate quality value and 3D overlap.

[0056] Among them, the target detection box refers to the box used to indicate the object to be detected, which is obtained based on the candidate quality value and the three-dimensional overlap.

[0057] After determining the candidate quality value corresponding to the candidate detection box based on the candidate category confidence and pixel center coordinates, the target detection box can be determined from multiple candidate detection boxes based on the candidate quality value and three-dimensional overlap. This achieves the filtering of low-quality and redundant candidate detection boxes and retains a few high-quality candidate detection boxes for output.

[0058] In this embodiment, by acquiring the detection output information of candidate detection boxes and camera intrinsic parameters, the detection output information includes: candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box. Based on the camera intrinsic parameters, pixel center coordinates, depth information, and size information, a two-dimensional envelope box corresponding to the candidate detection box is generated. Based on the two-dimensional envelope box, depth information, and rotation angle, the three-dimensional overlap between multiple candidate detection boxes is determined. Based on the candidate category confidence and pixel center coordinates, the candidate quality value corresponding to the candidate detection box is determined. Based on the candidate quality value and the three-dimensional overlap, the target detection box is determined from multiple candidate detection boxes. Thus, the post-processing calculation process can be decoupled to reduce computational overhead and effectively improve the accuracy and real-time performance of the obtained target detection box.

[0059] Figure 2 This is a schematic flowchart of a monocular three-dimensional target detection post-processing method proposed in another embodiment of this disclosure.

[0060] like Figure 2 As shown, the monocular 3D target detection post-processing method includes:

[0061] S201: Obtain multiple reference detection boxes output by the monocular 3D target detection model.

[0062] Among them, the monocular 3D object detection model refers to the neural network model used to perform monocular 3D object detection tasks. Monocular means single-stage object detection.

[0063] The reference detection box refers to the detection box obtained by performing 3D target detection based on a monocular 3D target detection model.

[0064] S202: Determine the reference quality value corresponding to the reference detection frame.

[0065] The reference quality value refers to the quality value corresponding to the reference detection frame.

[0066] Optionally, in some embodiments, when determining the reference quality value corresponding to the reference detection box, the reference center coordinates and reference category confidence of the reference detection box can be determined, the second center coordinates of the object to be detected corresponding to the reference detection box can be determined, the reference center confidence of the reference detection box can be determined based on the reference center coordinates and the second center coordinates, and the reference quality value can be determined based on the reference category confidence and the reference center confidence. Thus, the reference category confidence and the reference center confidence of the reference detection box can be effectively combined to effectively improve the reliability of the obtained reference quality value.

[0067] Here, the reference center coordinates refer to the pixel center coordinates corresponding to the reference detection box. The reference class confidence score refers to the class confidence score corresponding to the reference detection box.

[0068] The second center coordinates refer to the coordinates of the center point of the object to be detected corresponding to the reference detection box in the pixel coordinate system.

[0069] S203: Based on the reference quality value, determine the candidate detection box from multiple reference detection boxes.

[0070] In this embodiment of the disclosure, when determining candidate detection boxes from multiple reference detection boxes based on reference quality values, the number to be acquired (e.g., 1000) can be preset, and then the number of reference detection boxes to be acquired can be obtained from multiple reference detection boxes as the aforementioned candidate detection boxes based on reference quality values.

[0071] In other words, in this embodiment of the present disclosure, multiple reference detection boxes output by the monocular 3D target detection model can be obtained, and the reference quality value corresponding to the reference detection box can be determined. Based on the reference quality value, candidate detection boxes can be determined from the multiple reference detection boxes. Thus, the preliminary screening of multiple reference detection boxes can be achieved based on the reference quality value to ensure the practicality of the obtained candidate detection boxes and effectively improve the post-processing efficiency of monocular 3D target detection.

[0072] S204: Obtain the detection output information and camera intrinsic parameters of the candidate detection box. The detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box.

[0073] For a detailed description of S204, please refer to the above embodiments, which will not be repeated here.

[0074] S205: Determine the three-dimensional center coordinates of the candidate detection box based on camera intrinsic parameters, pixel center coordinates, and depth information.

[0075] Among them, the three-dimensional center coordinates refer to the coordinates of the center point of the candidate detection box in the three-dimensional coordinate system.

[0076] S206: Determine the coordinates of the three-dimensional vertices corresponding to the candidate detection box based on the three-dimensional center coordinates and size information.

[0077] Among them, the three-dimensional vertex coordinates refer to the coordinates of the candidate detection box corresponding to the eight vertices in three-dimensional space.

[0078] After obtaining the detection output information and camera intrinsic parameters of the candidate detection box, the three-dimensional center coordinates of the candidate detection box can be determined based on the camera intrinsic parameters, pixel center coordinates and depth information. Based on the three-dimensional center coordinates and size information, the three-dimensional vertex coordinates corresponding to the candidate detection box can be determined, thereby providing reliable reference information for the subsequent generation of two-dimensional envelope boxes.

[0079] S207: Based on the camera intrinsic parameters, determine the minimum envelope rectangle of the 3D vertex coordinates in the target pixel coordinate system as the 2D envelope box.

[0080] The target pixel coordinate system refers to the pixel coordinate system used to obtain the two-dimensional envelope.

[0081] In this embodiment of the disclosure, the target pixel coordinate system may be, for example, a pixel coordinate system based on the forward view.

[0082] In other words, after obtaining the detection output information of the candidate detection box and the camera intrinsic parameters, the embodiments of this disclosure can determine the three-dimensional center coordinates of the candidate detection box based on the camera intrinsic parameters, pixel center coordinates and depth information, determine the three-dimensional vertex coordinates corresponding to the candidate detection box based on the three-dimensional center coordinates and size information, and determine the minimum envelope rectangle of the three-dimensional vertex coordinates in the target pixel coordinate system as the two-dimensional envelope box based on the camera intrinsic parameters. Thus, the accuracy and practicality of the obtained two-dimensional envelope box can be effectively improved based on the mutual conversion between the two-dimensional data and the three-dimensional data corresponding to the candidate detection box.

[0083] S208: Determine the two-dimensional overlap between multiple candidate detection boxes based on the two-dimensional envelope.

[0084] Among them, the two-dimensional overlap refers to the overlap between the two-dimensional envelopes of different candidate detection boxes.

[0085] For example, if the overlapping area of ​​two-dimensional envelope A and two-dimensional envelope B is 80, and the sum of the areas of two-dimensional envelope A and two-dimensional envelope B is 180, then the two-dimensional overlap between two-dimensional envelope A and two-dimensional envelope B is: 80 / (108-80)=0.8.

[0086] S209: Determine the depth similarity value between multiple candidate detection boxes based on the depth information.

[0087] Among them, depth similarity value refers to the similarity value between the depth information of different candidate detection boxes.

[0088] For example, in this embodiment of the disclosure, the depth value of the center point corresponding to the candidate detection box can be obtained, and then the ratio between the center depth values ​​corresponding to different candidate detection boxes can be used as the aforementioned depth similarity value.

[0089] S210: Determine the angular similarity value between multiple candidate detection boxes based on the rotation angle.

[0090] Among them, the angle similarity value refers to the similarity value between the rotation angles of multiple candidate detection boxes.

[0091] For example, in this embodiment of the disclosure, when determining the angular similarity value between multiple candidate detection boxes based on the rotation angle, the difference between the rotation angles of the multiple candidate detection boxes can be determined and the difference can be used as the angular similarity value.

[0092] S211: Determine the three-dimensional overlap based on the two-dimensional overlap, depth similarity, and angle similarity.

[0093] For example, in this embodiment of the disclosure, when determining the three-dimensional overlap based on the two-dimensional overlap, depth similarity value and angle similarity value, the product of the two-dimensional overlap, depth similarity value and angle similarity value can be determined as the aforementioned three-dimensional overlap.

[0094] In other words, after determining the minimum envelope rectangle of the three-dimensional vertex coordinates in the target pixel coordinate system as a two-dimensional envelope box based on the camera intrinsic parameters, the two-dimensional overlap between multiple candidate detection boxes can be determined based on the two-dimensional envelope box. Based on the depth information, the depth similarity value between multiple candidate detection boxes can be determined. Based on the rotation angle, the angular similarity value between multiple candidate detection boxes can be determined. Based on the two-dimensional overlap, depth similarity value and angular similarity value, the three-dimensional overlap can be determined. Thus, the accuracy of the obtained three-dimensional overlap can be effectively improved by combining the two-dimensional overlap, depth similarity value and angular similarity value.

[0095] S212: Determine the candidate quality value corresponding to the candidate detection box based on the candidate category confidence and pixel center coordinates.

[0096] S213: Determine the target detection box from multiple candidate detection boxes based on the candidate quality value and 3D overlap.

[0097] For a detailed description of S212 and S213, please refer to the above embodiments, and they will not be repeated here.

[0098] In this embodiment, the 3D center coordinates of the candidate detection boxes are determined based on camera intrinsic parameters, pixel center coordinates, and depth information. The 3D vertex coordinates corresponding to the candidate detection boxes are determined based on the 3D center coordinates and size information. Based on the camera intrinsic parameters, the smallest envelope rectangle of the 3D vertex coordinates in the target pixel coordinate system is determined as the 2D envelope box. Thus, the accuracy and practicality of the obtained 2D envelope box can be effectively improved by the mutual conversion between the 2D and 3D data corresponding to the candidate detection boxes. The 2D overlap between multiple candidate detection boxes is determined based on the 2D envelope box. The depth similarity value between multiple candidate detection boxes is determined based on the depth information. The angular similarity value between multiple candidate detection boxes is determined based on the rotation angle. The 3D overlap is determined based on the 2D overlap, depth similarity value, and angular similarity value. Therefore, the accuracy of the obtained 3D overlap can be effectively improved by combining the 2D overlap, depth similarity value, and angular similarity value. By acquiring multiple reference detection boxes output by a monocular 3D object detection model, a reference quality value corresponding to each reference detection box is determined. Based on the reference quality value, candidate detection boxes are selected from these multiple reference detection boxes. This allows for preliminary screening of multiple reference detection boxes based on the reference quality value, ensuring the usability of the obtained candidate detection boxes and effectively improving the post-processing efficiency of monocular 3D object detection. Furthermore, by determining the reference center coordinates and reference class confidence of each reference detection box, the second center coordinates of the object to be detected corresponding to the reference detection box are determined. Based on the reference center coordinates and the second center coordinates, the reference center confidence of the reference detection box is determined. Finally, based on the reference class confidence and the reference center confidence, the reference quality value is determined. This effectively combines the reference class confidence and the reference center confidence of the reference detection box, thereby significantly improving the reliability of the obtained reference quality value.

[0099] Figure 3 This is a schematic flowchart of a monocular three-dimensional target detection post-processing method proposed in another embodiment of this disclosure.

[0100] like Figure 3 As shown, the monocular 3D target detection post-processing method includes:

[0101] S301: Obtain the detection output information and camera intrinsic parameters of the candidate detection box. The detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box.

[0102] S302: Generate a two-dimensional envelope corresponding to the candidate detection box based on camera intrinsic parameters, pixel center coordinates, depth information, and size information.

[0103] S303: Determine the 3D overlap between multiple candidate detection boxes based on the 2D envelope, depth information, and rotation angle.

[0104] The descriptions of S301 to S303 can be found in the above embodiments, and will not be repeated here.

[0105] S304: Determine the first center coordinates of the object to be detected corresponding to the candidate detection box.

[0106] The object to be detected refers to the object to be detected during the target detection process.

[0107] The first center coordinates refer to the coordinates of the center point of the object to be detected corresponding to the candidate detection box in the pixel coordinate system.

[0108] S305: Determine the candidate center confidence of the candidate detection box based on the pixel center coordinates and the first center coordinates.

[0109] Among them, candidate center confidence refers to the center confidence of the candidate detection box.

[0110] S306: Determine the candidate quality value based on the confidence of the candidate category and the confidence of the candidate center.

[0111] In this embodiment of the disclosure, when determining the candidate quality value based on the candidate category confidence and the candidate center confidence, the product of the candidate category confidence and the candidate center confidence can be used as the candidate quality value. Alternatively, the weight coefficients of the candidate category confidence and the candidate center confidence can be determined, and then the sum of the candidate category confidence and the candidate center confidence can be determined based on the obtained weight coefficients as the candidate quality value. There is no limitation on this.

[0112] In other words, in this embodiment of the present disclosure, after determining the three-dimensional overlap between multiple candidate detection boxes based on the two-dimensional envelope, depth information, and rotation angle, the first center coordinates of the candidate detection box corresponding to the object to be detected can be determined. Based on the pixel center coordinates and the first center coordinates, the candidate center confidence of the candidate detection box is determined. Based on the candidate category confidence and the candidate center confidence, the candidate quality value is determined. Thus, reliable reference information can be provided for the post-processing of monocular three-dimensional target detection, ensuring the accuracy of the obtained target detection box.

[0113] S307: Determine the classification information of multiple candidate detection boxes.

[0114] The classification information can refer to the relevant information about the object category indicated by multiple candidate detection boxes.

[0115] For example, in a vehicle driving scenario, multiple candidate detection boxes can be used to indicate multiple different categories of objects such as vehicles, pedestrians, and obstacles.

[0116] S308: Cluster multiple candidate detection boxes based on classification information to obtain multiple detection box sets, wherein each detection box set includes at least one candidate detection box.

[0117] The detection box set can be a set of candidate detection boxes that indicate the same type of object to be identified.

[0118] S309: Determine the target detection box corresponding to the detection box set based on the candidate quality value and the three-dimensional overlap.

[0119] In this embodiment of the disclosure, when determining the target detection box corresponding to the detection box set based on the candidate quality value and the three-dimensional overlap, a third-party target detection box determination device may be used to process the candidate quality value and the three-dimensional overlap to determine the target detection box. Alternatively, any other possible method may be used to determine the target detection box corresponding to the detection box set based on the candidate quality value and the three-dimensional overlap, such as engineering or mathematical methods, without limitation.

[0120] Optionally, in some embodiments, when determining the target detection box corresponding to the detection box set based on the candidate quality value and the three-dimensional overlap, a preset overlap threshold may be obtained. In response to the existence of candidate detection boxes in the detection box set, the candidate detection box corresponding to the candidate with the largest candidate quality value in the detection box set is updated as the target detection box based on the candidate quality value. Based on the three-dimensional overlap and the preset overlap threshold, the detection box to be screened corresponding to the target detection box is determined from the detection box set, and the detection box to be screened is deleted from the detection box set. Thus, non-maximum suppression for multiple candidate detection boxes can be achieved, thereby effectively reducing redundant information in the monocular three-dimensional target detection post-processing result and effectively improving the detection accuracy of the obtained target detection box.

[0121] Among them, the preset overlap threshold refers to the threshold value configured in advance for the overlap.

[0122] In this embodiment of the disclosure, different overlap thresholds can be configured for different objects to be detected, or a uniform overlap threshold can be configured for all objects to be detected, without any limitation.

[0123] Among them, the detection boxes to be screened out refer to the candidate detection boxes in the detection box set that are to be deleted.

[0124] For example, in this embodiment of the present disclosure, when determining the target detection box corresponding to the detection box set based on the candidate quality value and the three-dimensional overlap, the following steps may be included:

[0125] 1. Sort the candidate detection boxes in the detection box set according to their candidate quality values;

[0126] 2. Select the candidate detection box with the highest candidate quality value and add it to the final output list, then remove it from the detection box set;

[0127] 3. Determine the area of ​​all candidate detection boxes;

[0128] 4. Determine the 3D overlap between the candidate detection box with the highest quality value and other candidate detection boxes;

[0129] 5. Delete candidate detection boxes whose 3D overlap is greater than a preset overlap threshold;

[0130] 6. Repeat the above process until the bounding box list is empty.

[0131] In other words, in this embodiment of the present disclosure, after determining the candidate quality value based on the candidate category confidence and the candidate center confidence, the classification information of multiple candidate detection boxes can be determined. The multiple candidate detection boxes are then clustered based on the classification information to obtain multiple detection box sets. Each detection box set includes at least one candidate detection box. Based on the candidate quality value and the three-dimensional overlap, the target detection box corresponding to the detection box set is determined. This effectively improves the systematic nature of the target detection box determination process and ensures the detection effect of the obtained target detection box on different categories of objects to be detected.

[0132] In this embodiment, by determining the first center coordinates of the object to be detected corresponding to the candidate detection box, the candidate center confidence of the candidate detection box is determined based on the pixel center coordinates and the first center coordinates. The candidate quality value is determined based on the candidate category confidence and the candidate center confidence. This provides reliable reference information for the post-processing of monocular 3D object detection, ensuring the accuracy of the obtained target detection boxes. By determining the classification information of multiple candidate detection boxes, clustering is performed on these boxes to obtain multiple detection box sets. Each detection box set includes at least one candidate detection box. Based on the candidate quality value and 3D overlap, the target detection box corresponding to the detection box set is determined. This effectively improves the systematic nature of the target detection box determination process and ensures the detection effect of the obtained target detection boxes on different categories of objects to be detected. By obtaining a preset overlap threshold, in response to the existence of candidate detection boxes in the detection box set, the candidate detection box corresponding to the candidate with the largest candidate quality value in the detection box set is updated as the target detection box according to the candidate quality value. Based on the 3D overlap and the preset overlap threshold, the detection boxes to be screened corresponding to the target detection box are determined from the detection box set and deleted from the detection box set. In this way, non-maximum suppression can be achieved for multiple candidate detection boxes, thereby effectively reducing redundant information in the post-processing results of monocular 3D target detection and effectively improving the detection accuracy of the obtained target detection boxes.

[0133] For example, the monocular 3D target detection post-processing method proposed in the embodiments of this disclosure may include the following steps:

[0134] 1. First, for a large number of 3D candidate detection boxes, select T0p K to retain based on their quality. Here, the quality of the 3D candidate detection boxes is not only determined by their class confidence, but also by the distance of each point from the target center. Ultimately, the quality of the detection box is determined by both the class confidence and the center confidence. Here, K is set to 1000 to ensure that it is much larger than the actual number of targets.

[0135] 2. The output of the monocular 3D detection model is the center coordinates, depth, size of the predicted box, rotation angle, and other information of the detection box on the image. In order to obtain the 3D detection box, it is necessary to obtain the actual 3D center (x, y, z) of the detection box in the camera coordinate system based on the camera's intrinsic parameters and the center (x, y) and depth in the pixel coordinate system.

[0136] 3. Based on the center point coordinates obtained above and the size of the bounding box predicted by the model, the three-dimensional coordinates of the eight corner points are calculated. These coordinates are then mapped back to the pixel coordinate system of the forward view through the camera intrinsic parameters. The minimum envelope of the eight corner points in the pixel coordinate system is then solved as the basis for subsequent calculation of the Intersection over Union (IoU).

[0137] 4. Calculate the IoU between 2D envelope boxes as one of the criteria for measuring the overlap of 3D candidate boxes. At the same time, calculate the similarity between rotation angles and the similarity between detection box depths. The three factors together determine the overlap between 3D candidate boxes.

[0138] 5. Arrange the candidate 3D detections from high to low according to their quality scores. Based on the overlap between the 3D candidate boxes obtained above, use the maximum suppression algorithm to filter out low-quality and redundant candidate boxes and retain a few high-quality candidate boxes for output, thus efficiently realizing the monocular 3D detection task.

[0139] Figure 4 This is a schematic diagram of the structure of a monocular three-dimensional target detection post-processing device proposed in an embodiment of this disclosure.

[0140] like Figure 4 As shown, the monocular 3D target detection post-processing device 40 includes:

[0141] The first acquisition module 401 is used to acquire the detection output information of the candidate detection box and the camera intrinsic parameters. The detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box.

[0142] The generation module 402 is used to generate a two-dimensional envelope corresponding to the candidate detection box based on camera intrinsic parameters, pixel center coordinates, depth information and size information;

[0143] The first determining module 403 is used to determine the three-dimensional overlap between multiple candidate detection boxes based on the two-dimensional envelope, depth information and rotation angle.

[0144] The second determining module 404 is used to determine the candidate quality value corresponding to the candidate detection box based on the candidate category confidence and pixel center coordinates;

[0145] The third determination module 405 is used to determine the target detection box from multiple candidate detection boxes based on the candidate quality value and the three-dimensional overlap.

[0146] In some embodiments of this disclosure, such as Figure 5 As shown, Figure 5 This is a schematic diagram of a monocular 3D target detection post-processing device according to another embodiment of the present disclosure, wherein the generation module 402 is specifically used for:

[0147] The three-dimensional center coordinates of the candidate detection box are determined based on camera intrinsic parameters, pixel center coordinates, and depth information.

[0148] Based on the 3D center coordinates and size information, determine the 3D vertex coordinates corresponding to the candidate detection box;

[0149] Based on the camera intrinsic parameters, the smallest envelope rectangle of the 3D vertex coordinates in the target pixel coordinate system is determined as the 2D envelope box.

[0150] In some embodiments of this disclosure, the first determining module 403 is specifically used for:

[0151] Based on the two-dimensional envelope, determine the two-dimensional overlap between multiple candidate detection boxes;

[0152] Based on depth information, determine the depth similarity values ​​between multiple candidate detection boxes;

[0153] Determine the angular similarity value between multiple candidate detection boxes based on the rotation angle;

[0154] The three-dimensional overlap is determined based on the two-dimensional overlap, depth similarity, and angle similarity.

[0155] In some embodiments of this disclosure, the second determining module 404 is specifically used for:

[0156] Determine the first center coordinates of the object to be detected corresponding to the candidate detection box;

[0157] Based on the pixel center coordinates and the first center coordinates, determine the candidate center confidence of the candidate detection box;

[0158] The candidate quality value is determined based on the confidence scores of the candidate categories and the candidate centers.

[0159] In some embodiments of this disclosure, the third determining module 405 is specifically used for:

[0160] Determine the classification information of multiple candidate detection boxes;

[0161] Multiple candidate detection boxes are clustered based on classification information to obtain multiple detection box sets, wherein each detection box set includes at least one candidate detection box.

[0162] Based on the candidate quality value and 3D overlap, the target detection box corresponding to the detection box set is determined.

[0163] In some embodiments of this disclosure, the third determining module 405 is further configured to:

[0164] Obtain the preset overlap threshold;

[0165] In response to the existence of candidate detection boxes in the detection box set, the candidate detection box corresponding to the candidate with the highest candidate quality value in the detection box set is updated as the target detection box based on the candidate quality value.

[0166] Based on the three-dimensional overlap and the preset overlap threshold, the detection boxes to be screened are determined from the detection box set that correspond to the target detection box;

[0167] Remove the detection boxes to be screened from the detection box set.

[0168] In some embodiments of this disclosure, it also includes:

[0169] The second acquisition module 406 is used to acquire multiple reference detection boxes output by the monocular 3D target detection model;

[0170] The fourth determining module 407 is used to determine the reference quality value corresponding to the reference detection frame;

[0171] The fifth determination module 408 determines candidate detection boxes from multiple reference detection boxes based on the reference quality value.

[0172] In some embodiments of this disclosure, the fourth determining module 407 is specifically used for:

[0173] Determine the reference center coordinates and reference category confidence level of the reference detection box;

[0174] Determine the second center coordinates of the object to be detected corresponding to the reference detection frame;

[0175] The reference center confidence level of the reference detection box is determined based on the reference center coordinates and the second center coordinates.

[0176] The reference quality value is determined based on the reference category confidence level and the reference center confidence level.

[0177] It should be noted that the foregoing explanation of the monocular 3D target detection post-processing method also applies to the monocular 3D target detection post-processing device of this embodiment, and will not be repeated here.

[0178] In this embodiment, by acquiring the detection output information of candidate detection boxes and camera intrinsic parameters, the detection output information includes: candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box. Based on the camera intrinsic parameters, pixel center coordinates, depth information, and size information, a two-dimensional envelope box corresponding to the candidate detection box is generated. Based on the two-dimensional envelope box, depth information, and rotation angle, the three-dimensional overlap between multiple candidate detection boxes is determined. Based on the candidate category confidence and pixel center coordinates, the candidate quality value corresponding to the candidate detection box is determined. Based on the candidate quality value and the three-dimensional overlap, the target detection box is determined from multiple candidate detection boxes. Thus, the post-processing calculation process can be decoupled to reduce computational overhead and effectively improve the accuracy and real-time performance of the obtained target detection box.

[0179] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 6 The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0180] like Figure 6 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0181] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0182] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0183] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive".

[0184] although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0185] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0186] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable human interaction with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0187] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the monocular three-dimensional target detection post-processing method mentioned in the foregoing embodiments.

[0188] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the monocular three-dimensional target detection post-processing method as proposed in the foregoing embodiments of this disclosure.

[0189] To implement the above embodiments, this disclosure also proposes a computer program product that, when executed by an instruction processor, performs the monocular 3D target detection post-processing method as proposed in the foregoing embodiments of this disclosure.

[0190] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0191] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0192] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0193] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0194] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0195] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0196] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0197] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0198] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0199] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A post-processing method for monocular three-dimensional target detection, characterized in that, include: Obtain the detection output information and camera intrinsic parameters of the candidate detection box, wherein the detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box; Based on the camera intrinsic parameters, the pixel center coordinates, the depth information, and the size information, a two-dimensional envelope box corresponding to the candidate detection box is generated. The three-dimensional overlap between the multiple candidate detection boxes is determined based on the two-dimensional envelope, the depth information, and the rotation angle. Based on the candidate category confidence level and the pixel center coordinates, determine the candidate quality value corresponding to the candidate detection box; Based on the candidate quality value and the three-dimensional overlap, a target detection box is determined from a plurality of candidate detection boxes; The step of determining the three-dimensional overlap between multiple candidate detection boxes based on the two-dimensional envelope, the depth information, and the rotation angle includes: Based on the two-dimensional envelope, determine the two-dimensional overlap between the multiple candidate detection boxes; Based on the depth information, a depth similarity value is determined among the multiple candidate detection boxes; Based on the rotation angle, determine the angular similarity value between the multiple candidate detection boxes; The three-dimensional overlap is determined based on the two-dimensional overlap, the depth similarity value, and the angle similarity value. The step of determining the candidate quality value corresponding to the candidate detection box based on the candidate category confidence score and the pixel center coordinates includes: Determine the first center coordinates of the object to be detected corresponding to the candidate detection box; The candidate center confidence score of the candidate detection box is determined based on the pixel center coordinates and the first center coordinates. The candidate quality value is determined based on the confidence scores of the candidate categories and the candidate centers.

2. The method as described in claim 1, characterized in that, The step of generating a two-dimensional envelope corresponding to the candidate detection box based on the camera intrinsic parameters, the pixel center coordinates, the depth information, and the size information includes: The three-dimensional center coordinates of the candidate detection box are determined based on the camera intrinsic parameters, the pixel center coordinates, and the depth information. Based on the three-dimensional center coordinates and the size information, determine the three-dimensional vertex coordinates corresponding to the candidate detection box; Based on the camera intrinsic parameters, the minimum envelope rectangle of the three-dimensional vertex coordinates in the target pixel coordinate system is determined as the two-dimensional envelope box.

3. The method as described in claim 1, characterized in that, The step of determining a target detection box from a plurality of candidate detection boxes based on the candidate quality value and the three-dimensional overlap includes: Determine the classification information of multiple candidate detection boxes; Clustering is performed on the multiple candidate detection boxes based on the classification information to obtain multiple detection box sets, wherein each detection box set includes at least one of the candidate detection boxes; Based on the candidate quality value and the three-dimensional overlap, the target detection box corresponding to the detection box set is determined.

4. The method as described in claim 3, characterized in that, The step of determining the target detection box corresponding to the detection box set based on the candidate quality value and the three-dimensional overlap includes: Obtain the preset overlap threshold; In response to the existence of the candidate detection box in the detection box set, the candidate detection box corresponding to the candidate with the largest candidate quality value in the detection box set is updated to the target detection box according to the candidate quality value; Based on the three-dimensional overlap and the preset overlap threshold, a detection box to be screened out corresponding to the target detection box is determined from the detection box set; Remove the detection box to be screened from the set of detection boxes.

5. The method as described in claim 1, characterized in that, Also includes: Obtain multiple reference detection boxes output by the monocular 3D target detection model; Determine the reference quality value corresponding to the reference detection frame; The candidate detection box is determined from the plurality of reference detection boxes based on the reference quality value.

6. The method as described in claim 5, characterized in that, Determining the reference quality value corresponding to the reference detection frame includes: Determine the reference center coordinates and reference category confidence level of the reference detection box; Determine the second center coordinates of the object to be detected corresponding to the reference detection frame; The reference center confidence level of the reference detection box is determined based on the reference center coordinates and the second center coordinates. The reference quality value is determined based on the reference category confidence level and the reference center confidence level.

7. A monocular three-dimensional target detection post-processing device, characterized in that, include: The first acquisition module is used to acquire the detection output information and camera intrinsic parameters of the candidate detection box, wherein the detection output information includes: the candidate category confidence, pixel center coordinates, depth information, size information, and rotation angle corresponding to the candidate detection box; The generation module is used to generate a two-dimensional envelope corresponding to the candidate detection box based on the camera intrinsic parameters, the pixel center coordinates, the depth information and the size information; The first determining module is used to determine the three-dimensional overlap between multiple candidate detection boxes based on the two-dimensional envelope, the depth information, and the rotation angle. The second determining module is used to determine the candidate quality value corresponding to the candidate detection box based on the candidate category confidence and the pixel center coordinates. The third determining module is used to determine the target detection box from the plurality of candidate detection boxes based on the candidate quality value and the three-dimensional overlap. The first determining module is specifically used for: Based on the two-dimensional envelope, determine the two-dimensional overlap between the multiple candidate detection boxes; Based on the depth information, a depth similarity value is determined among the multiple candidate detection boxes; Based on the rotation angle, determine the angular similarity value between the multiple candidate detection boxes; The three-dimensional overlap is determined based on the two-dimensional overlap, the depth similarity value, and the angle similarity value. The second determining module is specifically used for: Determine the first center coordinates of the object to be detected corresponding to the candidate detection box; The candidate center confidence score of the candidate detection box is determined based on the pixel center coordinates and the first center coordinates. The candidate quality value is determined based on the confidence scores of the candidate categories and the candidate centers.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images