Method, apparatus, device and medium for determining object class for high-definition map

Abstract

This disclosure provides a method for determining object categories, relating to the field of autonomous driving technology, and particularly to computer vision technology and high-precision mapping technology. The specific implementation scheme is as follows: For N image frames, determine the weight of a first category of the target object in each image frame, obtaining N weights corresponding one-to-one with the N image frames, where there are M first categories, and each first category corresponds to at least one of the N weights, where N and M are both integers greater than or equal to 1; for the M first categories, determine a fusion weight for each first category based on at least one weight corresponding to it, obtaining M fusion weights; and determine a second category of the target object based on the M fusion weights. This disclosure also provides an apparatus, electronic device, and storage medium for determining object categories.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and more particularly to computer vision technology and high-precision mapping technology. More specifically, this disclosure provides a method, apparatus, electronic device, and storage medium for determining object categories. Background Technology

[0002] High-precision maps, also known as high-resolution maps, are used by autonomous vehicles. They possess accurate vehicle location information and rich road element data, helping cars anticipate complex road conditions such as slope, curvature, and heading, thus better avoiding potential risks. For example, autonomous vehicles can determine their current position by matching lane line information with data in a high-precision map. The classification of a specific object can be determined through manual annotation or by recognizing a single frame of an image. For instance, the line type or color of a point on a road lane can be identified using manual annotation or by recognizing a single frame of a road video. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for determining object categories.

[0004] According to a first aspect, a method for determining an object category is provided, the method comprising: for N image frames, determining the weight of a first category of a target object in each image frame to obtain N weights corresponding one-to-one with the N image frames, wherein there are M first categories, each first category corresponding to at least one of the N weights, and N and M are both integers greater than or equal to 1; for the M first categories, determining a fusion weight of the first category based on at least one weight corresponding to each first category to obtain up to M fusion weights; and determining a second category of the target object based on the M fusion weights.

[0005] According to a second aspect, an apparatus for determining an object category is provided, the apparatus comprising: a first determining module, configured to determine, for N image frames, a weight of a first category of a target object in each image frame, to obtain N weights corresponding one-to-one with the N image frames, wherein there are M first categories, each first category corresponding to at least one of the N weights, and N and M are both integers greater than or equal to 1; a second determining module, configured to determine a fusion weight of the M first categories based on at least one weight corresponding to each first category, to obtain M fusion weights; and a third determining module, configured to determine a second category of the target object based on the M fusion weights.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to the present disclosure.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the methods provided in this disclosure.

[0008] According to a fifth aspect, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided in this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1A This is an exemplary system architecture diagram illustrating a method and apparatus for determining object categories that can be applied according to an embodiment of this disclosure;

[0012] Figure 1B This is a schematic diagram illustrating an application scenario of a method and apparatus for determining object categories according to an embodiment of the present disclosure;

[0013] Figure 2 This is a flowchart of a method for determining an object category according to an embodiment of the present disclosure;

[0014] Figure 3 This is a flowchart of a method for determining an object category according to another embodiment of this disclosure;

[0015] Figure 4 This is a schematic diagram of multiple objects according to an embodiment of the present disclosure;

[0016] Figure 5 This is a schematic diagram of the output of a method for determining an object category according to an embodiment of the present disclosure;

[0017] Figure 6 This is a block diagram of an apparatus for determining an object category according to an embodiment of the present disclosure; and

[0018] Figure 7 This is a block diagram of an electronic device for a method of determining an object category according to an embodiment of the present disclosure. Detailed Implementation

[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] In related technologies, manual annotation can be used to identify the category of objects. However, manual annotation requires a lot of manpower and has a high time cost, so it can only be applied to small-scale object recognition scenarios.

[0021] While recognizing single-frame images can reduce labor and time costs, it may lead to incorrect object classifications due to issues such as recognition algorithms, object occlusion, shooting angle, and shooting distance.

[0022] A single frame can be a single image frame from surveillance video or video captured by a vehicle. Multiple image frames in a video can contain the same object, and the object's category can be identified based on each of these image frames. However, for the same object, different image frames may yield different categories.

[0023] Figure 1A This is an exemplary system architecture diagram illustrating a method and apparatus for determining object categories that can be applied according to an embodiment of this disclosure. It should be noted that... Figure 1A The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0024] like Figure 1A As shown, the system architecture 100 according to this embodiment may include multiple terminal devices 101, a network 102, and a server 103. The network 102 serves as a medium for providing a communication link between the terminal devices 101 and the server 103. The network 102 may include various connection types, such as wired and / or wireless communication links, etc.

[0025] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptops, etc.

[0026] The method for determining object categories provided in this disclosure can generally be executed by server 103. Correspondingly, the apparatus for determining object categories provided in this disclosure can generally be located in server 103. The method for determining object categories provided in this disclosure can also be executed by a server or server cluster that is different from server 103 and capable of communicating with terminal device 101 and / or server 103. Correspondingly, the apparatus for determining object categories provided in this disclosure can also be located in a server or server cluster that is different from server 103 and capable of communicating with terminal device 101 and / or server 103.

[0027] Figure 1B This is a schematic diagram illustrating an application scenario of a method and apparatus for determining object categories according to an embodiment of the present disclosure.

[0028] like Figure 1B As shown, road 104 may include four lanes: a first lane 1041, a second lane 1042, a third lane 1043, and a fourth lane 1044. Road 104 has three lane markings: a first lane marking 1051, a second lane marking 1052, and a third lane marking 1053. The first lane marking 1051 has multiple objects, two of which are object 106 and object 107.

[0029] The detection equipment can move across four lanes and is equipped with a camera.

[0030] During a video capture process, detection device A 108 can move on the first lane 1041 and capture video using its camera. During the movement of detection device A 108, multiple image frames in the captured first video contain object 106. For each image frame containing object 106 in the first video, a classification algorithm or semantic recognition algorithm can be used to determine the first category of object 106 in each image frame. In a subset of the image frames containing object 106, the first category of object 106 is a single dashed line. In another subset of the image frames containing object 106, the first category of object 106 is a single solid line.

[0031] During another video capture operation, detection device B 109 can move on lane 4 1044 and capture video using its camera. During the movement of detection device B 109, multiple image frames in the captured second video contain object 106. For each image frame containing object 106 in the second video, a classification algorithm or semantic recognition algorithm can be used to determine the first category of object 106 in each image frame. In a subset of the image frames containing object 106, the first category of object 106 is a single dashed line. In another subset of the image frames containing object 106, the first category of object 106 is a single solid line.

[0032] Figure 2 This is a flowchart of a method for determining an object category according to an embodiment of the present disclosure.

[0033] like Figure 2 As shown, the method 200 may include operations S210 to S230.

[0034] In operation S210, for N image frames, the weight of the first category of the target object in each image frame is determined, and N weights corresponding one-to-one with the N image frames are obtained.

[0035] In this embodiment of the disclosure, there are M first categories, and each first category corresponds to at least one of N weights, where N and M are both integers greater than or equal to 1.

[0036] In one example, M is a positive integer less than or equal to N.

[0037] For example, there can be 10 image frames, and based on these 10 image frames, the first category of the target object can be determined to be 3.

[0038] In this embodiment of the disclosure, the target object may be a point on the lane line of a road.

[0039] For example, the target object could be, for example Figure 1B The object 106 is on the first lane line 1051. For example, the target object could be, for example... Figure 1B Object 107 on the first lane line 1051.

[0040] In this embodiment of the disclosure, the N image frames can be image frames containing the target object.

[0041] For example, each of the N image frames contains, for example Figure 1B Object 106 in the context.

[0042] In this embodiment of the disclosure, the N image frames may be obtained from the video captured by the detection device.

[0043] For example, the detection device can be a mobile image acquisition device. In one example, the detection device can be a vehicle equipped with a camera.

[0044] In this embodiment of the disclosure, the N image frames may be obtained from one video or from multiple videos.

[0045] For example, for example Figure 1B The detection device A 108 can move on the first lane 1041 and record video using its camera device for video acquisition. N image frames can be obtained from the video recorded by the detection device A 108.

[0046] For example, for example Figure 1B The detection device B 109 can move in the fourth lane 1044 and record video using its camera device for video acquisition. N image frames can be obtained from the video recorded by the detection device B 109.

[0047] For example, for example Figure 1B The detection device A 108 can move on the first lane 1041, for example... Figure 1B Detection device B 109 can move on the fourth lane 1044 and record video using its own camera device for video acquisition. A total of N image frames can be obtained from the two videos recorded by detection device A 108 and detection device B 109.

[0048] In this embodiment of the disclosure, the first category of the target object may be the line type or color of a point on the lane line.

[0049] For example, line types can include single solid line, single dashed line, double solid line, double dashed line, left solid and right dashed line, and left dashed and right solid line, etc.

[0050] For example, colors can include yellow and white, etc.

[0051] In this embodiment of the disclosure, for each image frame, the actual distance between the target object and the detection device can be obtained to determine the first weight of the first category of the target object in the image frame.

[0052] For example, the first weight of the first category of the target object in the image frame can be determined by the following formula:

[0053]

[0054] d is the actual distance between the target object and the detection equipment, w i1 This is the first weight for the first category of the target object in this image frame. When the distance d between the detection device and the target object does not exceed d... lWhen the distance d is small, the image distortion of the image frame containing the target object is small, and the first weight of the target object in these image frames can be 1.0. When the distance d increases to d... h Subsequently, although the video captured by the detection device still contains the target object, the image frames are severely distorted due to the distance between the detection device and the target object. Therefore, the first category determined based on these image frames is inaccurate. In this case, the first weight of the target object in these image frames can be set to an empirical value of 0.6. In one example, f(d) is an inverse proportional function.

[0055] In one example, you can obtain, for example Figure 1B The first actual location of object 106 (e.g., geographic coordinates obtained through satellite positioning), and the information that can be obtained, for example... Figure 1B The second actual position (e.g., geographic coordinates obtained through satellite positioning) of the detection device A 108 corresponding to the image frame can be determined. Furthermore, the actual distance d between the detection device A 108 and the object 106 can be obtained.

[0056] In this embodiment of the disclosure, the weight of a first category of the target object in the image frame can be determined based on a first weight.

[0057] For example, the first weight can be used as the weight of the first category of the target object in the image frame.

[0058] In this embodiment of the disclosure, for each image frame, the first actual position of the target object and the second actual position of the detection device are obtained to determine the second weight of the first category of the target object in the image frame.

[0059] For example, the second actual position of the detection device is variable; it differs for different image frames. In one example, for each image frame, the second actual position of the detection device corresponding to that image frame can be obtained.

[0060] For example, the second weight of the first category of the target object in this image frame can be determined by the following formula:

[0061]

[0062] k is a positive integer greater than or equal to 0, where k represents the number of direction changes required for the detection device to reach the first actual position. i2 The second weight is the first category of the target object in this image frame.

[0063] In one example, for a single image frame, the lane in which the target object is located can be determined based on its first actual position. The lane in which the detection device is located can be determined based on its second actual position. Based on the lane in which the target object is located and the lane in which the detection device is located, the number of direction changes required for the detection device to reach the first actual position of the target object can be determined.

[0064] It should be noted that k can be the minimum number of forward direction changes required for the detection device to reach the first actual position.

[0065] For example, it can be based on, for example Figure 1B Based on the first actual location of object 106 (e.g., geographic coordinates obtained through satellite positioning), determine whether object 106 is located in lane 1041 or lane 1042. This can be done based on, for example... Figure 1B Based on the second actual location of detection device A 108 (e.g., geographical coordinates obtained through satellite positioning), the lane where detection device A 108 is located is determined to be the first lane 1041. According to the lane where detection device A 108 is located and the lane where object 106 is located, it can be determined that the number of direction changes required for detection device A 108 to reach the lane where object 106 is located is 0. That is, in this case, for example, k = 0 in Formula 2.

[0066] For example, it can be based on, for instance Figure 1B Based on the actual location of object 106 (e.g., geographic coordinates obtained through satellite positioning), determine whether object 106 is located in lane 1041 or lane 1042. This can be done based on, for example... Figure 1B Based on the actual location of detection device B 109 (e.g., geographical coordinates obtained through satellite positioning), the lane where detection device B 109 is located is determined to be the fourth lane, 1044. According to the lane where detection device B 109 is located and the lane where object 106 is located, it can be determined that the number of direction changes required for detection device B 109 to reach the lane where object 106 is located is 2, i.e., in this case, for example, k = 2 in Formula 2.

[0067] It should be noted that, for example, Figure 1B In application scenarios, the number of lane changes can also be understood as the number of times the detection device changes lanes. According to relevant laws and regulations, only one lane can be changed during a single lane change. For example... Figure 1B The detection equipment B 109 needs to change lanes at least twice to reach the second lane 1042.

[0068] In this embodiment of the disclosure, the weight of the first category of the target object in the image frame is determined based on the second weight.

[0069] For example, the second weight can be used as the weight of the first category of the target object in the image frame.

[0070] In this embodiment of the disclosure, for each image frame, a third weight of the first category of the target object in the image frame can be determined based on the first weight, the second weight, and the moving speed of the detection device.

[0071] For example, the movement speed of the detection device is variable. The movement speed of the detection device can also be different for different image frames. In one example, for each image frame, the movement speed of the detection device corresponding to that image frame can be obtained.

[0072] For example, the third weight of the first category of the target object in this image frame can be determined by the following formula:

[0073]

[0074] w i1 w represents the first weight of the first category of the target object in this image frame. i2 v represents the second weight of the first category of the target object in this image frame. i To detect the movement speed of the device corresponding to the image frame, w i3 This is the third weight for the first category of the target object in this image frame. The category determination is more accurate when the vehicle speed is slower. Based on the third weight obtained from Formula 3, the object's category can be determined more accurately.

[0075] In this embodiment of the disclosure, the weight of the first category of the target object in the image frame is determined based on the third weight.

[0076] For example, the third weight can be used as the weight of the first category of the target object in the image frame.

[0077] In operation S220, for M first categories, the fusion weight of the first category is determined according to at least one weight corresponding to each first category, thus obtaining M fusion weights.

[0078] For example, the fusion weight of the first category can be determined using the following formula:

[0079]

[0080] W I This represents the fusion weights for the first category, which corresponds to J weights out of N weights, where J is a positive integer greater than or equal to 1. J is a positive integer less than or equal to N.

[0081] In operation S230, the second category of the target object is determined based on M fusion weights.

[0082] In this embodiment of the disclosure, the first category corresponding to the largest fusion weight can be determined as the second category of the target object.

[0083] For example, in, for example Figure 1B During the movement of the first lane 1041, the detection device A 108 captures N image frames containing object 106 in the video. In one example, N = 5 and M = 2. In three of the five image frames, object 106 is identified as a single dashed line. Therefore, when the first category is a single dashed line, the fusion weight for the first category is 2.8. In two of the five image frames, object 106 is identified as a single solid line. Therefore, when the first category is a single solid line, the fusion weight for the first category is 1.2. The maximum fusion weight is 2.8, and the first category corresponding to the maximum fusion weight is a single dashed line. Therefore, the single dashed line can be identified as the second category of object 106.

[0084] The embodiments disclosed herein can effectively reduce the cost of manual annotation and more accurately determine the category of objects. They can resolve the problem of category determination errors caused by the distance between the detection device and the target object, and increase the possibility of automated production of lane line crowdsourced mapping.

[0085] In some embodiments, determining the weight of a first category of a target object in each image frame includes: for each image frame, performing the following operations: obtaining the actual distance between the target object and the detection device to determine a first weight of the first category of the target object in the image frame; obtaining a first actual position of the target object and a second actual position of the detection device to determine a second weight of the first category of the target object in the image frame; determining a third weight of the first category of the target object in the image frame based on the first weight, the second weight, and the moving speed of the detection device; and determining the weight of the first category of the target object in the image frame based on the third weight.

[0086] Figure 3 This is a flowchart of a method for determining an object category according to another embodiment of the present disclosure.

[0087] like Figure 3 As shown, this method can be used in, for example... Figure 2 The operation will be executed after S230. The following will be a detailed explanation of operations S340 to S350.

[0088] In operation S340, based on the category of at least one neighboring object adjacent to the target object, the number of second categories that are different from the target object among the at least one neighboring object is determined.

[0089] In this embodiment of the disclosure, the number of second categories that are different from the target object among the two adjacent objects is determined based on the categories of the two adjacent objects adjacent to the target object.

[0090] For example, along the forward direction of the detection equipment, one adjacent object is in front of the target object, and another adjacent object is behind the target object.

[0091] In one example, it can be determined that among two adjacent objects, the number of objects belonging to the second category that are different from the target object is 2.

[0092] In one example, it can be determined that the number of second-category objects that are different from the target object in two adjacent objects is 1.

[0093] In operation S350, in response to the quantity being greater than or equal to a preset quantity threshold, the second category of the target object is changed.

[0094] For example, along the forward direction of the detection equipment, one adjacent object is in front of the target object, and another adjacent object is behind the target object. In this case, the preset quantity threshold can be 2.

[0095] In one example, it can be determined that among two adjacent objects, the number of those belonging to a second category different from the target object is 2. The second category of the target object can be changed to the category of one of the adjacent objects.

[0096] In one example, it can be determined that among two adjacent objects, the number of those belonging to a second category different from the target object is 1. The second category of the target object can remain unchanged.

[0097] Through the embodiments disclosed herein, the category of the target object can be determined more accurately, avoiding abrupt changes in the type of certain points on the lane line.

[0098] Figure 4 This is a schematic diagram of a second category of multiple objects according to an embodiment of the present disclosure.

[0099] like Figure 4 As shown, in, for example Figure 2 After operation S230, a second category of multiple objects can be obtained. Figure 4 In this context, if the objects belong to different second categories, their shapes will differ. For example, objects 401 and 402 belong to different second categories. Objects 402 and 403 belong to the same second category. Figure 4 In this context, the forward direction of the detection equipment is the direction from object 401 to object 403.

[0100] For example, when the target object is object 401, along the forward direction of the detection device, an adjacent object 405 is in front of object 401, and another adjacent object 404 is behind object 401. At this time, it can be determined that the categories of both adjacent objects 404 and 405 are different from the target object 401. Therefore, the number of second categories different from object 401 is 2, and the second category of object 401 can be changed to the category of adjacent object 404 or adjacent object 405.

[0101] For example, when the target object is object 402, along the forward direction of the detection device, an adjacent object 406 is in front of object 402, and another adjacent object 405 is behind object 402. At this time, it can be determined that of the two adjacent objects 405 and 406, 405 has the same category as the target object 402, while 406 has a different category. Therefore, the number of objects in the second category different from object 402 is 1, and the second category of object 402 can be left unchanged.

[0102] For example, when the target object is object 403, along the forward direction of the detection device, an adjacent object 408 is in front of object 403, and another adjacent object 407 is behind object 403. At this time, it can be determined that the categories of both adjacent objects 407 and 408 are different from the target object 403. Since the number of second categories different from object 403 is 2, the second category of object 403 can be changed to the category of the adjacent object.

[0103] In one example Figure 4 The multiple objects shown are, for example Figure 1B Multiple points on lane 1051.

[0104] Figure 5 This is a schematic diagram of the output of a method for determining object categories according to an embodiment of the present disclosure.

[0105] like Figure 5 As shown, for example Figure 4 The second category of object 401 and the second category of object 403 are changed to the same category as the adjacent objects.

[0106] Figure 6 This is a block diagram of an apparatus for determining an object category according to an embodiment of the present disclosure.

[0107] like Figure 6 As shown, the device 600 may include a first determining module 610, a second determining module 620, and a third determining module 630.

[0108] The first determining module 610 is used to determine the weight of the first category of the target object in each of the N image frames for N image frames, and obtain N weights corresponding one-to-one with the N image frames. There are M first categories, and each first category corresponds to at least one of the N weights. N and M are both integers greater than or equal to 1.

[0109] The second determining module 620 is used to determine the fusion weight of each of the M first categories based on at least one weight corresponding to each first category, thereby obtaining M fusion weights.

[0110] The third determining module 630 is used to determine the second category of the target object based on the M fusion weights.

[0111] In some embodiments, the first determining module includes: a first acquiring submodule, configured to acquire, for each image frame, the actual distance between the target object and the detection device, so as to determine a first weight of a first category of the target object in the image frame; and a first determining submodule, configured to determine the weight of the first category of the target object in the image frame based on the first weight.

[0112] In some embodiments, the first determining module includes: a second obtaining submodule, configured to, for each image frame, obtain a first actual position of the target object and a second actual position of the detection device, so as to determine a second weight of a first category of the target object in the image frame; and a second determining submodule, configured to, based on the second weight, determine the weight of the first category of the target object in the image frame.

[0113] In some embodiments, the first determining module includes: an execution submodule, configured to perform related operations for each image frame by means of the following units: a first acquisition unit, configured to acquire the actual distance between the target object and the detection device to determine a first weight of a first category of the target object in the image frame; a second acquisition unit, configured to acquire a first actual position of the target object and a second actual position of the detection device to determine a second weight of the first category of the target object in the image frame; a first determining unit, configured to determine a third weight of the first category of the target object in the image frame based on the first weight, the second weight and the moving speed of the detection device; and a second determining unit, configured to determine the weight of the first category of the target object in the image frame based on the third weight.

[0114] In some embodiments, the first determining module is further configured to: determine a first weight of a first category of the target object in the image frame using the following formula: Where d represents the actual distance between the target object and the detection device, and w i1The first weight is assigned to the first category of the target object in the image frame.

[0115] In some embodiments, the first determining module is further configured to: determine a second weight of a first category of the target object in the image frame using the following formula: Where k is a positive integer greater than or equal to 0, k is the number of times the detection device needs to change its forward direction to reach the first actual position, and w i2 The second weight is the first category of the target object in the image frame.

[0116] In some embodiments, the first determining module is further configured to: determine a third weight of a first category of the target object in the image frame using the following formula: Among them, w i1 w represents the first weight of the first category of the target object in the image frame. i2 v is the second weight for the first category of the target object in this image frame. i The moving speed of the detection device corresponding to the image frame.

[0117] In some embodiments, the second determining module is further configured to: determine the fusion weight of the first category using the following formula: Among them, W I The first category is the fusion weight, which corresponds to J weights of the N weights, where J is a positive integer greater than or equal to 1.

[0118] In some embodiments, the third determining module is further configured to: determine the first category corresponding to the largest fusion weight as the second category of the target object.

[0119] In some embodiments, the device 600 further includes: a fourth determining module, configured to determine, based on the category of at least one adjacent object adjacent to the target object, the number of second categories different from the target object among the at least one adjacent object; and a changing module, configured to change the second category of the target object in response to the number being greater than or equal to a preset number threshold.

[0120] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0121] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0122] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0123] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0124] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the method for determining object categories. For example, in some embodiments, the method for determining object categories may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the method for determining object categories described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the method for determining object categories by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0131] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0132] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining the category of lane line objects in a high-precision map, comprising: For N image frames, the weight of the first category of the target object in each image frame is determined, resulting in N weights corresponding one-to-one with the N image frames. There are M first categories, and each first category corresponds to at least one of the N weights. N and M are both integers greater than or equal to 1. The N image frames are obtained from the video captured by the detection device, which is a mobile image acquisition device. The target object is a point on the lane line. For each of the M first categories, a fusion weight is determined based on at least one weight corresponding to each first category, resulting in M ​​fusion weights. as well as Based on the M fusion weights, the second category of the target object is determined. The weighting of the first category of the target object in each image frame includes: For each image frame, perform the following operations: The actual distance between the target object and the detection device is obtained to determine the first weight of the first category of the target object in the image frame; Obtaining the first actual position of the target object and the second actual position of the detection device to determine the second weight of the first category of the target object in the image frame includes: determining the second weight of the first category of the target object in the image frame using the following formula: in, A positive integer greater than or equal to 0. The number of times the detection device needs to change its direction of travel to reach the first actual position. The second weight for the first category of the target object in the image frame; Based on the first weight, the second weight, and the moving speed of the detection device, a third weight for the first category of the target object in the image frame is determined; and Based on this third weight, the weight of the first category of the target object in the image frame is determined.

2. The method according to claim 1, wherein, The weighting of the first category of the target object in each image frame includes: For each image frame, the actual distance between the target object and the detection device is obtained to determine the first weight of the first category of the target object in that image frame; Based on this first weight, the weight of the first category of the target object in the image frame is determined.

3. The method according to claim 1, wherein, The weighting of the first category of the target object in each image frame includes: For each image frame, the first actual position of the target object and the second actual position of the detection device are obtained to determine the second weight of the first category of the target object in that image frame; Based on the second weight, the weight of the first category of the target object in the image frame is determined.

4. The method according to claim 1 or 2, wherein, The first weight for determining the first category of the target object in the image frame includes: The first weight of the first category of the target object in the image frame is determined by the following formula: in, Based on the actual distance between the target object and the detection device, The first weight is assigned to the first category of the target object in the image frame.

5. The method according to claim 1, wherein, The third weight for determining the first category of the target object in the image frame includes: The third weight of the first category of the target object in the image frame is determined by the following formula: in, The first weight of the first category of the target object in the image frame. The second weight is assigned to the first category of the target object in the image frame. The moving speed of the detection device corresponding to the image frame.

6. The method according to claim 5, wherein, The determination of the fusion weight for the first category includes: The fusion weight for the first category is determined using the following formula: in, The fusion weight for the first category, which corresponds to the N weights. Each weight, It is a positive integer greater than or equal to 1.

7. The method according to claim 1, wherein, The second category for determining the target object includes: The first category corresponding to the largest fusion weight is determined as the second category of the target object.

8. The method according to claim 1, further comprising: Based on the category of at least one neighboring object adjacent to the target object, determine the number of second categories among the at least one neighboring objects that are different from the target object; In response to the quantity being greater than or equal to a preset quantity threshold, the second category of the target object is changed.

9. An apparatus for determining the category of lane line objects in a high-precision map, comprising: The first determining module is used to determine the weight of the first category of the target object in each of the N image frames for N image frames, thereby obtaining N weights that correspond one-to-one with the N image frames. There are M first categories, and each first category corresponds to at least one of the N weights. N and M are both integers greater than or equal to 1. The N image frames are obtained from the video captured by the detection device, which is a movable image acquisition device. The target object is a point on the lane line. The second determining module is used to determine the fusion weight of each of the M first categories based on at least one weight corresponding to each first category, thereby obtaining M fusion weights. as well as The third determining module is used to determine the second category of the target object based on the M fusion weights. The first determining module includes: The execution submodule is used to perform relevant operations for each image frame through the following units: The first acquisition unit is used to acquire the actual distance between the target object and the detection device in order to determine the first weight of the first category of the target object in the image frame; The second acquisition unit is used to acquire the first actual position of the target object and the second actual position of the detection device, so as to determine the second weight of the first category of the target object in the image frame; The first determining unit is configured to determine a third weight for a first category of the target object in the image frame based on the first weight, the second weight, and the moving speed of the detection device; and The second determining unit is used to determine the weight of the first category of the target object in the image frame based on the third weight. The first determining module is further configured to: The second weight of the first category of the target object in the image frame is determined by the following formula: in, A positive integer greater than or equal to 0. The number of times the detection device needs to change its direction of travel to reach the first actual position. The second weight is the first category of the target object in the image frame.

10. The apparatus according to claim 9, wherein, The first determining module includes: The first acquisition submodule is used to acquire the actual distance between the target object and the detection device for each image frame, so as to determine the first weight of the first category of the target object in the image frame; The first determining submodule is used to determine the weight of the first category of the target object in the image frame based on the first weight.

11. The apparatus according to claim 9, wherein, The first determining module includes: The second acquisition submodule is used to acquire, for each image frame, the first actual position of the target object and the second actual position of the detection device, so as to determine the second weight of the first category of the target object in the image frame; The second determining submodule is used to determine the weight of the first category of the target object in the image frame based on the second weight.

12. The apparatus according to claim 9 or 10, wherein, The first determining module is further configured to: The first weight of the first category of the target object in the image frame is determined by the following formula: in, Based on the actual distance between the target object and the detection device, The first weight is assigned to the first category of the target object in the image frame.

13. The apparatus according to claim 9, wherein, The first determining module is further configured to: The third weight of the first category of the target object in the image frame is determined by the following formula: in, The first weight of the first category of the target object in the image frame. The second weight is assigned to the first category of the target object in the image frame. The moving speed of the detection device corresponding to the image frame.

14. The apparatus according to claim 13, wherein, The second determining module is also used for: The fusion weight for the first category is determined using the following formula: in, The fusion weight for the first category, which corresponds to the N weights. Each weight, It is a positive integer greater than or equal to 1.

15. The apparatus according to claim 9, wherein, The third determining module is also used for: The first category corresponding to the largest fusion weight is determined as the second category of the target object.

16. The apparatus of claim 9, further comprising: The fourth determining module is used to determine, based on the category of at least one adjacent object adjacent to the target object, the number of second categories that are different from the target object among the at least one adjacent object; The module is configured to change the second category of the target object in response to the quantity being greater than or equal to a preset quantity threshold.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images