Environment data quality judgment method, system and device

Abstract

The embodiment of the invention provides an environmental data quality judgment method, system and device, and the method comprises the steps: obtaining a target image, and determining a first element based on the target image and a semantic segmentation model; acquiring environment data of a scene corresponding to the target image based on positioning information, projecting the environment data to the target image, and determining a second element; determining the recall rate of the first element based on the first element and the second element; and determining the quality of the environmental data based on the recall rate of the first element.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, system and device for determining the quality of environmental data. Background Art

[0002] Environmental data (such as vector diagrams corresponding to lane lines, stop lines, arrows, poles, traffic signs, and traffic lights) can usually be obtained based on positioning information. High-quality environmental data can be better applied to the navigation and control of autonomous driving, robots, etc.

[0003] Therefore, it is hoped that a method, system and device for judging the quality of environmental data can be provided to judge the quality of environmental data. Summary of the invention

[0004] One or more embodiments of the present specification provide a method for determining the quality of environmental data, the method comprising: acquiring a target image, and determining a first factor based on the target image and a semantic segmentation model; acquiring environmental data of a scene corresponding to the target image based on positioning information, projecting the environmental data onto the target image, and determining a second factor; determining a recall rate of the first factor based on the first factor and the second factor; and determining the quality of the environmental data based on the recall rate of the first factor.

[0005] One of the embodiments of the present specification provides a system for judging the quality of environmental data, the system comprising: an acquisition module, configured to acquire a target image and environmental data; a first determination module, configured to determine a first factor based on the target image and a semantic segmentation model; a projection module, configured to project the environmental data onto the target image to determine a second factor; a second determination module, configured to determine a recall rate of the first factor based on the first factor and the second factor; and a third determination module, configured to determine the quality of the environmental data based on the recall rate of the first factor.

[0006] One or more embodiments of the present specification provide a device for determining the quality of environmental data, the device comprising at least one storage medium and at least one processor; the at least one storage medium is used to store computer instructions; the at least one processor is used to execute the computer instructions to implement a method for determining the quality of environmental data. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0008] Figure 1is a schematic diagram of an application scenario of a system for determining the quality of environmental data according to some embodiments of this specification;

[0009] Figure 2 is an exemplary module diagram of a system for determining the quality of environmental data according to some embodiments of this specification;

[0010] Figure 3 is an exemplary flow chart of a method for determining the quality of environmental data according to some embodiments of this specification;

[0011] Figure 4 is an exemplary flow chart of determining a second recall rate corresponding to a non-line element according to some embodiments of this specification;

[0012] Figure 5 It is a schematic diagram of the structure of the device for judging the quality of environmental data according to some embodiments of this specification. DETAILED DESCRIPTION

[0013] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0014] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0015] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0016] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0017] Figure 1 It is a schematic diagram of an application scenario of a quality judgment system for environmental data shown in some embodiments of this specification. The system and method in this specification can be applied to the quality judgment of environmental data. The environmental data in this specification can be applied to control scenarios of mobile devices, such as autonomous driving scenarios, such as pure unmanned driving scenarios without drivers, assisted driving modules in vehicles, and other scenarios. The assisted driving module in the vehicle can autonomously control the vehicle's driving, and can also output driving prompts to the driver. Mobile devices can be applied to land, sea, aerospace, etc., or any combination thereof. Mobile devices may include taxis, private cars, ride-sharing, shared vehicles, buses, trains, motor vehicles, high-speed railways, subways, ships, airplanes, spacecraft, hot air balloons, bicycles, tricycles, motorcycles, robots, etc., or any combination thereof.

[0018] In some embodiments, Figure 1 As shown, the environmental data quality judgment system 100 may include a target scene 110, a collection device 111, a network 120, a terminal 130, a processor 140, and a storage device 150, etc.

[0019] The target scene 110 refers to a scene where the quality of the environmental data needs to be judged. For example, the target scene may be a certain intersection, etc. Exemplarily, the target scene 110 may include road signs, lane lines, traffic lights, etc.

[0020] The acquisition device 111 refers to a device for acquiring images and environmental data of a target scene. The acquisition device 111 may include an image acquisition device. The image acquisition device is used to acquire images of the target scene. The image acquisition device may include one or more devices with image acquisition functions (such as cameras, cameras, depth cameras, other vehicle-mounted image acquisition devices, etc.). The image acquisition device may be installed on a fixed and / or movable carrier (such as a vehicle, etc.). In some embodiments, the image acquisition device may include multiple. For example, multiple image acquisition devices may be installed above, on the left side, on the right side, etc. of the carrier. It can be understood that multiple image acquisition devices installed at different positions of the carrier can capture images of the target scene from multiple perspectives to ensure the integrity of the captured images.

[0021] In some embodiments, the acquisition device 111 may further include a positioning device (not shown in the figure). The positioning device is used to obtain environmental data of the target scene and may include one or more satellites. In some embodiments, the positioning device may be a global positioning system (GPS), a global navigation satellite system (GLONASS), a compass navigation system (COMPASS), a Beidou navigation satellite system, a Galileo positioning system, a quasi-zenith satellite system (QZSS), etc.

[0022] The network 120 may include any suitable network that facilitates information and / or data exchange of the environmental data quality judgment system. In some embodiments, one or more components of the environmental data quality judgment system (e.g., the acquisition device 111, the terminal 130, the processor 140, and the storage device 150, etc.) can communicate information and / or data with one or more other components in the system through the network 120. For example, the acquisition device 111 can send the acquired image of the target scene to the processor 140, etc. through the network 120.

[0023] The terminal 130 can provide functional components related to user interaction and can implement user interaction functions (such as providing or displaying information and data to users). The user can refer to a person who makes quality judgments on environmental data, etc. As just an example, the terminal 130 can be one of a mobile device, a tablet computer, a laptop computer, a desktop computer, etc., or other devices with input and / or output functions, or any combination thereof.

[0024] The processor 140 is capable of processing information and / or data related to the quality judgment system of environmental data to perform one or more functions described in this specification. In some embodiments, the processor 140 can be used to obtain a target image, determine a first element based on the target image and a semantic segmentation model; obtain environmental data of a scene corresponding to the target image based on positioning information, project the environmental data onto the target image, and determine a second element; determine the recall rate of the first element based on the first element and the second element; and determine the quality of the environmental data based on the recall rate of the first element. For detailed description of related content, please refer to Figure 3 and its related description.

[0025] In some embodiments, the processor 140 may include a central processing unit (CPU), a digital signal processor (DSP), a system on a chip (SoC), a microcontroller unit (MCU), a computer, a user console, etc., or any combination thereof. In some embodiments, the processor 140 may include a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 140 may be local or remote. In some embodiments, the processor 140 may be implemented on a cloud platform. By way of example only, a cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof. In some embodiments, the processor 140 may be integrated or included in one or more other components (e.g., terminal 130) of the quality judgment system 100 for environmental data.

[0026] The storage device 150 is capable of storing data, instructions and / or any other information. In some embodiments, the storage device 150 can store data acquired from the acquisition device 111 and / or the processor 140, such as target images, environmental data, etc. In some embodiments, the storage device 150 may include a large-capacity memory, a removable memory, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 150 may be executed on a cloud platform. In some embodiments, the storage device 150 may be connected to the network 120 to communicate with one or more other components of the quality judgment system for environmental data (e.g., the acquisition device 111, the processor 140, etc.). In some embodiments, the storage device 150 may be part of the processor 140.

[0027] It is worth noting that the quality judgment system 100 of environmental data is provided only for illustrative purposes and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various changes and modifications can be made according to the description of this specification. For example, the quality judgment system 100 of environmental data can also include a database, an information source, etc. For another example, the quality judgment system 100 of environmental data can be implemented on other devices to achieve similar or different functions. However, these changes and modifications will not deviate from the scope of this specification.

[0028] Figure 2 is an exemplary module diagram of a system for determining the quality of environmental data according to some embodiments of this specification. Figure 2 As shown, the quality judgment system of environmental data may include an acquisition module 210, a first determination module 220, a projection module 230, a second determination module 240, and a third determination module 250.

[0029] The acquisition module 210 is configured to acquire target images and environmental data.

[0030] The first determination module 220 is configured to determine a first factor based on the target image and the semantic segmentation model.

[0031] The projection module 230 is configured to project the environmental data onto the target image to determine the second factor.

[0032] The second determination module 240 is configured to determine the recall rate of the first factor based on the first factor and the second factor.

[0033] The third determination module 250 is configured to determine the quality of the environmental data based on the recall rate of the first factor.

[0034] For a detailed description of the functions involved in each module of the above-mentioned environmental data quality judgment system, please refer to the relevant part later in this specification, for example, Figure 3 , Figure 4 and related instructions.

[0035] It should be noted that the above description of the quality judgment of environmental data and its modules is only for the convenience of description and cannot limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from this principle. In some embodiments, Figure 2 The acquisition module 210, the first determination module 220, the projection module 230, the second determination module 240 and the third determination module 250 disclosed in the specification can be different modules in a system, or one module can realize the functions of two or more modules. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the protection scope of this specification.

[0036] Figure 3 300 is an exemplary flow chart of a method for determining the quality of environmental data according to some embodiments of this specification. In some embodiments, process 300 may be executed by a processor. Figure 3 As shown, process 300 includes the following steps.

[0037] Step 310 , obtaining a target image, and determining a first element based on the target image and a semantic segmentation model.

[0038] The target image refers to an image related to the target scene. For example, the target image can be an image of the target scene captured by the acquisition device. For more information about the target scene and acquisition device, see Figure 1 and its related parts. The target image can be acquired by an acquisition device, for example, by an image acquisition device in the acquisition device. The target image can include multiple elements, for example, lane lines, stop lines, arrows, traffic signs, and traffic lights.

[0039] In some embodiments, the target image may be a frame of image. In some embodiments, in order to improve the accuracy of the environmental data quality judgment, the target image may also include multiple frames.

[0040] The semantic segmentation model is used to segment the elements of the target image. The semantic segmentation model is a machine learning model, such as a convolutional neural network model, a deep neural network model, etc., or a combination thereof.

[0041] In some embodiments, the input of the semantic segmentation model includes one or more frames of target images, and the output includes the first element corresponding to each frame of the target image.

[0042] The first element refers to the element corresponding to each instance in the target image. For example, the element corresponding to the lane line, the element corresponding to the traffic sign, the element corresponding to the traffic light, etc. That is, the semantic segmentation model can segment each instance in the target image and use the segmented instance as the first element of the target image.

[0043] The semantic segmentation model can be obtained based on training samples and labels. In some embodiments, the training samples of the semantic segmentation model may include a large number of images, and the labels may include masks corresponding to instances of each image in the training samples. In some embodiments, the training samples and labels of the semantic segmentation model can be obtained based on historical data through manual collection and annotation.

[0044] In some embodiments, the processor may input a frame of target image into the semantic segmentation model, and the semantic segmentation model outputs the first element corresponding to the frame of target image. It is understandable that when the target image includes multiple frames, the processor may also input multiple frames of target image into the semantic segmentation model respectively, and the semantic segmentation model outputs the first element corresponding to each frame of target image respectively.

[0045] Step 320: Acquire environmental data of a scene corresponding to the target image based on the positioning information, and project the environmental data onto the target image to determine the second factor.

[0046] Positioning information refers to the positioning information of the target scene. The target scene refers to the scene when the target image is collected, that is, the scene where the quality of environmental data needs to be judged. Positioning information can be obtained based on carriers such as vehicles.

[0047] Environmental data refers to data such as vector diagrams corresponding to each instance in the target scene. Taking a target scene as an intersection as an example, environmental data may include vector diagrams corresponding to lane lines, vector diagrams corresponding to traffic signs, vector diagrams corresponding to traffic lights, etc.

[0048] The environmental data may be acquired based on the acquisition device 111. For example, the environmental data may be acquired based on a positioning device.

[0049] Furthermore, the processor may project the environmental data onto the target image to determine the second factor.

[0050] The second element refers to the element corresponding to the projection after the environmental data is projected onto the target image. Taking the target scene as an intersection as an example, the second element may include the projection obtained by projecting the vector map of lane lines, traffic signs, traffic lights, etc. onto the target image.

[0051] In some embodiments, the processor may project the environmental data onto the target image by various methods (such as spatial coordinate conversion, etc.), and determine the result obtained by the projection as the second factor. The image obtained after projecting the environmental data onto the target image may also be referred to as a projected image.

[0052] Step 330, based on the first factor and the second factor, determine the recall rate of the first factor.

[0053] The recall rate of the first element refers to the degree of match between the first element and the second element. For a certain frame of target image, the higher the degree of match between the first element and the second element, the higher the recall rate of the corresponding first element, indicating that the environmental data corresponding to the frame of target image is more complete and of higher quality. Among them, recall refers to an instance in the first element, and there is a corresponding projection in the second element. For example, if the first element includes a traffic sign, if the second element includes the projection corresponding to the traffic sign, then the traffic sign is recalled; if the second element does not include the projection corresponding to the traffic sign, then the traffic sign is not recalled.

[0054] In some embodiments, the first element includes a line element and a non-line element.

[0055] Line features refer to features corresponding to linear instances such as lane lines and stop lines. For example, the semantic segmentation model can detect lane lines, stop lines and other line features in the target image as 2D center polylines and output the polylines as line features.

[0056] Non-linear features refer to features corresponding to non-linear instances such as arrows, traffic signs, and traffic lights. For example, the semantic segmentation model can segment the pixels in the area where non-linear features such as traffic lights and traffic signs are located in the target image, and output the mask of the area corresponding to the non-linear feature as the non-linear feature.

[0057] In some embodiments, the second element includes a linear projection and a non-linear projection.

[0058] Line class projection refers to the projection corresponding to the vector diagram of line class instances in the environment data. For example, the line class projection may include the projection corresponding to the vector diagram of line class instances such as lane lines and stop lines in the environment data.

[0059] The non-line projection refers to the projection corresponding to the vector diagram of the non-line instance in the environmental data. For example, the non-line projection may include the projection corresponding to the vector diagram of the non-line instance such as arrows, traffic lights, and traffic signs in the environmental data.

[0060] In some embodiments, the recall rate of the first element includes a first recall rate corresponding to line elements and a second recall rate corresponding to non-line elements.

[0061] In some embodiments, the processor may determine a first recall rate corresponding to the line class element based on the line class element, the line class projection, and the error threshold.

[0062] The error threshold refers to the threshold for determining whether the matching between line features and line projections is successful, and can be preset in advance.

[0063] The first recall rate refers to the recall rate corresponding to line features.

[0064] In some embodiments, the processor may determine the first recall rate based on the line class projection and the line class element. Exemplarily, the processor may determine the ratio of the number of line class projections to the number of line class elements in the projected target image as the first element. For example, if a frame of target image includes 10 line class elements and its corresponding projection image includes 8 line class projections, the first recall rate of the frame of target image is 80%.

[0065] In some embodiments, there may be errors in the matching process of line elements and line projections. To reduce the errors and more accurately judge the quality of environmental data, the processor may determine the first recall rate corresponding to the line elements based on the line elements, line projections and error thresholds.

[0066] In some embodiments, the processor may determine the first recall rate based on the following steps S1-S3.

[0067] Step S1, setting a region of interest in a target image, and removing line elements in the target image whose distance from an image acquisition device is greater than a first preset threshold.

[0068] The distance between the line element and the image acquisition device can be determined based on the depth information of the target image. For example, the greater the depth information of a line element, the farther the element is from the image acquisition device.

[0069] In some embodiments, the processor may remove line elements whose depth information is greater than a first preset threshold, and retain the area corresponding to the remaining line elements as the area of ​​interest. The first preset threshold may be determined based on historical experience.

[0070] It is understandable that the greater the depth information of the line element, the farther it is from the image acquisition device, and the lower the accuracy of the semantic segmentation model in identifying it due to reasons such as the accuracy of the acquired image. Removing line elements with depth information greater than the first preset threshold can reduce errors and improve the judgment effect of the quality of environmental data.

[0071] Step S2, interpolating the line projection and determining a first error between the line projection and the line element.

[0072] In some embodiments, the processor may perform interpolation processing on the line projection based on a variety of interpolation algorithms (such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation algorithm, etc.), which is not limited here.

[0073] In some embodiments of the present specification, interpolation processing is performed on the line projection to supplement the line projection, facilitate better matching of the line projection and the line element, and improve the accuracy of the first error.

[0074] Furthermore, after the interpolation process, the processor may also determine a first error between the line projection and the line element.

[0075] The first error refers to the error when the line element is matched with the line projection. In some embodiments, the first error can be determined based on the straight-line distance between the interpolation point and the line element, the depth of the line element, and the offset distance of the line element during the interpolation process.

[0076] The straight-line distance between the interpolation point and the line feature may refer to the distance between the interpolation point and the corresponding point in the line feature in the coordinate system of the target image.

[0077] The depth of the line element refers to the depth information of the line element in the target image. In some embodiments, the depth of the line element may represent the distance between the line element and the image acquisition device.

[0078] The offset distance of a line element refers to the left-right offset distance between the line element and the image acquisition device in the target image, which can be determined based on the image acquisition device (such as a depth camera). The larger the offset distance of a line element, the larger the lateral displacement distance (left-right offset) between the line element and the image acquisition device.

[0079] The depth of the line element and the offset distance of the line element can be determined based on an image acquisition device (such as a depth camera).

[0080] Exemplarily, the first error may be determined based on the following formula (1): error1=dis(p,l)×αd z ×βd xy (1) Where error1 represents the first error; dis(p, l) represents the straight-line distance between the interpolation point and the line element; d z and d xy They represent the depth of the line element and the offset distance of the line element respectively; α and β are the preset weights, representing d z and d xy In some embodiments, the range of α and β is 0-1, and the first error can be limited to between 0 and 1, which is convenient for determining the confidence of the recall rate of each element later.

[0081] In some embodiments of the present specification, the first error is determined based on the straight-line distance between the interpolation point and the line element, the depth of the line element, and the offset distance of the line element. This can reduce the impact of distance on the accuracy of semantic segmentation of the image and improve the accuracy of the first error.

[0082] Step S3: determining a first recall rate based on the first error and the error threshold.

[0083] In some embodiments, the processor may determine the line element whose first error is less than an error threshold as an element that successfully matches the line projection, wherein the error threshold may be preset based on historical experience.

[0084] In some embodiments, the processor may determine the first recall rate based on the number of line elements that successfully match the line projection and the number of line elements output by the semantic segmentation model. For example, in a certain frame of target image, the number of line elements that successfully match the line projection is 3, and the number of line elements output by the semantic segmentation model is 5, then the first recall rate of the frame of target image is 60%.

[0085] In some embodiments of the present specification, the error in matching line elements and line projections is considered when determining the first recall rate, which can reduce the interference of line elements with a lower matching degree and improve the accuracy of the first recall rate.

[0086] In some embodiments, determining the recall rate of the first element further includes determining a second recall rate corresponding to the non-linear element based on the non-linear element and the non-linear projection. Figure 4 and its related description.

[0087] The processor may determine the first recall and the second recall as the recall of the first factor.

[0088] Step 340 , determining the quality of the environmental data based on the recall rate of the first factor.

[0089] In some embodiments, the processor may determine the quality of the environmental data based on the recall rate of the first element. For example, when the recall rate of the first element is higher, the processor may determine that the quality of the environmental data is higher.

[0090] In some embodiments, in order to reduce the impact of occasional errors (such as errors in the segmentation of the target image by the semantic segmentation model) and / or occlusion (such as occlusion of a certain element) that may exist in a single-frame target image, the processor may determine the average error of the first element corresponding to each instance based on multiple frames of target images; and determine the confidence of the first element corresponding to each instance based on the average error. Exemplarily, the confidence of the first element corresponding to each instance may be determined based on the following formula (2): Among them, confidence refers to the confidence of the first element corresponding to each instance; error n It represents the average error of the first element corresponding to each instance, including the average error of the first error corresponding to the line element and the second error corresponding to the non-line element in multiple frames of target images; p is a preset value, which can make the confidence distribution between 0 and 1; e is a natural constant.

[0091] In some embodiments, the processor may determine the quality of the environmental data based on the confidence of the first element and the recall rate of the first element. For example, the closer the confidence is to 1 and the greater the recall rate of the first element, the higher the quality of the environmental data. For example, the closer the confidence of a traffic sign is to 1 and the greater the recall rate is, the higher the quality of the environmental data corresponding to the traffic sign (such as the vector diagram corresponding to the traffic sign) is.

[0092] In some embodiments, when the second element is missing, the processor may supplement the environmental data based on the target image.

[0093] Among them, the omission of the second element means that when matching the first element and the second element, the second element corresponding to the first element cannot be successfully matched. For example, there are two first elements A and B, where the first element A has a second element A1 that can match it; and the first element B has no second element to match it, then it can be considered that the second element B1 corresponding to the first element B is missing, and the environmental data corresponding to the second element B1 needs to be supplemented.

[0094] In some embodiments, the processor may cluster the regions where the recall rate of the first element is lower than the recall rate threshold based on the average recall rate of the first element of the multiple frames of target images, and determine the number and range of the first elements in each cluster. When the number and range of the first element are both lower than the preset value, it can be determined that the second element of the region may be missing and the corresponding environmental data needs to be supplemented.

[0095] In some embodiments, the processor can collect instances corresponding to the first element corresponding to the missing second element through devices such as laser point cloud, and convert the collected results (such as into a vector diagram corresponding to the instance, etc.) and add them to the environmental data.

[0096] In some embodiments of the present specification, the quality of environmental data can be better judged based on the confidence level and the recall rate of the first factor to improve the accuracy; in addition, the quality of environmental data can be improved by supplementing the missing environmental data.

[0097] The method for judging the quality of environmental data described in some embodiments of this specification can effectively judge the quality of environmental data, reduce the cost of manual participation, and provide accurate and reliable environmental data for engineering applications (such as autonomous driving, navigation and control of robots, etc.); in addition, the method for judging the quality of environmental data described in the embodiments of this specification has no specific requirements for the type of environmental data, has a wide range of applications, and can also be applied to evaluate the automated production process of various elements, and has good versatility.

[0098] Figure 4 is an exemplary flow chart of determining the second recall rate corresponding to the non-line element according to some embodiments of this specification. In some embodiments, process 400 may be executed by a processor. Figure 4 As shown, process 400 includes the following steps.

[0099] Step 410: remove non-line elements in the target image whose distance from the image acquisition device is greater than a second preset threshold.

[0100] The distance between the non-linear element and the image acquisition device can be determined based on the depth information of the target image. For example, the greater the depth information of a non-linear element, the farther the element is from the image acquisition device.

[0101] In some embodiments, the processor may remove non-line elements whose depth information is greater than a second preset threshold, and retain the area corresponding to the remaining non-line elements as the area of ​​interest. The second preset threshold may be determined based on historical experience.

[0102] It is understandable that the larger the depth information of the non-linear element, the farther it is from the image acquisition device, and the lower the accuracy of the semantic segmentation model in identifying it due to the accuracy of the acquired image, etc. Removing the non-linear element whose depth information is greater than the second preset threshold can reduce errors and improve the judgment effect of the quality of environmental data.

[0103] Step 420, determining the mask area of ​​the non-linear element, the projection area of ​​the non-linear projection, and the intersection area of ​​the non-linear element and the non-linear projection; and determining the second error based on the intersection area and the projection area.

[0104] The mask area of ​​the non-linear feature refers to the mask area of ​​the non-linear feature segmented by the semantic segmentation model, and can be determined based on the first feature obtained by the semantic segmentation model.

[0105] The projection area of ​​the non-linear projection refers to the projection area after the non-linear data in the environment data is projected onto the target image, and can be determined based on the projection image. For more information about the projection image, see step 320.

[0106] The projection area of ​​the non-linear projection and the intersection area of ​​the non-linear element and the non-linear projection can be determined based on the projection image. For example, the intersection area can be the area of ​​the overlapped portion of the non-linear projection and the mask area of ​​the non-linear element in the projection image.

[0107] The second error refers to the error when the non-linear element is matched with the non-linear projection. In some embodiments, the second error can be determined based on the intersection area and the projection area. For example, the second error can be the ratio of the intersection area to the projection area, and the larger the ratio, the larger the second error.

[0108] It can be understood that, in order to improve the accuracy of the second error, the processor can collect multiple frames of target images, and determine the second errors corresponding to the respective second elements based on the multiple frames of target images.

[0109] In some embodiments, for non-ground elements such as traffic signs and traffic lights, the projection results may be blocked in the target images of some frames. To reduce the impact of the blockage, the processor may first determine whether there is an intersection between the projection results when determining the second error. For projection images with intersections, they need to be filtered out when determining the second error.

[0110] The second error can be used to determine the confidence of non-linear features. For more information about confidence, see Figure 3 .

[0111] Step 430 : determining a second recall rate based on the mask area, the intersection area, and the second error.

[0112] The second recall rate refers to the recall rate corresponding to non-line features.

[0113] In some embodiments, the processor may determine a second recall based on the mask area, the intersection area, and the second error.

[0114] For example, the processor may determine the second recall rate based on the mask area and the intersection area of ​​the target image in the multiple frames of the target image, in which the second error is lower than the preset value. Exemplarily, the second recall rate may be determined based on the following formula (3): recall=∑area intersect / ∑area mask (3) Among them, recall represents the second recall rate; area intersect Indicates the intersection area; area mask Represents the mask area.

[0115] In some embodiments of the present specification, the second recall rate determined based on the mask area, the intersection area and the second error can more accurately represent the recall of non-linear elements in the target image, thereby improving the accuracy of quality judgment of environmental data.

[0116] It should be noted that the above description of processes 300 and 400 is only for illustration and description, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to processes 300 and 400 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0117] Figure 5 1 is a schematic diagram of a structure of a quality judgment device for environmental data according to some embodiments of the present specification. The functions of the processor 140 or a portion thereof may be implemented by a computer via its hardware, software program, firmware, or a combination thereof. Although only one computer is shown for convenience, the computer functions related to the processor 140 as described herein may be implemented in a distributed manner on multiple similar platforms to distribute the processing load. In some embodiments, the computer may be a general purpose computer, or a computer with a specific purpose.

[0039] like Figure 5 As shown, the processor 140 may include a COM port 147, which may be connected to a network or connected from a network to facilitate data communication. The processor 140 may also include a central processing unit (CPU) 142 composed of one or more processors and used to execute program instructions. The computer platform may include an internal communication bus 141 and a data storage (e.g., a disk 145, a read-only memory (ROM) 143, a random access memory (RAM) 144). The data storage is used to store various data files processed and / or transmitted by the computer, programs in different forms, program instructions that may be executed by the CPU 142, etc. The processor 140 may also include an input / output (I / O) port 146, which is used for input / output flow between the processor 140 and other components (such as a storage device 150) in the quality judgment system of environmental data.

[0118] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0119] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0120] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some embodiments currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0121] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0122] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0123] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0124] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for judging the quality of environmental data, characterized in that: The method comprises: Acquire a target image, and determine a first element based on the target image and a semantic segmentation model; Acquire environmental data of a scene corresponding to the target image based on the positioning information, and project the environmental data onto the target image to determine a second factor; Determine a recall rate of the first factor based on the first factor and the second factor; The quality of the environmental data is determined based on the recall rate of the first factor.

2. The method according to claim 1, characterized in that The semantic segmentation model is a machine learning model, the input of the semantic segmentation model is the target image, and the output of the semantic segmentation model is the first element.

3. The method according to claim 1, characterized in that The first elements include line elements and non-line elements, and the second elements include line projections and non-line projections; the recall rate of the first elements includes a first recall rate corresponding to the line elements and a second recall rate corresponding to the non-line elements; The determining the recall rate of the first factor based on the first factor and the second factor includes: The first recall rate is determined based on the line class elements, the line class projection, and an error threshold.

4. The method according to claim 3, characterized in that The determining the first recall rate based on the line element, the line projection and the error threshold comprises: Performing interpolation processing on the line class projection and determining a first error between the line class projection and the line class element; The first recall rate is determined based on the first error and the error threshold.

5. The method according to claim 4, characterized in that The first error is determined based on a straight-line distance between an interpolation point processed by the interpolation and the line element, a depth of the line element, and an offset distance of the line element.

6. The method according to claim 3, characterized in that The determining the first recall rate based on the line element, the line projection and the error threshold comprises: The line elements in the target image whose distance from the image acquisition device is greater than a first preset threshold are removed.

7. The method according to claim 1, characterized in that The determining the recall rate of the first factor based on the first factor and the second factor includes: Determine the second recall rate based on non-line features and non-line projections, including: Removing the non-linear elements in the target image whose distance from the image acquisition device is greater than a second preset threshold; Determine a mask area of ​​the non-linear element, a projection area of ​​the non-linear projection, and an intersection area of ​​the non-linear element and the non-linear projection; and determine a second error based on the intersection area and the projection area; The second recall rate is determined based on the mask area, the intersection area, and the second error.

8. The method according to claim 1, characterized in that The method comprises: When the second element is missing, the environmental data is supplemented based on the target image.

9. A quality judgment system for environmental data, characterized in that: The system comprises: An acquisition module, configured to acquire target images and environmental data; A first determination module is configured to determine a first factor based on the target image and a semantic segmentation model; A projection module, configured to project the environmental data onto the target image to determine a second factor; A second determining module is configured to determine a recall rate of the first element based on the first element and the second element; The third determination module is configured to determine the quality of the environmental data based on the recall rate of the first factor.

10. A device for determining the quality of environmental data, characterized in that: The device includes at least one storage medium and at least one processor; The at least one storage medium is used to store computer instructions; The at least one processor is used to execute the computer instructions to implement the method for determining the quality of environmental data according to any one of claims 1 to 8.

Images