A Method for Analyzing a Validity of at Least One Preliminary Datum Label by an Electronic Computing Device, a Computer Program Product, a Non-Transitory Computer-Readable Storage Medium, and an Electronic Computing Device

Abstract

Method for analysing validity of image object 58-62 labels 64-68, comprising: using a first computing unit to provide preliminary object labels (datum) 64-68 for objects in an image 44; using a second

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the fields of data science and artificial intelligence, in particular with respect to the field of automobiles. More specifically, the present invention relates to a method for analyzing a validity of at least one preliminary datum label by an electronic computing device. Furthermore, the present invention relates to a corresponding computer program product, a corresponding non-transitory computer-readable storage medium, and a corresponding electronic computing device. BACKGROUND INFORMATION

[0002] It is known in the prior art that a software pipeline may be designed to automatically generate at least one data label as output from and corresponding to at least one input data point. The software pipeline may comprise at least one first artificial intelligence system and / or at least one software algorithm. The at least one software algorithm may be prepared manually and / or with assistance from at least one second artificial intelligence system. The data label output may then be used in training a third artificial intelligence system.

[0003] For example, a convolutional neural network trained for image classification may be used to generate a series of image classifications from a series of input images, whereupon the series of input images may be used in conjunction with the series of image classifications to train a different convolutional neural network for image classification.

[0004] It is furthermore known in the prior art that the at least one automatically generated data label may be at least partially checked by a human examiner. For example, incorrect data labels may be manually adjusted by the human examiner. Furthermore, the automatically generated data labels may be supplemented or replaced by at least one data label prepared manually by a human.

[0005] However, the software pipeline may output at least one incorrect data label, for example due to at least one limitation of the at least one first artificial intelligence and / or of the at least one software algorithm. Furthermore, checks performed by human examiners may fail to identify at least one erroneous data label.

[0006] These two situations may be particularly prevalent among input data points corresponding to edge cases. For instance, if the input data points were generated by sensors installed aboard a motor vehicle, at least some of the edge cases may correspond to unusual driving situations, and there may be a greater risk of incorrect data label output from the software pipeline for those input data points generated during unusual driving situations. As an example of an unusual driving situation, the motor vehicle may be driving along a tortuous and steeply inclined mountain road during a foggy night.

[0007] If at least one of the two situations transpires, the third artificial intelligence system may be trained, validated, and / or tested over a dataset containing at least one incorrect data label. The presence of the at least one incorrect data label within the dataset may diminish performance of the third artificial intelligence system. SUMMARY OF THE INVENTION

[0008] It is an object of the present invention to provide automated validity analysis for labels of visual data.

[0009] The object is solved by a method, a corresponding computer program product, a corresponding non-transitory computer-readable storage medium, and a corresponding electronic computing device according to the independent claims. Advantageous embodiments are presented in the dependent claims, the following description, and the figures.

[0010] One aspect of the invention relates to a method for analyzing a validity of at least one preliminary datum label by an electronic computing device. The at least one preliminary datum label and at least one corresponding visual datum are provided by a first computing unit of the electronic computing device, wherein at least one content feature of the at least one visual datum is potentially described by the at least one preliminary datum label. At least one predicted datum label is generated by a second computing unit of the electronic computing device from the at least one visual datum, wherein the at least one predicted datum label represents a predicted description of the at least one content feature. The at least one predicted datum label is compared to the at least one preliminary datum label by the electronic computing device, wherein depending on a decision criterion, the at least one predicted datum label is classified as discrepant by the electronic computing device. The validity of the at least one preliminary datum label is analyzed by the electronic computing device, depending on the comparison.

[0011] As a consequence of application of the above method, the validity of the at least one preliminary datum label is analyzed by the electronic computing device. Therefore, the object of the invention as stated earlier is fulfilled. In particular, the electronic computing device may determine at least one discrepancy between the at least one preliminary datum label and the at least one predicted datum label, which may suggest at least one error in the preliminary datum label.

[0012] The first computing unit and the second computing unit may be software components or hardware components or units combining at least one software component and at least one hardware component. One example of a hardware computing unit is a neural processing unit (NPU), also known in the prior art as an artificial intelligence accelerator (an Al accelerator) and a deep learning processor.

[0013] Another example of a hardware computing unit may be a second electronic computing device acting subordinately to a first electronic computing device. For instance, at a supercomputing data center, a single electronic computing device may issue commands to dozens of subordinate electronic computing devices. Such issuance of commands may proceed remotely, for example, in a cloud computing setting in which a client-side electronic computing device transmits computer instructions over the Internet to at least one server-side electronic computing device. In this example, the at least one server-side electronic computing device may transmit at least one processed data product back to the client-side electronic computing device.

[0014] One example of a software computing unit is a computer program that loads a convolutional neural network saved to the electronic computing device, for example on a non-transitory computer-readable storage medium of the electronic computing device, into a main memory of the electronic computing device and generates a series of output data from a series of input data via the convolutional neural network.

[0015] If the first computing unit and the second computing unit are both software computing units in the form of pre-trained neural networks, they may differ in at least one of: architecture (such as number of hidden layers, layer width, or activation functions applied), training data, and at least one training hyperparameter value. Differences in training data and / or at least one training hyperparameter value may not be explicitly recorded in the pre-trained neural network itself. For example, differences in these regards may manifest as differences in at least one weight value and / or at least one bias value characterizing at least one neuron of the neural network.

[0016] One example of a computing unit combining a software component and a hardware component is a computer program, accessed on the electronic computing device, which may be used to send control signals to a convolutional neural network stored on a remote server.

[0017] The first computing unit and the second computing unit may be separate instances of a single computer program code or separate hardware products manufactured according to a single specification. The first computing unit and the second computing unit may even be identical, referring to a single computing unit.

[0018] The at least one corresponding visual datum may be an output of an optical sensor, such as a photograph or a video from a dashboard camera. However, the at least one corresponding visual datum may be a datum used to determine visual information even if the at least one corresponding visual datum does not derive from an optical sensor. For example, the at least one corresponding visual datum may be an output of a radar device, a light detection and ranging (LIDAR) device, an inertial measurement unit (IMU) sensor, a global positioning system (GPS) sensor, or an ultrasonic sensor (USS) device. The at least one corresponding visual datum may even be a property of a sensor, such as an intrinsic parameter of an optical camera.

[0019] At least one motor vehicle with an at least partly automated driving functionality may be configured to generate at least one of the at least one visual datum. In particular, in order to generate large enough quantities of training data for suitable application of the method, a manufacturer may task a fleet of, for instance, several dozens of motor vehicles with the job of generating the data.

[0020] For example, sensors such as those previously mentioned may be installed on the at least one motor vehicle to generate visual data during various driving maneuvers of the at least one motor vehicle, such as changing lanes and parking. The generated data may then be transmitted to a manufacturer-operated data center from each of the at least one motor vehicle via a networking device such as a Wi-Fi card. Preliminary data labels corresponding to these generated data may be generated separately, at a later time.

[0021] However, the set of visual data referred to by the method may not correspond to a total set of visual data so collected. For example, an automated software may be applied to the total set of visual data in order to identify visual data of particular relevance to the method. Such an automated software may, for instance, prioritize visual data dissimilar to previously used visual data. Alternatively or in addition, such an automated software may, for instance, prioritize visual data which, when used as inputs to a pretrained neural network, result in low-confidence outputs (such as outputs whose confidence levels fall below a predefined confidence threshold). The electronic computing device may make use of only those visual data identified as of particular relevance to the method.

[0022] The at least one content feature of the at least one corresponding visual datum may be an individual subject of the at least one corresponding visual datum. For example, if a particular visual datum is a photograph taken by a dashboard camera of a motor vehicle, content features may include any bicyclists, pedestrians, signs, traffic cones, and other motor vehicles visible within the photograph.

[0023] The at least one content feature may be an environmental property associated with the at least one corresponding visual datum, rather than a single discrete entity. For example, the at least one content feature may be an air quality (such as a level of smog) or precipitation type (such as rainfall or snowfall) apparent in the at least one corresponding visual datum.

[0024] The at least one preliminary datum label and the at least one predicted datum label are potential descriptions of the at least one content feature. For example, if a particular visual datum is a photograph of a bicyclist on a rainy day, one corresponding datum label may be an indicator that the photograph contains a bicyclist. Another corresponding datum label may be an indicator that the photograph depicts a rainy scene.

[0025] Furthermore, the datum labels may be associated with particular regions of the at least one corresponding visual datum. In the example of the photograph of the bicyclist on the rainy day, the indicator that the photograph contains a bicyclist may include a designation within the photograph of a particular region to which the bicyclist corresponds. For instance, the designation may comprise a group of pixel IDs.

[0026] Each datum may have more than one label. For example, one particular visual datum may be a photograph containing both a fire hydrant and a crosswalk, in which case the particular visual datum may correspond to at least two labels. Similarly, a single label may apply to multiple data. For example, a fire hydrant may appear in more than one photograph analyzed by the method.

[0027] Another example of a datum label is a path label or predicted path attributed to an observer by which a visual datum is collected or to at least one content feature of the visual datum. For example, datum label may be a path prediction for a moving pedestrian captured by a photograph. Another example of a datum label is a semantic segmentation mask.

[0028] The decision criterion is used by the electronic computing device to determine whether a difference between the at least one predicted datum label and the at least one preliminary datum label qualifies as a discrepancy. Where the data labels designate particular regions of the at least one visual datum, the decision criterion may be, for example, a predefined pixel overlap threshold.

[0029] In the example of the photograph of the bicyclist on the rainy day, a corresponding preliminary datum label and a corresponding predicted datum label may agree that a bicyclist is depicted in the photograph. However, the preliminary datum label and the predicted datum label may designate different regions of the photograph as corresponding to the bicyclist.

[0030] In this example, the decision criterion may be that regions designated by analogous data labels share at least 95% of their pixel IDs. Consequently, if the two regions designated as corresponding to the bicyclist share at least 95% of their pixel IDs, the electronic computing device may classify the differing regions as non-discrepant.

[0031] However, the decision criterion may be trivial. For example, if the data labels identify content features without additional information (such as corresponding regions of the at least one visual datum), differences between the at least one predicted datum label and the at least one preliminary datum label may be automatically classified as discrepancies. In the example of the photograph of the bicyclist on the rainy day, if a corresponding predicted datum label indicates that a bicyclist is depicted in the photograph while a corresponding preliminary datum label indicates that a traffic cone is depicted in the photograph, the decision criterion may be used by the electronic computing device to automatically classify these datum labels as discrepant.

[0032] Analysis of the validity of the at least one preliminary datum label may result in at least one validity indicator. For example, the indicator may be an estimated likelihood of validity of the at least one preliminary datum label, which may depend on properties of any discrepancies identified. For example, if the datum labels are discrepant in designating different regions, the estimated likelihood of validity may depend on an extent to which the different regions do not overlap.

[0033] The invention comprises further developments providing additional benefits.

[0034] In a further development of the invention, the method comprises the additional step of generating, by the electronic computing device, a control signal to initiate training of at least one neural network by the electronic computing device with at least one of the at least one visual datum after analyzing the validity of the at least one preliminary datum label. For example, the training may be performed by a third computing unit of the electronic computing device.

[0035] For example, the at least one neural network may be downloaded to and / or installed in a motor vehicle with an at least partly automated driving functionality. The at least one neural network may be used to perform one or more of various computational tasks associated with operation of the at least partly automated driving functionality. Examples of computational tasks associated with operation of the at least partly automated driving functionality include object detection, navigation planning, and map generation (for instance, generation of a map of a vicinity of the motor vehicle).

[0036] In a further development, the method comprises two additional steps. In a first additional step, at least one file processing command is received by the electronic computing device from a user of the electronic computing device. The at least one file processing command concerns the at least one preliminary datum label and / or the at least one corresponding visual datum. In a second additional step, the file processing command is executed by the electronic computing device.

[0037] As a consequence of application of this development, a user of the electronic computing device may review at least one output of the method, such as the validity analysis and / or at least one identified discrepancy, and act on the at least one preliminary datum label and / or the at least one visual datum accordingly. For example, the file processing command may involve replacement of the at least one preliminary datum label with the at least one predicted datum label.

[0038] In a further development, the method comprises the additional step of generating at least one discrepancy report from the comparison by a visual language model of the electronic computing device. The at least one discrepancy report comprises a human-readable description of at least one discrepancy between the at least one preliminary datum label and the at least one predicted datum label. The at least one discrepancy report may contain additional text describing a context of the at least one discrepancy. For example, the at least one discrepancy report may describe at least one region of a photograph adjacent to an area related to the discrepancy. Furthermore, the at least one discrepancy report may contain other extracted semantic information, to which further heuristics of the electronic computing device may be applied.

[0039] The visual language model is a model that accepts the at least one visual datum as input and generates text as output. The discrepancy report may fulfill the steps of comparing the at least one predicted datum label to the at least one preliminary datum label and / or analyzing the validity of the at least one preliminary datum label.

[0040] In a further development, the method comprises the additional step of structuring the at least one discrepancy report by a large language model of the electronic computing device. For example, the at least one discrepancy report may be structured for more convenient access of at least one essential result of the discrepancy report.

[0041] In a further development, the method comprises the additional step of generating the at least one preliminary datum label with a neural network of the electronic computing device.

[0042] In a further development, the method comprises the additional step of visually delineating the at least one content feature, to which the at least one preliminary datum label and / or the at least one predicted datum label corresponds, within the at least one visual datum by the electronic computing device. The at least one visually delineated content feature pertains to a difference between the at least one preliminary datum label and the at least one predicted datum label.

[0043] In particular, the method is a computer-implemented method. Therefore, one aspect of the invention relates to a computer program product consisting of computer-readable instructions for performing a method according to the preceding aspect.

[0044] Another aspect of the invention relates to a non-transitory computer-readable storage medium comprising the computer program product according to the preceding aspect. Examples of computer-readable storage media include solid-state drives, SSDs; hard disk drives, HDDs; flash memory devices such as USB flash drives; and optical discs.

[0045] Another aspect of the invention relates to an electronic computing device, wherein the electronic computing device is configured for performing a method according to the preceding aspect. In particular, the method is performed by the electronic computing device.

[0046] An “electronic computing device” may in particular be understood as a data processing device, which comprises processing circuitry. The electronic computing device may therefore in particular process data to perform computing operations. This may also include operations to perform indexed accesses to a data structure, for example a look-up table, LUT.

[0047] In particular, the electronic computing device may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits, ASIC, one or more field-programmable gate arrays, FPGA, and / or one or more systems on a chip, SoC. The electronic computing device may also include one or more processors, for example one or more microprocessors, one or more central processing units, CPU, one or more graphics processing units, GPU, and / or one or more signal processors, in particular one or more digital signal processors, DSP or neural processing units, NPU. The electronic computing device may also include a physical or a virtual cluster of computers or other of said units.

[0048] In various embodiments, the electronic computing device includes one or more hardware and / or software interfaces and / or one or more memory units.

[0049] A memory unit may be implemented as a volatile data memory, for example a dynamic random access memory, DRAM, or a static random access memory, SRAM, or as a non-volatile data memory, for example a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, a flash memory or flash EEPROM, a ferroelectric random access memory, FRAM, a magnetoresistive random access memory, MRAM, or a phase-change random access memory, PCRAM.

[0050] Further advantages, features, and details of the invention derive from the following description of preferred embodiments as well as from the drawings. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figures and / or shown in the figures alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The novel features and characteristics of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0052] The drawings show in:

[0053] Fig. 1 an embodiment of an electronic computing device according to an aspect of the invention.

[0054] Fig. 2 a side-by-side view comparing a visual datum annotated with at least one preliminary datum label to the same visual datum annotated with at least one predicted datum label.

[0055] Fig. 3 a flowchart according to an embodiment of the method.

[0056] Fig. 4 a flowchart according to an embodiment of part of the method.

[0057] In the figures, the same elements or elements having the same function are indicated by the same reference signs. DETAILED DESCRIPTION

[0058] In the following detailed description of the example embodiments of the disclosure, references are made to the accompanying drawings that form parts hereof and in which are shown by way of illustration specific embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0059] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0060] In Fig. 1, an electronic computing device 10 is shown connected to a monitor 12. In this example, the electronic computing device 10 may also be connected to a computer keyboard 14 and a computer mouse 16. Furthermore, the electronic computing device 10 may comprise a non-transitory computer-readable storage medium 18.

[0061] For simplicity, several computation-related features of the embodiment, such as specific programs and data, are represented symbolically as blocks on the electronic computing device 10. For example, the non-transitory computer-readable storage medium 18 is represented as a block encapsulating several other computation-related features, and is represented on a front surface of the electronic computing device 10. Despite this manner of representation, the non-transitory computer-readable storage medium 18 may, in this example, be a hard disk drive (HDD) internal to the electronic computing device 10.

[0062] In this example, the electronic computing device 10 may be running a validity analysis program 20 stored in the non-transitory computer-readable storage medium 18, whose user interface 22 is shown on the monitor 12. The user interface 22 may comprise various interactive elements.

[0063] A visual data file location text field 24 may be used to specify an address within the non-transitory computer-readable storage medium 18 from which at least one visual datum 26 may be retrieved by the electronic computing device 10.

[0064] Similarly, a preliminary datum label source text field 28 may be used to specify a first computing unit 32 of the electronic computing device 10 from which the electronic computing device 10 may retrieve at least one preliminary datum label 30 concerning the at least one visual datum 26. At least one content feature of the at least one visual datum 26 is potentially described by the at least one preliminary datum label 30.

[0065] In this example, the preliminary datum label source text field 28 may be filled in with a file location associated with a particular neural network that generates the at least one preliminary datum label 30.

[0066] Similarly, a predicted datum label source text field 34 may be used to specify a particular second computing unit 36 of the electronic computing device 10. The second computing unit 36 may be invoked by the electronic computing device 10 to generate at least one predicted datum label 38 from the at least one visual datum 26. The at least one predicted datum label 38 represents a predicted description of the at least one content feature.

[0067] For example, a file location associated with the second computing unit 36 may be entered into the predicted datum label source text field 34. An analyze-button 40 may be used to begin generating the at least one predicted datum label 38 and comparing the at least one preliminary datum label 30 to the at least one predicted datum label 38.

[0068] A left display pane 42 may be used by the validity analysis program 20 to display a currently viewed visual datum 44 (which is one of the at least one visual datum 26) with annotations according to at least one currently viewed preliminary datum label 46 (which is at least one of the at least one preliminary datum label 30) to which the currently viewed visual datum 44 corresponds. A right display pane 48 may be used by the validity analysis program 22 display the currently viewed visual datum 44 with annotations according to at least one currently viewed predicted datum label 50 (which is at least one of the at least one predicted datum label 34) to which the currently viewed visual datum 44 corresponds. The display panes 42, 44 are the subject of Fig. 2.

[0069] The user interface 22 may feature several buttons with which different file processing commands may be transmitted to the electronic computing device.

[0070] An approve-button 52 may be used to save the currently viewed visual datum 44 with the at least one currently viewed preliminary datum label 46 to an output directory of the validity analysis program 20, deleting the at least one currently viewed predicted datum label 50. Similarly, a replace-button 54 may be used to save the currently viewed visual datum 44 with the at least one currently viewed predicted datum label 50, deleting the at least one currently viewed preliminary datum label 46.

[0071] A delete-button 56 may be used to delete the currently viewed visual datum 44, the at least one currently viewed preliminary datum label 46, and the currently viewed predicted datum label 50. The delete-button 56 may be advantageous in situations where the currently viewed visual datum 44 is determined to be too ambiguous or of too low quality to be stored in the output directory of the validity analysis program 20. For example, if a user of the validity analysis program 20 determines that both the at least one currently viewed preliminary datum label 46 and the at least one currently viewed predicted datum label 50 are incorrect, the currently viewed visual datum 44 may be suspected to be of low quality.

[0072] Fig. 2 is a companion to Fig. 1 and shows an example of what may be visualized using the display panes 42, 48 of Fig. 1. The same currently viewed visual datum 44 may occupy both display panes 42, 48. In this example, the currently viewed visual datum 44 is a photograph of, from left to right, a traffic cone 58, a yield sign 60, and a pedestrian 62, all of which represent content features 58, 60, 62 of the currently viewed visual datum 44 and, by extension, of the at least one visual datum 26. However, the display panes 42, 48 may display multiple visual data simultaneously, for example if a particular ultrasonic sensor reading accompanies a particular photograph.

[0073] The at least one currently viewed preliminary datum label 46 and the at least one currently viewed predicted datum label 50 may be object classification labels attached to individual bounding boxes, wherein each individual bounding box potentially corresponds to a particular classified object.

[0074] In the left display pane 42, the currently viewed visual datum 44 is annotated according to the at least one currently viewed preliminary datum label 46. A first currently viewed preliminary datum label 64 bounds the traffic cone 58 and identifies the traffic cone 58 as a traffic cone. A second currently viewed preliminary datum label 66 bounds the yield sign 60 but identifies the yield sign 60 as a pedestrian. The third currently viewed preliminary datum label 68 bounds the pedestrian 62 and identifies the pedestrian 62 as a pedestrian.

[0075] In the right display pane 48, the currently viewed visual datum 44 is annotated according to the at least one currently viewed predicted datum label 50. A first currently viewed predicted datum label 70 bounds the traffic cone 58 and identifies the traffic cone 58 as a traffic cone. A second currently viewed predicted datum label 72 bounds the yield sign 60 and identifies the yield sign 60 as a yield sign. A third currently viewed predicted datum label 74 bounds a region only partially overlapping with the pedestrian 62 but identifies the region as a pedestrian.

[0076] In this example, a user operating the validity analysis program 20 may click the approve-button 52 when selecting the first currently viewed preliminary datum label 64 and the third currently viewed preliminary datum label 68. By contrast, the user may click the replace-button 54 when selecting the second currently viewed preliminary datum label 66.

[0077] Fig. 3 is a flowchart depicting a general form of the method.

[0078] In a first step G1 of the method, the at least one preliminary datum label 30 and the at least one corresponding visual datum 26 are provided by the first computing unit 32. In a second step G2 of the method, the at least one predicted datum label 38 is generated by the second computing unit 36 from the at least one visual datum 26.

[0079] The second step G2 may comprise various component steps. One component step of the second step G2 may involve feeding various input data into the second computing unit 36. For example, the second computing unit 36 may be tasked with generation of a map of a vicinity of a motor vehicle from multiple data sources, in which case this component step may involve feeding data from at least one radar sensor and at least one LIDAR sensor into the second computing unit 36. In this case, a second component step of the second step G2 may comprise the generation of the map.

[0080] In a third step G3 of the method, the at least one predicted datum label 38 is compared to the at least one preliminary datum label 30 by the electronic computing device 10, wherein depending on a decision criterion, the at least one preliminary datum label 30 is classified as discrepant by the electronic computing device 10.

[0081] In a fourth step G4 of the method, the validity of the at least one preliminary datum label 30 is analyzed by the electronic computing device 10, depending on the comparison G3. For example, the electronic computing device 10 may filter out each of the at least one preliminary datum label 30 depending on whether a discrepancy was found during comparison G3. That is, each preliminary datum label 30 for which no discrepancy was found may be analyzed to be valid, whereas each preliminary datum label 34 which at least one discrepancy was found may be analyzed to be invalid.

[0082] Fig. 4 is a flowchart depicting a section of an exemplary embodiment of the method. Specifically, Fig. 4 shows steps that may be taken by the electronic computing device 10 following determination of a discrepancy in step G3. In this example, the first computing unit 32 and the second computing unit 36 may have made use of several visual data 26 in order to generate each of several scenes, such as obstacle maps combining photographic data, light detection and ranging (LIDAR) data, and ultrasonic sensor (USS) data.

[0083] In a first step E1 of the section, the several visual data 26 may be fed into a discrepancy visualizer of the electronic computing device 10. That is, the at least one content feature 58, 60, 62 to which the at least one preliminary datum label 30 and / or the at least one predicted datum label 38 corresponds may be visually delineated within the at least one visual datum 26 by the electronic computing device 10, wherein the at least one content feature 58, 60, 62 pertains to the discrepancy between the at least one preliminary datum label 30 and the at least one predicted datum label 38.

[0084] In a second step E2 of the section, at least one discrepancy report may be generated from the comparison G3 by a visual language model of the electronic computing device 10, wherein the at least one discrepancy report comprises a human-readable description of at least one discrepancy between the at least one preliminary datum label 30 and the at least one predicted datum label 38. The visual language model may be supplemented with a large language model, for example for proofreading the at least one discrepancy report.

[0085] In a third step E3 of the section, the at least one discrepancy report may be structured by a large language model of the electronic computing device 10. The large language model used for structuring the at least one discrepancy report may be different from a large language model used for proofreading the at least one discrepancy report.

[0086] In a fourth step E4 of the section, a validity of the at least one preliminary datum label 30 is analyzed by the electronic computing device 10. The at least one visual datum 26, the at least one preliminary datum label 30, and the at least one predicted datum label 38 may consequently be filtered out of the method by the electronic computing device 10, depending on the at least one discrepancy report. For example, at least one of the at least one preliminary datum label 30 may be added to a blacklist of the electronic computing device or a whitelist of the electronic computing device.

[0087] In a fifth step E5 of the section, a control signal is generated by the electronic computing device 10 to initiate training of at least one neural network by the electronic computing device 10 with the at least one visual datum 26.

[0088] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0089] The terms “comprises,” “comprising,” or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method. reference signs Electronic computing device Monitor Computer keyboard Computer mouse Non-transitory computer-readable storage medium Validity analysis program User interface Visual data file location text field Visual datum Preliminary datum label source text field Preliminary datum label First computing unit Predicted datum label source text field Second computing unit Predicted datum label Analyze-button Left display pane Currently viewed visual datum Currently viewed preliminary datum label Right display pane Currently viewed predicted datum label Approve-button Replace-button Delete-button Traffic cone Yield sign Pedestrian First currently viewed preliminary datum label Second currently viewed preliminary datum label Third currently viewed preliminary datum label First currently viewed predicted datum label Second currently viewed predicted datum label 19 74 Third currently viewed predicted datum label G1-G4 Steps of the general method E1-E5 Steps of an exemplary embodiment of the method

Claims

1. A method for analyzing a validity of at least one preliminary datum label (30) by an electronic computing device (10), comprising the steps of:- providing (G1) the at least one preliminary datum label (30) and at least one corresponding visual datum (26) by a first computing unit (32) of the electronic computing device (10), wherein at least one content feature (58, 60, 62) of the at least one visual datum (26) is potentially described by the at least one preliminary datum label (30);- generating (G2) at least one predicted datum label (38) by a second computing unit (36) of the electronic computing device (10) from the at least one visual datum (26), wherein the at least one predicted datum label (38) represents a predicted description of the at least one content feature (58, 60, 62);- comparing (G3) the at least one predicted datum label (38) to the at least one preliminary datum label (30) by the electronic computing device (10), wherein depending on a decision criterion of the electronic computing device (10), the at least one predicted datum label (38) is classified as discrepant by the electronic computing device (10); and- analyzing (G4) the validity of the at least one preliminary datum label (30) by the electronic computing device (10), depending on the comparison (G3).

2. The method according to claim 1, comprising the additional step of generating (E5), by the electronic computing device (10), a control signal to initiate training of at least one neural network by the electronic computing device (10) with at least one of the at least one visual datum (26) after analyzing the validity of the at least one preliminary datum label (30).

3. The method according to claims 1 or 2, comprising the additional steps of - receiving at least one file processing command, concerning the at least one preliminary datum label (30) and / or the at least one corresponding visual datum (26), by the electronic computing device (10) from a user of the electronic computing device (10); and- executing the file processing command by the electronic computing device (10).

4. The method according to any one of claims 1 to 3, comprising the additional step of generating (E2) at least one discrepancy report from the comparison by a visual language model of the electronic computing device (10), wherein the at least one discrepancy report comprises a human-readable description of at least one discrepancy between the at least one preliminary datum label (30) and the at least one predicted datum label (38).

5. The method according to claim 4, comprising the additional step of structuring (E3) the at least one discrepancy report by a large language model of the electronic computing device (10).

6. The method according to any one of claims 1 to 5, comprising the additional step ofgenerating the at least one preliminary datum label (30) with a neural network of the electronic computing device (10).

7. The method according to any one of claims 1 to 6, comprising the additional step ofvisually delineating (E1) the at least one content feature (58, 60, 62), to which the at least one preliminary datum label (30) and / or the at least one predicted datum label (38) corresponds, within the at least one visual datum (26) by the electronic computing device (10), wherein the at least one content feature (58, 60, 62) pertains to a difference between the at least one preliminary datum label (30) and the at least one predicted datum label (38).

8. A computer program product containing computer-readable instructions for performing a method according to any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium (18) comprising the computer program product according to claim 8.

10. An electronic computing device (10) configured to perform a method according to any one of claims 1 to 7.

Images