Method and system for data generation

Abstract

The present invention provides a system for data generation, comprising a capturing unit for obtaining a point cloud dataset of an object, and a processing unit for receiving the point cloud dataset of the object. The processing unit is configured to pre-process the point cloud dataset of the object, generate one or more cumulative datasets from the pre-processed point cloud dataset of the object, and operate a plurality of modules for generating at least one feature map dataset from the pre-processed point cloud dataset of the object, generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, generating one or more feature page datasets from corresponding feature page portion datasets, and constructing the cumulative dataset from any one or a combination of the pre-processed point cloud dataset, the feature map dataset, and the feature page datasets.

Description

[0001]

[0002] METHOD AND SYSTEM FOR DATA GENERATION

[0003] FIELD OF INVENTION

[0004] The present invention generally relates to generation of data from existing datasets. More specifically, a method and system for data generation, in which the generated data is to be involved within object inspection workflows based on machine learning.

[0005] BACKGROUND OF THE INVENTION

[0006] Wire bonding is a process of creating electrical interconnections between semiconductors (or other integrated circuits) and silicon chips using bonding wires. During the wire bonding process, it is crucial that the wires are properly bonded to ensure a reliable connection between a chip and a base. For this, various wire bond inspection techniques have been developed.

[0007] However, conventional wire bond inspection techniques, which may be performed via two-dimensional vision inspection from still images, struggle to identify and classify wire bonds in an accurate manner. In view of the emerging semiconductor devices or products becoming more intricate, this problem is further exacerbated as wire bonding configurations incorporated within the semiconductor device or product become increasingly complex. As such, the limitations of inspecting wire bonds through two- dimensional vision inspection become increasingly obvious and evident.

[0008] Wire bond inspection techniques based on three-dimensional inspection techniques potentially offer a more comprehensive understanding of wire bond geometry, allowing for improved object inspection such as object identification and object classification. However, it comes with its own set of challenges.

[0009] Notably, three-dimensional inspection may produce a vast amount of raw point cloud data generated from three-dimensional scans, wherein the raw point cloud data of the wire bond structures are complex and noisy.

[0010] The challenges introduced by three-dimensional inspection may be overcome by leveraging on machine learning techniques for managing these point clouds. However, this in turn introduces yet another technical challenge, which relates to a feature extraction issue when implementing the aforementioned machine learning techniques. This is because the raw point cloud dataset representing a three-dimensional wire bond is sparse, and it may be unable to accentuate significant features related to the wire bond for machine learning-based wire bond inspection.

[0011] It is plausible that performing data transformation, such as a convolution calculation, may allow features to be extracted from the raw point cloud dataset, in which a feature- extracted point cloud dataset is generated. However, this feature-extracted point cloud dataset may lack a number of significant features from the raw point cloud dataset. Should this feature-extracted point cloud dataset be provided to a machine learning model for training, the machine learning model may be unable to learn these significant features, and thus the trained machine learning model may be inaccurate and inefficient. In addition, this feature -extracted point cloud dataset may have lost important connectivity information (e.g. relationship between features) during the data transformation process.

[0012] There are several prior arts that may relate to data generation, or data processing, of point cloud data. Among them include the United States Patent Application US2015123969A1, which discloses a method and apparatus for detecting a three- dimensional (3D) point cloud point of interest (POI). The apparatus comprises a 3D point cloud data acquirer to acquire 3D point cloud data, a shape descriptor to generate a shape description vector describing the shape of a surface in which a pixel point of a 3D point cloud and a neighbouring point of the pixel point is located, and a POI extractor to extract a POI based on the shape description vector.

[0013] Yet another disclosed technology includes the United States Patent Application US20180293740A1, which discloses a system for data fusion processing to identify obscured objects. Its system comprises a volumetric data source comprising three- dimensional (3D) imaging data of a scene, and a two-dimensional (2D) image source comprising 2D image data of the scene. The system also includes a processor operable to process the 2D data and the 3D data to generate a model of a material obscuring an object in the scene from sensors providing the 2D data and the 3D data. The processor is further operable to refine the model with detection data of the material from the volumetric data source, to detect the material obscuring the object based on the refined model, to generate an image of the scene, and to remove data pertaining to the material from the image to reveal the object in the image.

[0014] However, further inspection of the aforementioned prior art reveals that none of them disclose means of data generation that can potentially overcome the aforementioned challenges. Accordingly, it is desirable for a solution that enables a raw point cloud dataset to be extracted and constructed for significant features therein to be accentuated, so that the efficiency and robustness of training a machine learning model is enhanced, and would allow a trained machine learning model to perform better three-dimensional object inspection in real-world applications, such as three-dimensional wire bonding inspection.

[0015] SUMMARY OF INVENTION

[0016] The main objective of the invention is to provide a method and system for data generation that accentuates point cloud feature extraction and construction, to address the efficiency and robustness issues faced when training and deploying a machine learning model for three-dimensional object inspection, preferably for wire bond inspection.

[0017] Advantageously, the method and system for data generation provided by the present invention shall allow a machine learning model to be trained in such a way that significant information is captured from a raw point cloud dataset to be part of the generated data.

[0018] The present invention intends to provide a method for data generation, characterised in that the method comprises the steps of obtaining at least one point cloud dataset of an object, by at least one capturing unit, receiving the point cloud dataset of the object, by at least one processing unit, pre-processing the point cloud dataset of the object to produce a pre-processed point cloud dataset, by the processing unit, and generating one or more cumulative datasets from the pre-processed point cloud dataset of the object, by the processing unit. The step of generating one or more cumulative datasets from the pre-processed point cloud dataset of the object, by the processing unit further comprises the steps of generating at least one feature map dataset from the pre-processed point cloud dataset of the object, by a feature map generation module operated by the processing unit, generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, by a feature page portion generation module operated by the processing unit, generating one or more feature page datasets from corresponding feature page portion datasets, by a feature page generation module operated by the processing unit, and constructing the cumulative dataset from any one or a combination of the pre-processed point cloud dataset of the object, the feature map dataset, and the feature page datasets, by a cumulative data construction module operated by the processing unit.

[0019] Preferably, the method further comprises the step of generating subsequent feature page portions for a subsequent feature page dataset, by the feature page portion generation module, based on at least one feature page portion that was involved in the generation of a previous feature page dataset.

[0020] Preferably, the step of pre-processing the point cloud dataset of the object, by a processing unit, further comprises the steps of sampling the point cloud dataset of the object, by a sampling module of a pre-processing module for forming a sampled point cloud dataset, filtering the sampled point cloud dataset of the object, by a filtering module of the pre-processing module for forming the pre-processed point cloud dataset.

[0021] Preferably, the step of generating at least one feature map dataset from the pre- processed point cloud dataset of the object, by a feature map generation module operated by the processing unit further comprises the steps of executing kernel computation for the pre-processed point cloud dataset, by the feature map generation module, and executing point grouping-based mean distance calculation, by the feature map generation module.

[0022] Preferably, the step of generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, by a feature page portion generation module operated by the processing unit, further comprises the steps of performing generation of a first portion of a current feature page dataset, by a transformation module of the feature page portion generation module, based on any one of the feature map dataset or from at least generated feature page portion dataset, and performing generation of a second portion of the current feature page dataset, by a first fusion module of the feature page portion generation module, based on any one or both the feature map dataset or at least one generated feature page portion dataset, and the first portion of one feature page dataset.

[0023] Preferably, the step of generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, by a feature page portion generation module operated by the processing unit, further comprises the steps of performing generation of a third portion of the current feature page dataset, by a second fusion module of the feature page portion generation module, based on the second portion of the current feature page dataset, and performing generation of a fourth portion of the current feature page dataset, by the transformation module, based on any one or a combination of the feature map dataset or at least one generated feature page portion dataset, the first portion of the current feature page dataset, and the third portion of the current feature page dataset.

[0024] Preferably, the step of generating one or more feature page datasets from corresponding feature page portion datasets, by a feature page generation module operated by the processing unit, further comprises the step of constructing the current feature page dataset, by the feature page generation module, based on any one or a combination of the first portion of the current feature page dataset, the second portion of the current feature page dataset, the third portion of the current feature page dataset, and the fourth portion of the current feature page dataset.

[0025] Preferably, the method further comprises the steps of generating at least one trained object inspection model using the cumulative datasets, by the processing unit, in which at least one model is selected therefrom, and storing the trained object inspection model that was selected in at least one storage unit, together with its related information, by the processing unit.

[0026] Preferably, the method further comprises the steps of loading a trained object inspection model and its related information from at least one storage unit, by the processing unit, performing inspection of the object, by the processing unit, in which the cumulative dataset corresponding to the object is provided to the trained object inspection model as inputs, and providing one or more inspection results, by the processing unit, in which the inspection results are outputs from the trained object inspection model.

[0027] The present invention further intends to provide a system for data generation, characterised in that the system comprises at least one capturing unit for obtaining at least one point cloud dataset of an object, and at least one processing unit for receiving the point cloud dataset of the object, pre-processing the point cloud dataset of the object to produce a pre-processed point cloud dataset, and generating one or more cumulative datasets from the pre-processed point cloud dataset of the object. The processing unit operates a plurality of modules that include a feature map generation module for generating at least one feature map dataset from the pre-processed point cloud dataset of the object, a feature page portion generation module for generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, a feature page generation module for generating one or more feature page datasets from corresponding feature page portion datasets, and a cumulative data construction module for constructing the cumulative dataset from any one or a combination of the pre-processed point cloud dataset of the object, the feature map dataset, and the feature page datasets.

[0028] One skilled in the art will readily appreciate that the invention is well adapted to carry out the objects and obtain the ends and advantages mentioned, as well as those inherent therein. The embodiments described herein are not intended as limitations on the scope of the invention.

[0029] BRIEF DESCRIPTION OF DRAWINGS

[0030] These and other features, aspects, and advantages of the present invention will become better understood, when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0031] FIG. 1 is a block diagram illustrating an example system for data generation that corresponds to the method for data generation, according to an example embodiment of the present invention.

[0032] FIG. 2 is a block diagram illustrating the interactions between the components within the system for data generation, according to an example embodiment of the present invention.

[0033] FIG. 3 is a flowchart illustrating example steps of a method for data generation, according to an example embodiment of the present invention.

[0034] FIG. 4 is a flowchart illustrating example steps performed by a configuration module, which relate to generating one or more configuration settings, according to an example embodiment of the present invention.

[0035] FIG. 5 is a flowchart illustrating example steps performed by a pre-processing module, which relates to pre-processing a raw point cloud dataset of an object, according to an example embodiment of the present invention.

[0036] FIG. 6 is a flowchart illustrating example steps performed by a sampling module, which relate to sampling the raw point cloud dataset for forming a sampled point cloud dataset, according to an example embodiment of the present invention.

[0037] FIG. 7 is an illustrative depiction of the steps described in the flowchart of FIG. 6.

[0038] FIG. 8 is a flowchart illustrating example steps performed by a filtering module, which relate to filtering the sampled point cloud dataset for forming the pre-processed point cloud dataset, according to an example embodiment of the present invention.

[0039] FIG. 9 is an illustrative depiction of the steps described in the flowchart of FIG. 8.

[0040] FIG. 10 is a flowchart illustrating example steps performed by a cumulative dataset generation module, which relates to generating one or more cumulative datasets from the pre-processed point cloud dataset, according to an example embodiment of the present invention.

[0041] FIG. 11 is a flowchart illustrating example steps performed by a feature map generation module, which relates to generating one or more feature page datasets, according to an example embodiment of the present invention.

[0042] FIG. 12 is an illustrative depiction of the steps described in the flowchart of FIG. 11.

[0043] FIG. 13 is a flowchart illustrating example steps further performed by the feature map generation module, which relate to executing kernel computations for the pre-processed point cloud dataset for generating a kernel -computed point cloud dataset, according to an example embodiment of the present invention.

[0044] FIG. 14 is an illustrative depiction of the steps described in the flowchart of FIG. 13.

[0045] FIG. 15 is a flowchart illustrating example steps further performed by the feature map generation module, which relate to executing point grouping-based mean distance calculation upon the pre-processed point cloud dataset, according to an example embodiment of the present invention.

[0046] FIG. 16 is an illustrative depiction of the steps described in the flowchart of FIG. 15.

[0047] FIG. 17 is a flowchart illustrating example steps performed by a feature page portion generation module and a feature page generation module, which relate to generating one or more feature page portion datasets and one or more feature page datasets, according to an example embodiment of the present invention.

[0048] FIG. 18 is a flowchart illustrating example steps performed by a transformation module, which relate to generating a first portion of one feature page dataset.

[0049] FIG. 19 is an illustrative depiction of the steps described in the flowchart of FIG. 18.

[0050] FIG. 20 is a flowchart illustrating example steps further performed by the transformation module, which relate to executing kernel computation for a first variable, according to an example embodiment of the present invention.

[0051] FIG. 21 is an illustrative depiction of the steps described in the flowchart of FIG. 20.

[0052] FIG. 22 is a flowchart illustrating example steps further performed by the transformation module, which relate to generating a first row -based feature change map dataset, according to an example embodiment of the present invention.

[0053] FIG. 23 is an illustrative depiction of the steps described in the flowchart of FIG. 22.

[0054] FIG. 24 is a flowchart illustrating example steps further performed by the transformation module, which relate to generating a first column-based feature change map dataset, according to an example embodiment of the present invention.

[0055] FIG. 25 is an illustrative depiction of the steps described in the flowchart of FIG. 24.

[0056] FIG. 26 is a flowchart illustrating example steps further performed by a first fusion module, which relates to generating a second portion of one feature page dataset, according to an example embodiment of the present invention.

[0057] FIG. 27 is an illustrative depiction of the steps described in the flowchart of FIG. 26.

[0058] FIG. 28 is a flowchart illustrating example steps performed by a transformation module, which relates to generating a third portion of one feature page dataset.

[0059] FIG. 29 is an illustrative depiction of the steps described in the flowchart of FIG. 28.

[0060] FIG. 30 is a flowchart illustrating example steps further performed by the transformation module, which relate to executing kernel computation for the second portion of the feature page dataset, according to an example embodiment of the present invention.

[0061] FIG. 31 is an illustrative depiction of the steps described in the flowchart of FIG. 30.

[0062] FIG. 32 is a flowchart illustrating example steps further performed by the transformation module, which relate to generating a second row-based feature change map dataset, according to an example embodiment of the present invention.

[0063] FIG. 33 is an illustrative depiction of the steps described in the flowchart of FIG. 32.

[0064] FIG. 34 is a flowchart illustrating example steps further performed by the transformation module, which relate to generating a second column-based feature change map dataset, according to an example embodiment of the present invention.

[0065] FIG. 35 is an illustrative depiction of the steps described in the flowchart of FIG. 34.

[0066] FIG. 36 is a flowchart illustrating example steps performed by a second fusion module, which relates to generating a fourth portion of one feature page dataset, according to an example embodiment of the present invention.

[0067] FIG. 37 is an illustrative depiction of the steps described in the flowchart of FIG. 35.

[0068] FIG. 38 is a flowchart illustrating example steps performed by the feature page generation module, which relates to generating the feature page dataset, according to an example embodiment of the present invention.

[0069] FIG. 39 is an illustrative depiction of the steps described in the flowchart of FIG. 38.

[0070] FIG. 40 is a flowchart illustrating example steps performed by the cumulative data construction module, which relates to generating the cumulative dataset, according to an example embodiment of the present invention.

[0071] FIG. 41 is an illustrative depiction of the steps described in the flowchart of FIG. 40.

[0072] FIG. 42 is a flowchart illustrating example steps that may be further included in the method for data generation, which involves generating a trained object inspection model based on the generated data, according to an example embodiment of the present invention.

[0073] FIG. 43 is an illustrative depiction of the steps described in the flowchart of FIG. 42.

[0074] FIG. 44 is a flowchart illustrating example steps that may be further included in the method for data generation, which involves performing object inspection based on the generated data that is provided to the trained object inspection model, according to an example embodiment of the present invention.

[0075] FIG. 45 is an illustrative depiction of the steps described in the flowchart of FIG. 45.

[0076] FIG. 46 illustrates a diagram that depicts graphs that compare training loss between a setup that utilised the method and system of the present invention for training a machine learning model, and a setup that does not utilise the method and system of the present invention fortraining a machine learning model.

[0077] FIG. 47 illustrates a diagram that depicts graphs that compare training accuracy between a setup that utilised the method and system of the present invention fortraining a machine learning model, and a setup that does not utilise the method and system of the present invention for training a machine learning model.

[0078] DETAILED DESCRIPTION OF THE INVENTION

[0079] The present invention provides a method and system for data generation. In particular, at least one raw point cloud dataset captured from an object is made to undergo a series of steps for one or more cumulative datasets to be generated therefrom.

[0080] Furthermore, the method and system for data generation of the present invention may further provide training of a machine learning model based on one or more cumulative datasets that were generated. It is also to be noted that the cumulative datasets may further accentuate significant features of the raw point cloud dataset in a quantitative manner, hence, the resulting trained machine learning model may have a higher degree of accuracy compared to a machine learning model trained without the cumulative datasets.

[0081] Furthermore, the method and system for data generation of the present invention may further provide object inspection for identification and classification of its related information, based on at least one raw point cloud dataset captured from an object. The raw point cloud dataset is made to undergo a series of processing steps for one or more cumulative datasets to be generated therefrom, in which the cumulative dataset is provided to a trained machine learning model for the object, or its attributes thereof, to be identified and classified. This may include defects of the object.

[0082] According to the concept of the invention, there is included a plurality of modules operated by processing unit(s) that receives the captured raw point cloud dataset from a capturing unit for generating the cumulative dataset. In particular, the modules may operate in conjunction and process the raw point cloud dataset to generate the cumulative dataset from any one or a combination of (i) a pre-processed point cloud dataset, (ii) a feature map dataset, and (iii) a plurality of feature page datasets.

[0083] In particular, each feature page dataset may be generated from a combination of portioned feature page datasets. More specifically, the generation of subsequent feature page portions for a subsequent feature page dataset, by the feature page portion generation module, may be based on at least one feature page portion that was involved in the generation of a previous feature page dataset (i.e. a previous feature page portion of the previous feature page dataset).

[0084] It is to be noted that one feature map dataset may represent a map of features that are derived from the pre-processed point cloud dataset. More specifically, it may have a numerical set of data that corresponds to an initial set of patterns that may be within the pre-processed point cloud dataset.

[0085] It is to be noted that one feature page dataset may represent yet another map of features that is derived from either the feature map dataset or one previous feature page dataset. With that, the feature page dataset may have a numerical set of data that corresponds to a succeeding set of patterns that may be within the initial set of patterns of one feature map dataset or a previous set of patterns of one previous feature page dataset. With that, it may be stated that the succeeding set of patterns may represent finer or more detailed patterns within the initial set of patterns or the previous set of patterns.

[0086] The invention will now be described in greater detail, by way of example, with reference to the drawings.

[0087] FIG. 1 illustrates is a block diagram representation of a system 5000 for data generation. As shown, the system 5000 may comprise at least one capturing unit 5100, at least one processing unit 5200, at least one storage unit 5300, and at least one humanmachine interface 5400, or any combination thereof.

[0088] Regarding the capturing unit 5100, it may be configured to receive one or more objects . These objects may further be inspected by the capturing unit 5100. The capturing unit 5100 may comprise a plurality of sensors that allow data to be captured from the object that is provided thereto. In a preferred embodiment, it has sensors that may enable a raw point cloud dataset to be obtained from the object. These sensors may relate to Light Detection and Ranging (LiDAR) technology, photogrammetry technology, or the like. The raw point cloud dataset may be in a form of a three-dimensional point cloud format, in which the object is represented by one or more data points distributed in an arbitrary space wherein the data points may have coordinate values assigned thereto.

[0089] Regarding the processing unit 5200, it may be interfaced with any one or a combination of the capturing unit 5100, the storage unit 5300, and the human -machine interface 5400. Preferably, it may be, but shall not be limited to, a conventional processor, an application-specific integrated circuit (ASIC), a field -programmable gate array (FPGA), a graphics processing unit (GPU), or a combination thereof.

[0090] Furthermore, the processing unit 5200 may have at least one application software running thereon for operating a plurality of modules.

[0091] The plurality of modules operated by the processing unit 5200 may include a pre- processing module 5210, which may comprise one or more internal modules that include a sampling module 5211 and a filtering module 5212. These modules are configured to perform steps that relate to pre-processing the raw point cloud dataset, for it to become a pre-processed point cloud dataset.

[0092] The plurality of modules operated by the processing unit 5200 may further include a cumulative dataset generation module 5220, which may comprise one or more internal modules that include a feature map generation module 5221, a feature page portion generation module 5222, a feature page generation module 5223, and a cumulative data construction module 5224. Furthermore, the feature page portion generation module 5222 may comprise one or more internal modules that include a transformation module 52221, a first fusion module 52222, and a second fusion module 52223. These modules are configured to perform steps that relate to generating one or more cumulative datasets from the pre-processed point cloud dataset.

[0093] The plurality of modules operated by the processing unit 5200 may further include a configuration module 5230. This module is configured to perform steps that relate to generating at least one configuration settings for the system 5000.

[0094] The plurality of modules operated by the processing unit 5200 may further include a model training module 5240. In particular, the model training module 5240 may operate algorithms related to generation and training of one or more machine learning models, which may include artificial neural network algorithms, convolutional neural network algorithms, or the like. More specifically, the model training module 5240 may generate at least one trained machine learning model based on the cumulative datasets that are provided thereto.

[0095] The plurality of modules operated by the processing unit 5200 may further include an object inspection module 5250. In particular, this module may be configured to deploy one trained object inspection model for inspection to be performed upon an object provided to the system 5000 based on its cumulative dataset derived therefrom.

[0096] The plurality of modules operated by the processing unit 5200 may further include a user interface module 5260. In particular, this module may be configured to be in communication with the human-machine interface unit 5400 to provide at least one graphical user interface to be displayed thereon. With this, at least one user may interact with the system 5000 via the graphical user interface displayed on the human-machine interface unit 5400 for providing manual input of configuration settings, viewing generated cumulative datasets, viewing training results for an object inspection model undergoing training, or viewing inspection results outputted from a trained object inspection model.

[0097] It should be noted that the aforementioned modules may be substantially in communication with each other. Furthermore, the aforementioned modules need not be in a software embodiment, and they may be in a hardware embodiment where they are directly connected to the processing unit 5200.

[0098] Regarding the storage unit 5300, it may further comprise at least one non-volatile memory unit 5310 for data to be stored therein in a permanent manner or in a semipermanent manner, and at least one volatile memory unit 5320 for data to be stored therein in a non-permanent manner. Embodiments for such units are well understood by a person skilled in the art and not elaborated further. In particular, the non-volatile memory unit 5310 may store data files such as program files related to the modules, cumulative datasets that were generated from the object, weights and biases of one or more trained object inspection models, inspection results of the object, or the like. In particular, the volatile memory unit 5320 may store temporary data files are created during the runtime of the modules or the system 5000.

[0099] Regarding the human-machine interface unit 5400, it may be in the form of a device that has a display for enabling graphics of a graphical user interface provided by the user interface module 5260 to be displayed thereon. Furthermore, it may be configured to receive inputs from the user via external peripherals such as a mouse and a keyboard, or via touch-screen technology such as capacitive-sensing, or the like.

[0100] FIG. 2 illustrates a block diagram of the system 5000 for data generation, which further illustrates the interactions between the components and modules within the present invention as previously described for FIG. 1.

[0101] With reference to FIG. 2, the capturing unit 5100 may obtain a raw point cloud dataset from an object.

[0102] With reference to FIG. 2, the raw point cloud dataset is provided to the pre-processing module 5210 as its input. In particular, the raw point cloud dataset may be provided to the sampling module 5211, and subsequently, the filtering module 5212, for a pre- processed point cloud dataset to be generated.

[0103] With reference to FIG. 2, the pre-processed point cloud dataset is provided to the cumulative dataset generation module 5220.

[0104] With reference to FIG. 2, the cumulative data construction module 5224 of the cumulative dataset generation module 5220 may be provided with the pre-processed point cloud dataset as one of its inputs (i.e. "pre-processed point cloud').

[0105] With reference to FIG. 2, the feature map generation module 5221 of the cumulative dataset generation module 5220 may receive the pre-processed point cloud dataset as its input, and it shall generate a feature map dataset as its output. This feature map dataset may then be provided to the cumulative data construction module 5224 of the cumulative dataset generation module 5220 as one of its inputs (i.e. “feature map”).

[0106] With reference to FIG. 2, for generation of a first feature page (i.e. “feature Page / ") during a first pass or iteration, the feature page portion generation module 5222 may be provided with the feature map dataset (i.e. “feature map”), generated by the feature map generation module 5221, as its input, for one or more feature page portions of the first feature page to be generated as its outputs. More specifically:

[0107] • a transformation module 52221 of the feature page portion generation module 5222 is provided with the feature map dataset (i.e. “feature map”) as its input, for a first portion of a first feature page dataset (i.e. “feature map 1.1”) to be produced as its output;

[0108] • a first fusion module 52222 of the feature page portion generation module 5222 is provided with the feature map dataset (i.e. “feature map”) and the first portion of the first feature page dataset (i.e. “feature map 1.1”) as its inputs, for a second portion of the first feature page dataset (i.e. “feature map 1.2”) to be produced as its output;

[0109] • a transformation module 52221 of the feature page portion generation module 5222 is provided with a second portion of the first feature page dataset (i.e. “feature map 1.2”) as its input, for a third portion of a first feature page dataset (i.e. “feature map 1.3”) to be produced as its output; and

[0110] • a second fusion module 52223 of the feature page portion generation module 5222 is provided with the feature map dataset (i.e. “feature map”), the first portion of the first feature page dataset (i.e. “feature map 1.1”), and the third portion (i.e. “feature map 1.3”) of the first feature page dataset as its inputs, for a fourth portion of the first feature page dataset (i.e. “feature map 1.4”) to be produced as its output;

[0111] With reference to FIG. 2, for generation of a first feature page dataset (i.e. “feature page #1 ”) during the first pass or iteration, the feature page generation module 5223 may receive the first portion of the first feature page dataset (i.e. "feature map 7.7”), the second portion of the first feature page dataset (i.e. "feature map 1.2”), the third portion of the first feature page dataset (i.e. “feature map 1.3”), and the fourth portion of the first feature page dataset (i.e. “feature map 1.4”). With this, the feature page generation module 5223 may produce the first feature page dataset (i.e. “feature page #1”). This first feature page dataset (i.e. “feature page #7”) may then be provided to the cumulative data construction module 5224 as one of its inputs.

[0112] With reference to FIG. 2, for generation of a subsequent feature page dataset during a subsequent pass or iteration, the feature page portion generation module 5222 may be provided with a fourth portion of a previous feature page dataset as its input, for one or more feature page portions of a subsequent feature page to be generated as its outputs for the subsequent feature page dataset to be generated therefrom.

[0113] For example, for a second feature page dataset (i.e. “feature page #2”) to be generated:

[0114] • the transformation module 52221 of the feature page portion generation module 5222 is provided with the fourth portion of the first feature page dataset (i.e. “feature map 1.4”) as its input, for a first portion of the second feature page dataset (i.e. “feature map 2.1”, not shown) to be produced as its output;

[0115] • the first fusion module 52222 of the feature page portion generation module 5222 is provided with the fourth portion of the first feature page dataset (i.e. “feature map 1.4”) and the first portion of the second feature page dataset (i.e. “feature map 2.1 ”, not shown) as its inputs, for a second portion of the second feature page dataset (i.e. “feature map 2.2”, not shown) to be produced as its output;

[0116] • the transformation module 52221 is provided with the second portion of the second feature page dataset (i.e. “feature map 2.2”, not shown) as its input, for a third portion of the second feature page dataset (i.e. “feature map 2.3”, not shown) to be produced as its output;

[0117] • the second fusion module 52223 is provided with the fourth portion of the first feature page dataset (i.e. “feature map 1.4”), the first portion of the second feature page dataset (i.e. “feature map 2.3”, not shown), and the third portion of the second feature page dataset (i.e. “feature map 2.3”, not shown) as its inputs, for a fourth portion of the second feature map dataset (i.e. “feature map 2.4”, not shown) to be produced as its output;

[0118] The generation of subsequent feature page datasets, until the final feature page dataset (i.e. “feature page n ”), may involve modules and steps that are similar to that of the generation of the second feature page dataset.

[0119] With reference to FIG. 2, it is to be noted that the cumulative data construction module 5224, which had received the pre-processed point cloud dataset (i.e. “pre-processed point cloud”), the feature map dataset (i.e. “feature map”), and the plurality of feature pages datasets (i.e. “feature page #1 ”, “feature page #2”, ..., “feature page #n ”) as its inputs, shall produce a cumulative dataset as its output. This output may be regarded as a final output from the cumulative dataset generation module 5220.

[0120] With reference to FIG. 2, the cumulative dataset produced by the cumulative dataset generation module 5220 may either be stored in the storage unit 5300, be provided to the object inspection module 5250 for inspection of the object, or a combination thereof.

[0121] It is to be noted that for the rest of the description, the described hardware and software components of the system 5000 may or may not be directly implicated. However, it is to be understood by a skilled person that the descriptions of the hardware and software components described above provide support for the rest of the description, and may also further complement the rest of the description.

[0122] From hereon, one or more flowcharts pertaining to the method of the present invention data generation are to be described. It is to be noted that the steps described in these flowcharts are not to be interpreted as non-limiting, and minor modifications to the steps (e.g. additions, omissions, repetitions, swaps, or the like) are permissible by a skilled person without substantial deviation from as described.

[0123] FIG. 3 illustrates a flowchart describing the example steps of a method for data generation, according to an example embodiment of the present invention.

[0124] The process steps of FIG. 3 may begin with step 301, which may involve generating one or more configuration settings. This step may be performed by the configuration module 5230 operated by the processing unit 5200.

[0125] Following step 301 is step 302. Step 302 may involve providing at least one object to at least one capturing unit 5100. The objects that are provided to the capturing unit 5100 may be, by way of example, semiconductor components each having a plurality of wire bonds. These objects may be provided to the capturing unit 5100 via a transport means that may be, by way of example, a transport unit, a conveyor, or the like.

[0126] Following step 302 is step 303. Step 303 may involve obtaining at least one raw point cloud dataset of the object. This step may be performed by the capturing unit 5100. More specifically, the raw point cloud dataset may be in a form of a three-dimensional point cloud format, in which the object is represented by one or more data points distributed in an arbitrary space wherein the data points may have coordinate values assigned thereto. By way of example, the arbitrary space may be in the form of a three- dimensional Cartesian space, and the data points may each have Cartesian coordinate values assigned thereto that correspond to their positions along any one or a combination of an x-axis, a y-axis, and a z-axis.

[0127] Following step 303 is step 304. Step 304 may involve receiving the raw point cloud dataset of the object, by the processing unit 5200.

[0128] Following step 304 is step 305. Step 305 may involve pre-processing the raw point cloud dataset of the object. This step may be performed by the pre-processing module 5210 operated by the processing unit 5200. It is to be noted that this step may further involve any one or a combination of sampling and filtering of the raw point cloud dataset for the removal of unwanted components such as noise or unwanted data. With this, a pre-processed point cloud dataset is produced.

[0129] Following step 305 is step 306. Step 306 may involve generating one or more cumulative datasets from the pre-processed point cloud dataset. This step may be performed by the cumulative dataset generation module 5220 operated by the processing unit 5200.

[0130] Finally, following step 306 is step 307. Step 307 may involve storing the cumulative dataset(s) that were generated, in the storage unit 5300. This step may be performed by the processing unit 5200. In particular, the cumulative datasets may be stored within the storage unit 5300 in a permanent manner or in a semi-permanent manner.

[0131] FIG. 4 illustrates a flowchart describing the example steps that relate to generating one or more configuration settings. The steps of FIG. 4 may be performed by the configuration module 5230 operated by the processing unit 5200. The steps of FIG. 4 may be regarded as sub-steps of step 301 of FIG. 3.

[0132] The steps of FIG. 4 may begin with step 401, which may involve prompting a user to select either one of an automatic generation or manual generation of a system configuration, by the configuration module 5230. In particular, the configuration module 5230 may be interfaced with a user interface module 5260, for displaying a corresponding graphical user interface on a human-machine interface unit 5400, so that the user is prompted.

[0133] Following step 401 is step 402. Step 402 is a decision step that involves (i) determining that the user had selected automatic generation, or (ii) determining that the user had selected manual generation. Should the former be determined, step 402 proceeds to step 403. Else, step 402 proceeds to step 407.

[0134] In step 403, since it was determined that the user had selected automatic generation, the step of determining an available memory resource in the storage unit 5300, is performed by the configuration module 5230. In particular, the configuration module 5230 may determine either a current capacity of volatile memory that is available in the storage unit 5300, a current capacity of non-volatile memory that is available in the storage unit 5300, or a combination thereof.

[0135] Following step 403 is step 404. Step 404 involves determining a required memory for completing one feature page dataset, by the configuration module 5230.

[0136] Following step 404 is step 405. Step 405 involves determining a maximum number of feature page datasets that are capable of being generated, by the configuration module 5230.

[0137] Following step 405 is step 406. Step 406 involves generating the configuration settings based on the number maximum of feature pages that was determined, by the configuration module 5230.

[0138] In step 407, since it was determined that the user had selected manual generation, the step of allowing the user to provide inputs for the user to generate the configuration settings, is performed by the configuration module 5230.

[0139] The final step is step 408. Step 408 may follow from step 406 or step 407. Step 408 involves storing the configuration settings that had been generated in the storage unit 5300, by the configuration module 5230.

[0140] FIG. 5 illustrates a flowchart describing example steps that may relate to pre-processing the raw point cloud dataset of an object. The steps of FIG. 5 may be performed by a pre-processing module 5210 operated by the processing unit 5200. The steps in FIG. 5 may be regarded as sub-steps of step 305 of FIG. 3.

[0141] The steps of FIG. 5 may begin with step 501, which may involve sampling the raw point cloud dataset, by a sampling module 5211 of the pre-processing module 5210. As a result, a sampled point cloud dataset may be produced.

[0142] Finally, following step 501 is step 502. Step 502 may involve filtering the sampled point cloud dataset, by a filtering module 5212 of the pre-processing module 5210. As a result, a filtered point cloud dataset is produced which is to be considered as the pre- processed point cloud dataset.

[0143] FIG. 6 illustrates a flowchart describing example steps that may relate to sampling the raw point cloud dataset. The steps of FIG. 6 may be performed by the sampling module 5211. The steps in FIG. 6 may be regarded as sub-steps of step 501 of FIG. 5. The steps of FIG. 6 are illustratively depicted in FIG. 7.

[0144] The steps of FIG. 6 may begin with step 601, which may involve loading the raw point cloud dataset, by the sampling module 5211. The raw point cloud dataset may be loaded from the capturing unit 5100, the storage unit 5300, or a combination thereof.

[0145] Following step 601 is step 602. Step 602 may involve sampling data points within the raw point cloud dataset, by the sampling module 5211. This step may be done based on the configuration setting that was previously generated.

[0146] Finally, following step 602 is step 603. Step 603 may involve producing the sampled point cloud dataset, by the sampling module 5211. As shown in FIG. 7, the sampled point cloud dataset has a fewer number of points compared to the raw point cloud dataset.

[0147] FIG. 8 illustrates a flowchart describing example steps that may relate to filtering the sampled point cloud dataset. The steps of FIG. 8 may be performed by the filtering module 5212. The steps in FIG. 8 may be regarded as sub-steps of step 502 of FIG. 5. The steps of FIG. 8 are illustratively depicted in FIG. 9.

[0148] The steps of FIG. 8 may begin with step 801, which may involve loading the sampled point cloud dataset, by the filtering module 5212. The sampled point cloud dataset may be loaded from sampling module 5211, the storage unit 5300, or a combination thereof.

[0149] Following step 801 is step 802. Step 802 may involve determining one or more ranges for a region of interest within the sampled point cloud dataset by the filtering module 5212. In particular, the ranges for the region of interest may be along any one or a combination of an x-axis [xmin, xmax\. a y-axis \ymin, ymax\, and a z-axis [zmin, zmax] .

[0150] Following step 802 is step 803. Step 803 may involve selecting one unique or distinct point within the sampled point cloud dataset, by the filtering module 5212.

[0151] Following step 803 is step 804. Step 804 is a decision step that involves (i) determining that the selected point is within the range of the region of interest, or (ii) determining that the selected point is not within or outside the range of the region of interest. Should the former be determined, step 804 proceeds to step 803 and repeats therefrom. Else, should the latter be determined, step 804 proceeds to step 805.

[0152] In step 805, since it was determined that the selected point is not within or outside the range of the region of interest, the step of removing the selected point from the sampled point cloud dataset is performed by the filtering module 5212.

[0153] Step 806 may follow from step 804 or step 805. Step 806 may be yet another decision step that involves (i) determining that all points in the point cloud dataset have been inspected, or (ii) determining that there are still remaining points in the point cloud dataset that are yet to be inspected. Should the former be determined, step 806 proceeds to step 807. Else, should the latter be determined, step 806 returns to step 803 and repeats therefrom.

[0154] Finally, in step 807, since it was determined that all points in the sampled point cloud dataset have been inspected, the step of producing a filtered point cloud dataset is performed by the filtering module 5212. The filtered point cloud dataset may be regarded as the pre-processed point cloud dataset. As shown in FIG. 9, the pre- processed point cloud dataset has a fewer number of points compared to the sampled point cloud dataset.

[0155] FIG. 10 illustrates a flowchart describing example steps that may relate to generating one or more cumulative datasets from the pre-processed point cloud dataset. The steps of FIG. 10 may be performed by a cumulative dataset generation module 5220 operated by the processing unit 5200. The steps in FIG. 10 may be regarded as sub-steps of step

[0156] 306 of FIG. 3

[0157] The steps of FIG. 10 may begin with step 1001, which may involve loading the pre- processed point cloud dataset, by the cumulative dataset generation module 5220. The pre-processed point cloud dataset may be loaded from the pre-processing module 5210, the storage unit 5300, or a combination thereof.

[0158] Following step 1001 is step 1002. Step 1002 may involve generating at least one feature map dataset, by a feature map generation module 5221 of the cumulative dataset generation module 5220.

[0159] Following step 1002 is step 1003. Step 1003 may involve generating one or more feature page portion datasets by a feature page portion generation module 5222 of the cumulative dataset generation module 5220, and one or more feature page datasets, by a feature page generation module 5223 of the cumulative dataset generation module 5220.

[0160] Finally, following step 1003 is step 1004. Step 1004 may involve performing the generation of the cumulative dataset, by a cumulative data construction module 5224 of the cumulative dataset generation module 5220, from any one or a combination of (i) the pre-processed point cloud dataset, (ii) the feature map dataset generated by the feature map generation module 5221, and (iii) the feature page datasets generated by the feature page generation module 5223.

[0161] FIG. 11 illustrates a flowchart describing example steps that may relate to generating at least one feature map dataset. The steps of FIG. 11 may be performed by the feature map generation module 5221. The steps in FIG. 11 may be regarded as sub-steps of step 1002 of FIG. 10. FIG. 12 illustratively depicts the steps of FIG. 11 and these steps may be described with reference therewith.

[0162] The steps of FIG. 11 may begin with step 1101, which may involve loading the pre- processed point cloud dataset, by the feature map generation module 5221. The pre- processed point cloud dataset may be loaded from the pre-processing module 5210, the storage unit 5300, or a combination thereof.

[0163] With reference to FIG. 12, the pre-processed point cloud dataset may be represented as a table having one or more rows and columns that correspond to coordinate values of each point of the point cloud. Its row headings may each represent a point index in the pre-processed point cloud dataset, and its column headings may represent a location of a point index along the x-axis, y-axis, and z-axis.

[0164] Following step 1101 is step 1102. Step 1102 is a decision step that involves (i) determining that addition of kernel computation is required, or (ii) determining that addition of kernel computation is redundant. Should the former be determined, step 1102 proceeds to step 1103. Else, should the latter be determined, step 1102 proceeds to step 1104.

[0165] In step 1103, since it was determined that addition of kernel computation is required, the step of executing kernel computation for the pre-processed point cloud dataset is performed by the feature map generation module 5221. With that, a kernel -computed pre-processed point cloud dataset is produced as shown in FIG. 12.

[0166] Step 1104 follows from step 1102 or step 1103. Step 1104 is yet another decision step that involves (i) determining that addition of a point grouping-based mean distance calculation is required, or (ii) determining that addition of a point grouping-based mean distance calculation is redundant. Should the former be determined, step 1104 proceeds to step 1105. Else, should the latter be determined, step 1104 proceeds to step 1106.

[0167] In step 1105, since it was determined that addition of a point grouping-based mean distance calculation is required, the step of executing point grouping-based mean distance calculation is performed by the feature map generation module 5221. With that, a point grouped pre-processed point cloud dataset is produced as shown in FIG. 12.

[0168] Finally, following step 1104 or step 1105 is step 1106. Step 1106 involves producing the feature map dataset, by feature map generation module 5221, as shown in FIG. 12. In particular, the feature map dataset produced by horizontally concatenating the kernel- computed pre-processed point cloud dataset and the point grouped pre-processed point cloud dataset. With that, as shown in FIG. 12, the feature map dataset may have shared row headings that each correspond to point indexes, while having combined column headings and corresponding entries from both the kernel-computed pre-processed point cloud dataset and the point grouped pre-processed point cloud dataset.

[0169] FIG. 13 illustrates a flowchart describing example steps that may relate to executing kernel computation for the pre-processed point cloud dataset. The steps of FIG. 13 may be performed by the feature map generation module 5221. The steps in FIG. 13 may be regarded as sub-steps of step 1103 of FIG. 11. FIG. 14 illustratively depicts the steps of FIG. 13 and these steps may be described with reference therewith.

[0170] The steps of FIG. 13 may begin with step 1301, which may involve loading the pre- processed point cloud dataset, by the feature map generation module 5221. The pre- processed point cloud dataset may be loaded from the pre-processing module 5210, the storage unit 5300, or a combination thereof.

[0171] Following step 1301 is step 1302. Step 1302 involves performing convolution computation based on a kernel, by the feature map generation module 5221, in which the kernel may be user-defined or pre-calculated. In particular, the kernel may be in the form of a matrix having an array of values as shown in FIG. 14. Furthermore, the convolutional operation may be performed between numerical entries within the tabular representation of the pre-processed point cloud dataset and the kernel.

[0172] Following step 1302 is step 1303. Step 1303 is a decision step that involves (i) determining that adding activation is required, or (ii) determining that adding activation is redundant. Should the former be determined, step 1303 proceeds to step 1304. Else, should the latter be determined, step 1303 proceeds to step 1305.

[0173] In step 1304, since it was determined that adding activation is required, the step of performing activation computation is performed by the feature map generation module 5221. In particular, the data may be projected to any one of a linear function, a nonlinear function, or a user-defined function. These functions may be previously defined in the generated configuration setting.

[0174] Step 1305 may follow from step 1303 or step 1304. Step 1305 is a decision step that involves (i) determining that adding normalisation is required, or (ii) determining that adding normalisation is redundant. Should the former be determined, step 1005 proceeds to step 1306. Else, should the latter be determined, step 1305 proceeds to step 1307.

[0175] In step 1306, since it was determined that adding normalisation is required, the step of performing normalisation computation is performed by the feature map generation module 5221. In particular, the data may be normalised for it to be within a user-defined range. This range may be previously defined in the generated configuration setting.

[0176] Finally, step 1307 may follow from step 1305 or step 1306. Step 1307 involves producing a kernel computed pre-processed point cloud dataset, by the feature map generation module 5221. The kernel computed pre-processed point cloud dataset may be represented in the form of a table, in which its rows headers each correspond to a point index, and its columns headers each correspond to a feature index.

[0177] FIG. 15 illustrates a flowchart describing example steps that may relate to executing point grouping-based mean distance calculation. The steps of FIG. 15 may be performed by the feature map generation module 5221. The steps in FIG. 15 may be regarded as sub-steps of step 1105 of FIG. 11. FIG. 16 illustratively depicts the steps of FIG. 15 and these steps may be described with reference therewith.

[0178] The steps of FIG. 15 may begin with step 1501, which may involve loading the pre- processed point cloud dataset, by the feature map generation module 5221. The pre- processed point cloud dataset may be loaded from within feature map generation module 5221, the storage unit 5300, or a combination thereof.

[0179] Following step 1501 is step 1502. Step 1502 involves selecting one unique point index from the pre-processed point cloud dataset, by the feature map generation module 5221.

[0180] Following step 1502 is step 1503. Step 1503 involves computing one or more distances between the selected point index and each of the other points, by the feature map generation module 5221. This is illustratively shown in FIG. 16.

[0181] Following step 1503 is step 1504. Step 1504 involves sorting the point indexes of the pre-processed point cloud dataset, and their corresponding information, according to the distance that was computed in Step 1503, by the feature map generation module 5221. In particular, the sorting may be done for the distances to be in an ascending order from a minimum distance to a maximum distance.

[0182] Following step 1504 is step 1505. Step 1505 involves arranging the points within the pre-processed point cloud dataset into one or more groups, by the feature map generation module 5221. In particular, this may be done based on at least one set of defined rules. An example of the defined rules is illustratively shown in FIG. 16 (“Point Grouping”), which had shown the points being arranged in groups of five. It is to be noted that the defined rules that relate to the number of groups to be made may be based on configuration settings defined by the user. By way of a further example, should there be 100 points in the pre-processed point cloud dataset and the configuration settings defined by the user requires 10 groups to be made, each group shall have 10 points.

[0183] Following step 1505 is step 1506. Step 1506 involves calculating a mean distance for the selected point index with respect to each point within each group, by the feature map generation module 5221. In particular, calculation of the mean distance may be done by averaging the computed distances previously computed in step 1503 based on the number of points within one group.

[0184] Following step 1506 is step 1507. Step 1507 involves generating an entry in a mean distance map dataset, by the feature map generation module 5221. The mean distance map dataset may be represented as a table, in which its row headers each correspond to a point index, and its column headers each correspond to a mean distance of the point index with respect to each group of points.

[0185] For example, with reference to FIG. 16, for an example entry in the mean distance map dataset “MDn”, the formula for its calculation may be as expressed in Formula (1) shown below:

[0186] Following step 1507 is step 1508. Step 1508 is a decision step that involves (i) determining that all points in the pre-processed point cloud dataset have been processed, or (ii) determining that there are still points in the pre-processed point cloud dataset that are yet to be processed. Should the former be determined, step 1508 proceeds to step 1510. Else, should the latter be determined, step 1508 proceeds to step 1509.

[0187] In step 1509, since it was determined that there are still points in the pre-processed point cloud dataset that are yet to be processed, the step of selecting another unique point index within the pre-processed point cloud dataset is performed by the feature map generation module 5221. With that, step 1509 returns to step 1503 and repeats therefrom.

[0188] Finally, in step 1510, since it was determined that all points in the pre-processed point cloud dataset have been processed, the step of producing the mean distance map dataset is performed by the feature map generation module 5221.

[0189] It is to be noted that, alternatively, the feature map generation module 5221 may perform point grouping -based median distance calculation. The aforementioned steps of FIG. 15 may be performed in a similar manner, but with the mean distance calculation replaced with a median distance calculation instead.

[0190] FIG. 17 illustrates a flowchart describing example steps that may relate to generating one or more feature page portion datasets by a feature page portion generation module 5222 of the cumulative dataset generation module 5220, and one or more feature page datasets, by a feature page generation module 5223 of the cumulative dataset generation module 5220. The steps in FIG. 17 may be regarded as sub-steps of step 1003 of FIG. 10.

[0191] The steps of FIG. 17 may begin with step 1701, which may involve loading the feature map dataset by the feature page generation module 5223. The feature map dataset may be loaded from the feature map generation module 5221, the storage unit 5300, or a combination thereof.

[0192] Following step 1701 is step 1702. Step 1702 may involve setting the feature map dataset as a first variable, by the feature page portion generation module 5222.

[0193] Following step 1702 is step 1703. Step 1703 may involve generating a first portion of a current feature page dataset, by a transformation module 52221 of the feature page portion generation module 5222. This step may be done based on the first variable that is provided thereto.

[0194] Following step 1703 is step 1704. Step 1704 may involve generating a second portion of the current feature page dataset, by a first fusion module 52222 of the feature page portion generation module 5222. This step may be done based on any one or both:

[0195] • the first variable, and

[0196] • the first portion of the feature page dataset that was generated by the transformation module 52221 is step 1703.

[0197] Following step 1704 is step 1705. Step 1705 may involve generating a third portion of the current feature page dataset, by the transformation module 52221. This step may be done based on the second portion of the current feature map dataset generated by the first fusion module 52222 in step 1704.

[0198] Following step 1705 is step 1706. Step 1706 may involve generating a fourth portion of the current feature page dataset, by a second fusion module 52223 of the feature page portion generation module 5222. This step may be done based on any one or a combination of

[0199] • the first variable,

[0200] • the first portion of the current feature page dataset that was previously generated by the transformation module 52221 in step 1703, and

[0201] • the third portion of the current feature page dataset that was generated by the transformation module 52221 in step 1705.

[0202] Following step 1706 is step 1707. Step 1707 may involve generating the current feature page dataset, by the feature page generation module 5223. This step may be done based on any one or a combination of

[0203] • the first portion of the current feature page dataset that was generated by the transformation module 52221 in step 1703,

[0204] • the second portion of the current feature page dataset that was generated by the first fusion module 52222 in step 1704,

[0205] • the third portion of the current feature page dataset that was generated by the transformation module 52221 in step 1705, and

[0206] • the fourth portion of the current feature page dataset that was generated by the second fusion module 52223 in step 1706.

[0207] Following step 1707 is step 1708. Step 1708 is a decision step that involves (i) determining that all feature page datasets have been generated, or (ii) determining that there are remaining feature page datasets yet to be generated. Should the former be determined, step 1708 proceeds to step 1711. Should the latter be determined, step 1708 proceeds to step 1709.

[0208] In step 1709, since it was determined that there are remaining feature pages datasets yet to be generated, the step of setting the fourth portion of the current feature page dataset as the first variable, is performed by the feature page portion generation module 5222. This is so that the fourth portion of the current feature page dataset is used for generating feature page portions of a subsequent feature page dataset.

[0209] Following this, step 1709 proceeds to step 1710. Step 1710 involves setting the subsequent feature page dataset as the current feature page dataset to be generated, by the feature page portion generation module 5222 and the feature page generation module 5223. With this, step 1710 may return to step 1703 and repeat therefrom.

[0210] Finally, in step 1711, since it was determined that all feature page datasets have been generated, the step of providing all of the generated feature page datasets to the cumulative data construction module 5224, is performed by the feature page generation module 5223.

[0211] FIG. 18 illustrates a flowchart describing example steps that may relate to generating a first portion of one feature page dataset. The steps of FIG. 18 may be performed by the transformation module 52221. The steps in FIG. 18 may be regarded as sub-steps of step 1703 of FIG. 17. FIG. 19 illustratively depicts the steps of FIG. 18 and these steps may be described with reference therewith.

[0212] The steps of FIG. 18 may begin with step 1801, which may involve loading the first variable by the transformation module 52221. The first variable may be either one of the feature map dataset, or a fourth portion of a previous feature page dataset.

[0213] Following step 1801 is step 1802. Step 1802 is a decision step that involves (i) determining that addition of kernel computation is required, or (ii) determining that addition of kernel computation is redundant. Should the former be determined, step 1802 proceeds to step 1803. Else, should the latter be determined, step 1802 proceeds to step 1804.

[0214] In step 1803, since it was determined that addition of kernel computation is required, the step of executing kernel computation for the first variable is performed by the transformation module 52221. With this, a kernel-computed first variable is produced.

[0215] Step 1804 may follow from step 1802 or step 1803. Step 1804 is yet another decision step that involves (i) determining that generation of a first row-based feature change map dataset is required, or (ii) generation of the first row-based feature change map dataset is redundant. Should the former be determined, step 1804 proceeds to step 1805. Else, should the latter be determined, step 1804 proceeds to step 1806.

[0216] In step 1805, since it was determined that generation of a first row -based feature change map dataset is required, the step of generating the first row -based feature change map dataset is performed by the transformation module 52221. With that, the first row-based feature change map dataset is produced.

[0217] Following step 1804 or step 1805 is step 1806. Step 1806 is yet another decision step that involves (i) determining that generation of a first column-based feature change map dataset is required, or (ii) generation of a first column-based feature change map dataset is redundant. Should the former be determined, step 1806 proceeds to step 1807. Else, should the latter be determined, step 1806 proceeds to step 1808.

[0218] In step 1807, since it was determined that generation of a first column-based feature change map dataset is required, the step of generating the first column-based feature change map dataset is performed by the transformation module 52221. With that, the first column-based feature change map dataset is produced.

[0219] Finally, step 1808 may follow from step 1806 or step 1807. Step 1808 involves producing the first portion of the feature page dataset by the transformation module 52221. In particular, the first portion of the feature page dataset may be produced by horizontally concatenating the kernel-computed first variable, the first row-based feature change map dataset, and the first column-based feature change map dataset. With that, as shown in FIG. 19, the first portion of the feature page dataset may have shared row headings that each correspond to point indexes, while having combined column headings and corresponding entries from the kernel-computed first variable, the first row-based feature change map dataset, and the first column-based feature change map dataset.

[0220] FIG. 20 illustrates a flowchart describing example steps that may relate to executing kernel computation for the first variable. The steps of FIG. 20 may be performed by the transformation module 52221. The steps in FIG. 20 may be regarded as sub-steps of step 1803 of FIG. 18. FIG. 21 illustratively depicts the steps of FIG. 20 and these steps may be described with reference therewith.

[0221] The steps of FIG. 20 may begin with step 2001, which may involve loading the first variable, by the transformation module 52221. The first variable may be either one of the feature map dataset, or a fourth portion of a previous feature page dataset.

[0222] Following step 2001 is step 2002. Step 2002 involves performing convolution computation based on a kernel, by the transformation module 52221, in which the kernel may be user-defined or pre-calculated. In particular, the kernel may be in the form of a matrix having an array of values as shown in FIG. 21. Furthermore, the convolutional operation may be performed between numerical entries within a tabular representation of the first variable and the kernel.

[0223] Following step 2002 is step 2003. Step 2003 is a decision step that involves (i) determining that adding activation is required, or (ii) determining that adding activation is redundant. Should the former be determined, step 2003 proceeds to step 2004. Else, should the latter be determined, step 2003 proceeds to step 2005.

[0224] In step 2004, since it was determined that adding activation is required, the step of performing activation computation is performed by the transformation module 52221. In particular, the data of the first variable may be projected to any one of a linear function, a non-linear function, or a user-defined function. These functions may be previously defined in the generated configuration setting.

[0225] Step 2005 may follow from step 2003 or step 2004. Step 2005 is a decision step that involves (i) determining that adding normalisation is required, or (ii) determining that adding normalisation is redundant. Should the former be determined, step 2005 proceeds to step 2006. Else, should the latter be determined, step 2005 proceeds to step 2007.

[0226] In step 2006, since it was determined that adding normalisation is required, the step of performing normalisation computation is performed by the transformation module 52221. In particular, the data may be normalised for it to be within a user-defined range, for example, -1 to 1. This range may be previously defined in the generated configuration setting.

[0227] Finally, step 2007 may follow from step 2005 or step 2006. Step 2007 involves producing a kernel-computed first variable by the transformation module 52221. The kernel-computed first variable may be in represented in the form of a table with a plurality of entries, in which its row headers correspond to a point index, and its column headers correspond to a feature index of the point index.

[0228] FIG. 22 illustrates a flowchart describing example steps that may relate to generating a first row -based feature change map dataset. The steps of FIG. 22 may be performed by the transformation module 52221. The steps in FIG. 22 may be regarded as substeps of step 1805 of FIG. 18. FIG. 23 illustratively depicts the steps of FIG. 22 and these steps may be described with reference therewith.

[0229] The steps of FIG. 22 may begin with step 2201, which may involve loading the first variable, by the transformation module 52221. The first variable may be either one of the feature map dataset, or a fourth portion of a previous feature page dataset.

[0230] Following step 2201 is step 2202. Step 2202 involves duplicating a first row from the first variable for it to become a first row of the first row-based feature change map dataset, by the transformation module 52221.

[0231] Following step 2202 is step 2203. Step 2203 involves selecting one unique row from the first variable, by the transformation module 52221.

[0232] Following step 2203 is step 2204. Step 2204 is a decision step that involves (i) determining that the unique row that was selected is the first row, or (ii) determining that the unique row that was selected is a row other than the first row. Should the former be determined, step 2204 proceeds to step 2205. Else, should the latter be determined, step 2204 proceeds to step 2206.

[0233] In step 2205, since it was determined that the unique row that was selected is the first row of the first variable, the selected row is skipped. With that, step 2205 returns to step 2203 and continues therefrom.

[0234] In step 2206, since it was determined that the unique row that was selected is a row other than the first row, the step of performing a subtraction between (i) a value within (i) each column of the selected row and (ii) a value within each column of a row of the first variable that is prior to the selected row, is performed by the transformation module 52221. With that, one or more first subtraction results are obtained.

[0235] Following step 2206 is step 2207. Step 2207 involves generating a new row entry for the first row -based feature change map dataset, wherein each column of the row entry has a corresponding first subtraction result, by the transformation module 52221.

[0236] Following step 2207 is step 2208. Step 2208 is a decision step that involves (i) determining that all unique rows have been selected, or (ii) determining that there are still unique rows that remain unselected. Should the former be determined, step 2208 proceeds to step 2209. Else, should the latter be determined, step 2208 returns to step 2203 and repeats therefrom.

[0237] Finally, in step 2209, since it was determined that all unique rows have been selected, the step of producing the first row-based feature map dataset, is performed by the transformation module 52221. The first row -based feature map dataset may be represented in the form of a table with a plurality of entries, in which its row headers correspond to a point index, and its column headers correspond to a feature index of the point index.

[0238] FIG. 24 illustrates a flowchart describing example steps that may relate to generating a first column-based feature change map dataset. The steps of FIG. 24 may be performed by the transformation module 52221. The steps in FIG. 24 may be regarded as sub-steps of step 1807 of FIG. 18. FIG. 25 illustratively depicts the steps of FIG. 24 and these steps may be described with reference therewith.

[0239] The steps of FIG. 24 may begin with step 2401, which may involve loading the first variable, by the transformation module 52221. The first variable may be either one of the feature map dataset, or a fourth portion of a previous feature page dataset.

[0240] Following step 2401 is step 2402. Step 2402 involves duplicating a first column from the first variable for it to become a first column of the first column-based feature change map dataset, by the transformation module 52221.

[0241] Following step 2402 is step 2403. Step 2403 involves selecting one unique column from the first variable, by the transformation module 52221.

[0242] Following step 2403 is step 2404. Step 2404 is a decision step that involves (i) determining that the unique column that was selected is the first column, or (ii) the unique column that was selected is a column other than the first column. Should the former be determined, step 2404 proceeds to step 2405. Else, should the latter be determined, step 2404 proceeds to step 2406.

[0243] In step 2405, since it was determined that the unique column that was selected is the first column, the selected column is skipped. With that, step 2405 returns to step 2403 and continues therefrom.

[0244] In step 2406, since it was determined that the unique column that was selected is a column other than the first column of the first variable, the step of performing a subtraction between (i) a value within each row of the selected column and, (ii) a value within each row of a column of the first variable that is prior to the selected column, is performed by the transformation module 51221. With that, one or more second subtraction results are obtained.

[0245] Following step 2406 is step 2407. Step 2407 involves generating a new column entry for the column-based feature change map dataset, wherein each row of the column entry has a corresponding second subtraction result, by the transformation module 51221.

[0246] Following step 2407 is step 2408. Step 2408 is a decision step that involves (i) determining that all unique columns have been selected, or (ii) determining that there are still unique columns that remain unselected. Should the former be determined, step 2408 proceeds to step 2409. Else, should the latter be determined, step 2408 returns to step 2403 and repeats therefrom.

[0247] Finally, in step 2409, since it was determined that all unique columns have been selected, the step of producing the first column-based feature map dataset, is performed by the transformation module 52221. The first column-based feature map dataset may be represented in the form of a table with a plurality of entries, in which its row headers correspond to a point index, and its column headers correspond to a feature index of the point index.

[0248] FIG. 26 illustrates a flowchart describing example steps that may relate to generating a second portion of one feature page dataset. The steps of FIG. 26 may be performed by the first fusion module 52222. The steps in FIG. 26 may be regarded as sub-steps of step 1704 of FIG. 17. FIG. 27 illustratively depicts the steps of FIG. 28 and these steps may be described with reference therewith.

[0249] The steps of FIG. 26 may begin with step 2601, which may involve loading the first variable, by the first fusion module 52222. The first variable may be either one of the feature map dataset, or a fourth portion of a previous feature page dataset.

[0250] Following step 2601 is step 2602. Step 2602 involves loading the first portion of the feature page dataset, which was generated by the transformation module 52221, by the first fusion module 52222.

[0251] Following step 2602 is step 2603. Step 2603 involves combining both the first variable and the first portion of the feature page dataset, via an add operation, to produce one or more first addition results, by the first fusion module 52222. For example, assuming that the first variable and the first portion of the feature page dataset have a dimension of the same, a corresponding entry in the first variable is added with a corresponding entry in the first portion of the feature page dataset as shown in FIG. 27.

[0252] Following step 2603 is step 2604. Step 2604 involves constructing a second portion of the feature page dataset from the first addition results by the first fusion module 52222. With that, the second portion of the feature page dataset is produced by the first fusion module 52222. As shown in FIG. 27, the second portion of the feature page dataset may be represented as atable, in which its row headers each correspond to a point index, and its column headers are corresponding data of the point index.

[0253] FIG. 28 illustrates a flowchart describing example steps that may relate to generating a third portion of one feature page dataset. The steps of FIG. 28 may be performed by the transformation module 52221. The steps in FIG. 28 may be regarded as sub-steps of step 1705 of FIG. 17. FIG. 29 illustratively depicts the steps of FIG. 28 and these steps may be described with reference therewith.

[0254] The steps of FIG. 28 may begin with step 2801, which may involve loading the second portion of the feature page dataset by the transformation module 52221.

[0255] Following step 2801 is step 2802. Step 2802 is a decision step that involves (i) determining that addition of kernel computation is required, or (ii) determining that addition of kernel computation is redundant. Should the former be determined, step 2802 proceeds to step 2803. Else, should the latter be determined, step 2802 proceeds to step 2804.

[0256] In step 2803, since it was determined that addition of kernel computation is required, the step of executing kernel computation for the second portion of the feature page dataset, is performed by the transformation module 52221. With this, a kernel- computed second portion of the feature page dataset is produced.

[0257] Step 2804 may follow from step 2802 or step 2803. Step 2803 is yet another decision step that involves (i) determining that generation of a second row -based feature change map dataset is required, or (ii) generation of the second row -based feature change map dataset is redundant. Should the former be determined, step 2804 proceeds to step 2805. Else, should the latter be determined, step 2804 proceeds to step 2806.

[0258] In step 2805, since it was determined that generation of a second row-based feature change map dataset is required, the step of generating the second row-based feature change map dataset is performed by the transformation module 52221. With that, the second row -based feature change map dataset is produced.

[0259] Following step 2804 or step 2805 is step 2806. Step 2806 is yet another decision step that involves (i) determining that generation of a second column-based feature change map dataset is required, or (ii) determining that generation of a second column-based feature change map dataset is redundant. Should the former be determined, step 2806 proceeds to step 2807. Else, should the latter be determined, step 2806 proceeds to step 2808.

[0260] In step 2807, since it was determined that generation of a second column-based feature change map dataset is required, the step of generating the second column-based feature change map dataset is performed by the transformation module 52221. With that, the second column-based feature change map dataset is produced.

[0261] Finally, step 2808 may follow from step 2806 or step 2807. Step 2808 involves producing the third portion of the feature page dataset by the transformation module 52221. In particular, the third portion of the feature page dataset may be produced by horizontally concatenating the kernel -computed second portion of the feature page dataset, the second row-based feature change map dataset, and the second columnbased feature change map dataset. With that, as shown in FIG. 29, the third portion of the feature page dataset may have shared row headings that each correspond to point indexes, while having combined column headings and corresponding entries from the kernel-computed second portion of the feature page dataset, the second row-based feature change map dataset, and the second column-based feature change map dataset.

[0262] FIG. 30 illustrates a flowchart describing example steps that may relate to executing kernel computation for the second portion of the feature page dataset. The steps of FIG. 30 may be performed by the transformation module 52221. The steps in FIG. 30 may be regarded as sub-steps of step 2803 of FIG. 28. FIG. 31 illustratively depicts the steps of FIG. 30 and these steps may be described with reference therewith.

[0263] The steps of FIG. 30 may begin with step 3001, which may involve loading the second portion of the feature page dataset, by the transformation module 52221.

[0264] Following step 3001 is step 3002. Step 3002 involves performing convolution computation based on a kernel, by the transformation module 52221, in which the kernel may be user-defined or pre-calculated. In particular, the kernel may be in the form of a matrix having an array of values as shown in FIG. 31. Furthermore, the convolutional operation may be performed between numerical entries within a tabular representation of the second portion of the feature page dataset and the kernel.

[0265] Following step 3002 is step 3003. Step 3003 is a decision step that involves (i) determining that adding activation is required, or (ii) determining that adding activation is redundant. Should the former be determined, step 3003 proceeds to step 3004. Else, should the latter be determined, step 3003 proceeds to step 3005.

[0266] In step 3004, since it was determined that adding activation is required, the step of performing activation computation upon the second portion of the feature page dataset is performed by the transformation module 52221. In particular, the data of the second portion of the feature page dataset may be projected to any one of a linear function, a non-linear function, or a user-defined function. These functions may be previously defined in the generated configuration setting.

[0267] Step 3005 may follow from step 3003 or step 3004. Step 3005 is a decision step that involves (i) determining that adding normalisation is required, or (ii) determining that adding normalisation is redundant. Should the former be determined, step 3005 proceeds to step 3006. Else, should the latter be determined, step 3005 proceeds to step 3007.

[0268] In step 3006, since it was determined that adding normalisation is required, the step of performing normalisation computation is performed upon the second portion of the feature page dataset by the transformation module 52221. In particular, the data may be normalised for it to be within a user-defined range, for example, -1 to 1. This range may be previously defined in the generated configuration setting.

[0269] Finally, step 3007 may follow from step 3005 or step 3006. Step 3007 involves producing a kernel -computed second portion of the feature page dataset by the transformation module 52221. The kernel-computed second portion of the feature page dataset may be represented in the form of a table with a plurality of entries, in which its row headers correspond to a point index, and its column headers correspond to a feature index of the point index.

[0270] FIG. 32 illustrates a flowchart describing example steps that may relate to generating a second row-based feature change map dataset. The steps of FIG. 32 may be performed by the transformation module 52221. The steps in FIG. 32 may be regarded as sub-steps of step 2805 of FIG. 28. FIG. 33 illustratively depicts the steps of FIG. 32 and these steps may be described with reference therewith.

[0271] The steps of FIG. 32 may begin with step 3201, which may involve loading the second portion of the feature page dataset, by the transformation module 52221.

[0272] Following step 3201 is step 3202. Step 3202 involves duplicating a first row from the second portion of the feature page dataset for it to become a first row of the second row- based feature change map dataset, by the transformation module 52221.

[0273] Following step 3202 is step 3203. Step 3203 involves selecting one unique row from the second portion of the feature page dataset, by the transformation module 51221.

[0274] Following step 3203 is step 3204. Step 3204 is a decision step that involves (i) determining that the unique row that was selected is the first row, or (ii) determining that the unique row that was selected is a row other than the first row. Should the former be determined, step 3204 proceeds to step 3205. Else, should the latter be determined, step 3204 proceeds to step 3206.

[0275] In step 3205, since it was determined that the unique row that was selected is the first row of the second portion of the feature page dataset, the selected row is skipped. With that, step 3205 returns to step 3203 and repeats therefrom.

[0276] In step 3206, since it was determined that the unique row that was selected is a row other than the first row, the step of performing a subtraction between (i) a value within each column of the selected row and (ii) a value within each column of a row of the second portion of the feature page dataset that is prior to the selected row, is performed by the transformation module 52221. With that, one or more third subtraction results are obtained.

[0277] Following step 3206 is step 3207. Step 3207 involves generating a new row entry for the second row-based feature change map dataset, wherein each column of the row entry has a corresponding third subtraction result, by the transformation module 52221.

[0278] Following step 3207 is step 3208. Step 3208 is a decision step that involves (i) determining that all unique rows have been selected, or (ii) determining that there are still unique rows that remain unselected. Should the former be determined, step 3208 proceeds to step 3209. Else, should the latter be determined, step 3208 returns to step 3203 and repeats therefrom.

[0279] Finally, in step 3209, since it was determined that all unique rows have been selected, the step of producing the second row-based feature map dataset, is performed by the transformation module 52221. The second row -based feature map dataset may be in represented in the form of a table with a plurality of entries, in which its row headers correspond to a point index, and its column headers correspond to a feature index of the point index.

[0280] FIG. 34 illustrates a flowchart describing example steps that may relate to generating a second column-based feature change map dataset. The steps of FIG. 34 may be performed by the transformation module 52221. The steps in FIG. 34 may be regarded as sub-steps of step 2807 of FIG. 28. FIG. 35 illustratively depicts the steps of FIG. 34 and these steps may be described with reference therewith.

[0281] The steps of FIG. 34 may begin with step 3401, which may involve loading the second portion of the feature page dataset, by the transformation module 52221.

[0282] Following step 3401 is step 3402. Step 3402 involves duplicating a first column from the second portion of the feature page dataset for it to become a column of a second column-based feature change map dataset, by the transformation module 52221.

[0283] Following step 3402 is step 3403. Step 3403 involves selecting one unique column from the second portion of the feature page dataset, by the transformation module 52221.

[0284] Following step 3403 is step 3404. Step 3404 is a decision step that involves (i) determining that the unique column that was selected is the first column, or (ii) the unique column that was selected is a column other than the first column. Should the former be determined, step 3404 proceeds to step 3405. Else, should the latter be determined, step 3404 proceeds to step 3406.

[0285] In step 3405, since it was determined that the unique column that was selected is the first column, the selected column is skipped. With that, step 3405 returns to step 3403 and repeats therefrom.

[0286] In step 3406, since it was determined that the unique column that was selected is a column other than the first column of the second portion of the feature page dataset, the step of performing a subtraction between (i) a value within each row of the selected column and (ii) a value within each row of a column of the second portion of the feature page dataset that is prior to the selected column, is performed by the transformation module 52221. With this, one or more fourth subtraction results are obtained.

[0287] Following step 3406 is step 3407. Step 3407 involves generating a new column entry for the second column-based feature change map dataset, wherein each row of the column entry has a corresponding fourth subtraction result, by the transformation module 52221.

[0288] Following step 3407 is step 3408. Step 3408 is a decision step that involves (i) determining that all unique columns have been selected, or (ii) determining that there are still unique columns that remain unselected. Should the former be determined, step 3408 proceeds to step 3409. Else, should the latter be determined, step 3408 returns to step 3403 and repeats therefrom.

[0289] Finally, in step 3409, since it was determined that all unique columns have been selected, the step of producing the second column-based feature map dataset, is performed by the transformation module 52221. The second column -based feature map dataset may be in represented in the form of a table with a plurality of entries, in which its row headers correspond to a point index, and its column headers correspond to a feature index of the point index.

[0290] FIG. 36 illustrates a flowchart describing the example steps that may relate to generating a fourth portion of one feature page dataset. The steps of FIG. 36 may be performed by a second fusion module 52223. The steps in FIG. 36 may be regarded as sub-steps of step 1706 of FIG. 17. FIG. 37 illustratively depicts the steps of FIG. 36 and these steps may be described with reference therewith.

[0291] The steps of FIG. 36 may begin with step 3601, which may involve loading the first variable, by the second fusion module 52223. The first variable may be either one of the feature map dataset, or a fourth portion of a previous feature page dataset.

[0292] Following step 3601 is step 3602. Step 3602 may involve loading the first portion of the feature page dataset, which was generated by the transformation module 52221, by the second fusion module 52223.

[0293] Following step 3602 is step 3603. Step 3603 may involve loading the third portion of the feature page dataset, which was generated by the transformation module 52221, by the second fusion module 52223.

[0294] Following step 3603 is step 3604. Step 3604 may involve combining the first variable, the first portion of the feature page dataset, and the third portion of the feature page dataset, via an addition operation, to produce one or more second addition results, by the second fusion module 52223. For example, assuming that the first variable, the first portion of the feature page dataset, and the third portion of the feature page dataset, have a dimension of the same, a corresponding entry in the first variable, the first portion of the feature page dataset, and the third portion of the feature page dataset, are added together as shown in FIG. 37.

[0295] Following step 3604 is step 3605. Step 3605 may involve constructing a fourth portion of the feature page dataset from the second addition results, by the second fusion module 52223. With that, the fourth portion of the feature page dataset is produced by the second fusion module 52223. As shown in FIG. 37, the fourth portion of the feature page dataset may be represented as a table, in which its row headers each correspond to a point index, and its column headers are corresponding data of the point index.

[0296] With this, it should be noted that computational modules of the feature page portion generation module 5222, which includes the transformation module 52221, the fusion modules 52222, 52223, and other ancillary modules related to row-change feature extraction and column-change feature extraction, may each processes data in an independent manner. Their outputs are each subsequently combined by the feature page generation module 5223 to produce the corresponding feature page dataset.

[0297] In particular, each module of the feature page portion generation module 5222 is designed to extract finer or more specific features from a previous layer, wherein the previous layer may be the feature map dataset, one previous feature page portion dataset of one previous feature page dataset, or one previous feature page portion dataset of one current feature page dataset.

[0298] In particular, regarding the transformation module 52221 of the feature page portion generation module 5222, a user may configure its kernel computations to be, by way of example, in the form of a straight-line kernel to extract straight-line features, (e.g. one of the kernel examples as shown in FIG. 14), or the like.

[0299] In particular, the modules of the feature page portion generation module 5222 that relate to row-change feature extraction and column-change feature extraction are tasked with extracting features that represent changes between rows and columns in the previous layer.

[0300] In particular, the fusion modules 52222, 52223 of the feature page portion generation module 5222 allows for two or more feature page portions to combine to emphasise specific features.

[0301] By way of example, consider an example input layer that has information that relate to a square and a circle. As this example input layer is provided to the feature page portion generation module 5222, the transformation module 52221 may apply a kernel computation to extract a first layer (i.e. the first feature page portion) that highlights or emphasises vertical straight-line regions of these shapes. In particular, vertical edges of the square, which are vertical straight-line regions, will be highlighted. Since a circle does not have vertical straight-line regions, it shall not be highlighted.

[0302] Subsequently, the example input layer and the extracted first layer (i.e. the first feature page portion) that relates to the vertical straight-line regions are fused by the first fusion module 52222. With this, a second layer is generated (i.e. the second feature page portion), in which both the square and the circle are still visible (i.e. their information are retained), with vertical edges of the square being highlighted or emphasised with a higher intensity.

[0303] Subsequently, the transformation module 52221 may apply a kernel computation upon the second layer (i.e. the second feature page portion) to extract a third layer (i.e. the third feature page portion) that highlights or emphasises horizontal straight-line regions within the shapes. In particular, horizontal edges of the square, which are horizontal straight-line regions, will be highlighted. Since a circle does not have horizontal straight-line regions, it shall not be highlighted.

[0304] Subsequently, the example input layer, the extracted first layer (i.e. the first feature page portion) that relates to the vertical straight-line regions, and the extracted third layer (i.e. the third feature page portion) that relates to the horizontal straight-line regions, are fused by the second fusion module 52223. With this, a fourth layer is generated (i.e. the fourth feature page portion), whereby the vertical edges and horizontal edges of the square are highlighted or emphasised with a higher intensity.

[0305] These extracted layers may be considered critical features of an object under inspection, which are to be provided to machine learning and / or deep learning models forthem to learn and be trained to make accurate predictions. By performing an approach that involves sequentially combining all these features, the machine learning and / or deep learning models shall gain a clearer understanding of how features are extracted at each step when they operate based on their corresponding algorithms.

[0306] FIG. 38 illustrates a flowchart describing the example steps that may relate to generating one feature page dataset. The steps of FIG. 38 may be performed by the feature page generation module 5223. The steps in FIG. 38 may be regarded as substeps of step 1707 of FIG. 17. FIG. 39 illustratively depicts the steps of FIG. 38 and these steps may be described with reference therewith.

[0307] The steps of FIG. 38 may begin with step 3801. Step 3801 may involve loading any one or a combination of (i) the first portion of a current feature page dataset, (ii) the second portion of the current feature page dataset, (iii) the third portion of the current feature page dataset, and (iv) the fourth portion of the current feature page dataset, from the feature page portion generation module 5222, by the feature page generation module 5223.

[0308] Following step 3801 is step 3802. Step 3802 may involve stacking all the datasets in a horizontal manner, according to their point index, by the feature page generation module 5223.

[0309] With that, as per step 3803 the feature page dataset is produced by the feature page generation module 5223. As shown in FIG. 39, since the feature page dataset is produced from the horizontal stacking as previously described, the feature page dataset may have shared row headings that each correspond to a point index, while having horizontally stacked column headings and corresponding entries from the first portion of the feature page dataset, the second portion of the feature page dataset, the third portion of the feature page dataset, and the fourth portion of the feature page dataset.

[0310] FIG. 40 illustrates a flowchart describing the example steps that may relate to constructing the cumulative dataset. The steps of FIG. 40 may be performed by the cumulative data construction module 5224. The steps in FIG. 40 may be regarded as sub-steps of step 1004 of FIG. 10. FIG. 41 illustratively depicts the steps of FIG. 40 and these steps may be described with reference therewith.

[0311] The steps of FIG. 40 may begin with step 4001. Step 4001 may involve loading any one or a combination of (i) the pre-processed point cloud dataset, (ii) the feature map dataset generated by the feature map generation module 5221, and (iii) the feature page datasets generated by the feature page generation module 5223, by the cumulative data construction module 5224.

[0312] Following step 4001 is step 4002. Step 4002 may involve stacking all the datasets in a horizontal manner, according to their point index, by the cumulative data generation module 5224.

[0313] Finally, following step 4002 is step 4003. Step 4003 may involve producing the cumulative dataset, by the cumulative data construction module 5224. As shown in FIG. 41, since the cumulative dataset is produced from the horizontal stacking as previously described, the cumulative dataset may have shared row headings that each correspond to a point index, while having horizontally stacked column headings and corresponding entries from the pre-processed point cloud dataset, the feature map dataset, and the feature page datasets. With that, the cumulative dataset may be said to represent captured information of the object in a historical manner from beginning to end.

[0314] FIG. 42 illustrates a flowchart describing the example steps that may be further included in the method and system for data generation. In particular, the steps of FIG. 42 may relate to generating a trained object inspection model based on the cumulative datasets that were generated by the cumulative dataset generation module 5220. The steps of FIG. 42 are illustratively depicted in FIG. 43.

[0315] The steps of FIG. 42 may begin with step 4201, which may involve loading one or more cumulative datasets from the storage unit 5300, by the model training module 5240.

[0316] Following step 4201 is step 4202. Step 4202 involves dividing the cumulative datasets into at a training dataset and a validation dataset based on a pre -determined ratio, by the model training module 5240.

[0317] Following step 4202 is step 4203. Step 4203 involves arranging the cumulative datasets, within the training dataset and the validation dataset, into one or more distinct categories, by the model training module 5240. The categories may relate to categories of the cumulative dataset along any one or a combination of an x-axis, a y-axis, and a z-axis, or more specifically, the first three columns of the cumulative datasets.

[0318] Following step 4203 is step 4204. Step 4204 involves training one or more object inspection models based on the training dataset and the validation dataset, by the model training module 5240.

[0319] Following step 4204 is step 4205. Step 4205 involves producing one or more trained object inspection models, by the model training module 5240.

[0320] Following step 4205 is step 4206. Step 4206 involves validating an accuracy of each trained object inspection model, by the model training module 5240.

[0321] Following step 4206 is step 4207. Step 4207 involves selecting a trained object inspection model that has a validation accuracy that is higher than the validation accuracies of the other trained object inspection models, by the model training module 5240.

[0322] Finally, following step 4207 is step 4208. Step 4208 may involve storing the trained object inspection model that was selected, together with its related information, within the storage unit 5300, by the model training module 5240.

[0323] FIG. 44 illustrates a flowchart describing the example steps that may be further included in the method and system for data generation. In particular, the steps of FIG. 44 may relate to performing object inspection. The steps of FIG. 44 are illustratively depicted in FIG. 45.

[0324] The steps of FIG. 44 may begin with step 4401, which may involve providing at least one object to at least one capturing unit 5100.

[0325] Following step 4401 is step 4402. Step 4402 may involve obtaining at least one raw point cloud dataset of the object, by the capturing unit 5100.

[0326] Following step 4402 is step 4403. Step 4403 may involve receiving the raw point cloud dataset of the object, by at least one processing unit 5200.

[0327] Following step 4403 is step 4404. Step 4404 may involve pre-processing the raw point cloud dataset of the object, by the pre-processing unit 5210 operated by the processing unit 5200. The further sub-steps of step 4404 may be similar to the descriptions of FIGS 5 - 9.

[0328] Following step 4404 is step 4405. Step 4405 may involve generating one or more cumulative datasets from the point cloud dataset, by the cumulative dataset generation module 5220 operated by the processing unit 5200. The further sub-steps of step 4405 may be similar to the descriptions of FIGS 10 - 41.

[0329] Following step 4405 is step 4406. Step 4406 may involve loading a trained object inspection model and its related information from the storage unit 5300, by the object inspection module 5250 operated by the processing unit 5200.

[0330] Following step 4406 is step 4407. Step 4407 may involve performing inspection of the object, by the object inspection module 5250 operated by the processing unit 5200, in which the cumulative dataset of the object is provided to the trained object inspection model as its input.

[0331] Following step 4407 is step 4408. Step 4408 may involve providing inspection results, by the object inspection module 5250. The inspection results, may be any one or a combination of the identification results and the classification results, and may be outputs from the trained object inspection model. In particular, these results may be related to identified or classified attributes of the object, which may include defects of the object.

[0332] Finally, following step 4408 is step 4409. Step 4409 involves any one or a combination of storing the inspection results in the storage unit 5300 by the object inspection module 5250, and displaying the inspection results on the human-machine interface unit 5400 via the user interface module 5260.

[0333] An experiment was carried out to validate the method and system of the present invention, and the results are as shown in FIGS 46 - 47. Two machine learning models of the same, and one or more objects were prepared. A first machine learning model was trained based on the cumulative datasets of the objects that were generated by the method and system of the present invention (i.e. “with Cumulative 3D point cloud data second machine learning model was trained based on raw point cloud datasets directly obtained from the objects without using the method and system of the present invention (i.e. “without Cumulative 3D point cloud data formation (C3DF) ”).

[0334] As shown in FIG. 46, during the training of the machine learning models, the first machine learning model exhibited training loss that dropped faster and is lower compared to the second machine learning model, which indicates that the first machine learning model had learnt features of the objects better than the second machine learning model. Furthermore, as shown in FIG. 47, during the training of the machine learning models, the first machine learning model reached a higher accuracy within a smaller number of epochs compared to the second machine learning model. The sequential approach provided by the present invention, as demonstrated in the experiment, allows the machine learning and / or deep learning models to learn features from objects in a more efficient manner, resulting in faster learning and improved predictive performance.

[0335] With this, a method and system for data generation have been provided and described. Whilst the method and system of the present invention had been broadly described to be applicable to any form of object for machine learning applications, it is preferred that the present invention is involved in a three-dimensional machine learning-based wire bond inspection workflow for addressing its related challenges, so that wire bonds may be inspected in a more accurate manner from its raw point cloud datasets.

[0336] The present disclosure includes as contained in the appended claims, as well as that of the foregoing description. Although this invention has been described in its preferred form with a degree of particularity, it is understood that the present disclosure of the preferred form has been made only by way of example and that numerous changes in the details of construction and the combination and arrangements of parts may be resorted to without departing from the scope of the invention.

Claims

CLAIMS1. A method for data generation, characterised in that the method comprises the steps of obtaining at least one point cloud dataset of an object, by at least one capturing unit (5100); receiving the point cloud dataset of the object, by at least one processing unit (5200); pre-processing the point cloud dataset of the object to produce a pre-processed point cloud dataset, by the processing unit (5200); and generating one or more cumulative datasets from the pre-processed point cloud dataset of the object, by the processing unit (5200); wherein the step of generating one or more cumulative datasets from the pre- processed point cloud dataset of the object, by the processing unit (5200) further comprises the steps of generating at least one feature map dataset from the pre-processed point cloud dataset of the object, by a feature map generation module (5221) operated by the processing unit (5200); generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, by a feature page portion generation module (5222) operated by the processing unit (5200), generating one or more feature page datasets from corresponding feature page portion datasets, by a feature page generation module (5223) operated by the processing unit (5200), and constructing the cumulative dataset from any one or a combination of the pre- processed point cloud dataset of the object, the feature map dataset, and the feature page datasets, by a cumulative data construction module (5224) operated by the processing unit (5200).

2. The method according to claim 1, further comprising the step of generating subsequent feature page portions for a subsequent feature page dataset, by the feature page portion generation module (5222), based on at least one feature page portion that was involved in the generation of a previous feature page dataset.

3. The method according to claim 1, wherein the step of pre-processing the point cloud dataset of the object, by a processing unit (5200), further comprises the steps of sampling the point cloud dataset of the object, by a sampling module (5211) of a pre-processing module (5210) for forming a sampled point cloud dataset; and filtering the sampled point cloud dataset of the object, by a filtering module (5212) of the pre-processing module (5210) for forming the pre-processed point cloud dataset.

4. The method according to claim 1, wherein the step of generating at least one feature map dataset from the pre-processed point cloud dataset of the object, by a feature map generation module (5221) operated by the processing unit (5200), further comprises the steps of executing kernel computation for the pre-processed point cloud dataset, by the feature map generation module (5221); and executing point grouping-based mean distance calculation, by the feature map generation module (5221).

5. The method according to claim 1, wherein the step of generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, by a feature page portion generation module (5222) operated by the processing unit (5200), further comprises the steps of performing generation of a first portion of a current feature page dataset, by a transformation module (52221) of the feature page portion generation module (5222), based on any one of the feature map dataset or from at least generated feature pageportion dataset, and performing generation of a second portion of the current feature page dataset, by a first fusion module (52222) of the feature page portion generation module (5222), based on any one or both the feature map dataset or at least one generated feature page portion dataset, and the first portion of one feature page dataset.

6. The method according to claim 5, wherein the step of generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset, by a feature page portion generation module (5222) operated by the processing unit (5200), further comprises the steps of performing generation of a third portion of the current feature page dataset, by a second fusion module (52223) of the feature page portion generation module (5223), based on the second portion of the current feature page dataset; and performing generation of a fourth portion of the current feature page dataset, by the transformation module (52221), based on any one or a combination of the feature map dataset or at least one generated feature page portion dataset; the first portion of the current feature page dataset; and the third portion of the current feature page dataset.

7. The method according to claim 6, wherein the step of generating one or more feature page datasets from corresponding feature page portion datasets, by a feature page generation module (5223) operated by the processing unit (5200), further comprises the step of constructing the current feature page dataset, by the feature page generation module (5223), based on any one or a combination of: the first portion of the current feature page dataset; the second portion of the current feature page dataset;the third portion of the current feature page dataset; and the fourth portion of the current feature page dataset.

8. The method according to claim 7, further comprising the steps of generating at least one trained object inspection model using the cumulative datasets, by the processing unit (5200), in which at least one model is selected therefrom; and storing the trained object inspection model that was selected in at least one storage unit (5300), together with its related information, by the processing unit (5200).

9. The method according to claim 7, further comprising the steps of loading a trained object inspection model and its related information from at least one storage unit (5300), by the processing unit (5200); performing inspection of the object, by the processing unit (5200), in which the cumulative dataset corresponding to the object is provided to the trained object inspection model as inputs; and providing one or more inspection results, by the processing unit (5200), in which the inspection results are outputs from the trained object inspection model.

10. A system (5000) for data generation, characterised in that the system comprises at least one capturing unit (5100) for obtaining at least one point cloud dataset of an object; and at least one processing unit (5200) for receiving the point cloud dataset of the object; pre-processing the point cloud dataset of the object to produce a pre- processed point cloud dataset; and generating one or more cumulative datasets from the pre-processed point cloud dataset of the object, wherein the processing unit (5200) operates a plurality of modules that includea feature map generation module (5221) for generating at least one feature map dataset from the pre-processed point cloud dataset of the object; a feature page portion generation module (5222) for generating one or more feature page portion datasets from the feature map dataset or from at least one generated feature page portion dataset; a feature page generation module (5223) for generating one or more feature page datasets from corresponding feature page portion datasets; and a cumulative data construction module (5224) for constructing the cumulative dataset from any one or a combination of the pre-processed point cloud dataset of the object, the feature map dataset, and the feature page datasets.

Images