Method for extracting 3D shape of object using multiple contactless sensors
The method integrates multiple non-contact sensors with neural network validation and weight-based data correction to overcome accuracy issues in 3D shape scanning, achieving precise 3D shape data for complex objects.
Patent Information
- Application Number
- KR1020250053068
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2025-04-23
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2045-04-23
AI Technical Summary
Existing non-contact sensor methods for 3D shape scanning face challenges in accuracy due to external lighting conditions, object reflectivity, and complexity of curved surfaces, making it difficult to precisely reproduce fine detailed shapes, especially in industrial applications.
A method using a plurality of non-contact sensors that involves generating positional relationship information, point cloud data, determining weight information, and utilizing an artificial neural network model to validate and correct sensor data, integrating data from multiple sensors to generate precise 3D shape data.
Enhances measurement accuracy by minimizing errors and noise from individual sensors, enabling stable generation of high-precision 3D shape data even for complex objects, improving overall scan quality and reliability.
Smart Images

Figure 112025046160705-PAT00004_ABST
Abstract
Description
Technology Field
[0001] The present disclosure relates to a method for extracting the 3D shape of an object using a plurality of non-contact sensors, and specifically, to a method for determining the weights of data generated by a plurality of non-contact sensors including a first sensor and a second sensor, and finally generating 3D shape data corresponding to the object to be measured by aligning point cloud data corresponding to at least a part of the object to be measured generated by each sensor data according to the positional relationship of the sensors. Background Technology
[0002] Technology for scanning the three-dimensional shape of objects is essential in various industrial fields, such as manufacturing, quality control, and robot vision. Generally, contact or non-contact sensors are used to acquire the 3D shape of an object.
[0003] Contact sensors acquire point data by having the sensor probe physically touch the surface of an object, which can provide relatively high precision. However, they have drawbacks, such as very long measurement times and difficulty in applying them to objects with complex or flexible shapes. In particular, there are significant practical limitations to deploying them in mass production processes or automated lines.
[0004] In contrast, the non-contact method has the advantage of rapidly acquiring the shape of an object and being able to flexibly integrate with automation systems. However, measurement accuracy in the non-contact method can be degraded by external lighting conditions, the reflectivity of the object, and the complexity of the curved surface; in particular, due to sensor limitations, it is difficult to precisely reproduce fine detailed shapes.
[0005] Therefore, there is a demand in the industry for a technology that can efficiently extract the 3D shape of an object using non-contact sensors while ensuring higher accuracy than existing methods.
[0006] Korean published patent KR 2025-0043223 discloses an intelligent anomaly detection method and device. The problem to be solved
[0007] The present disclosure is devised in response to the aforementioned background technology, and specifically relates to a method for determining the weights of data generated by a plurality of non-contact sensors including a first sensor and a second sensor, and for finally generating 3D shape data corresponding to a measurement target object by aligning point cloud data corresponding to at least a part of a measurement target object generated by each sensor data according to the positional relationship of the sensors.
[0008] Meanwhile, the technical problem that the present disclosure aims to solve is not limited to the technical problem mentioned above, and various technical problems may be included within the scope obvious to a person skilled in the art from the contents described below. means of solving the problem
[0009] According to an embodiment of the present disclosure for realizing the aforementioned objectives, a method is disclosed for performing on a computing device to extract a 3D shape of an object using a plurality of non-contact sensors. The method may include: generating positional relationship information between a first sensor and a second sensor based on the positions of a first sensor and a second sensor; generating first point cloud data corresponding to at least a part of a target object based on the first sensor; generating second point cloud data corresponding to at least a part of the target object based on the second sensor; determining weight information for the first point cloud data and the second point cloud data; and generating 3D shape data corresponding to the target object based on the first point cloud data, the second point cloud data, and the weight information.
[0010] In one embodiment, the method may further include the step of generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model prior to the step of determining weight information for the first point cloud data and the second point cloud data.
[0011] In one embodiment, the step of generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model may include: a step of detecting whether the first sensor is abnormal based on inputting the first point cloud data into the artificial neural network model; a step of detecting whether the second sensor is abnormal based on inputting the second point cloud data into the artificial neural network model; and a step of generating the validation result based on whether the first sensor is abnormal and whether the second sensor is abnormal.
[0012] In one embodiment, the artificial neural network model is an autoencoder-based artificial neural network model, and based on inputting the first point cloud data into the artificial neural network model, the step of detecting whether the first sensor is abnormal may include: the step of detecting whether the first sensor is abnormal based on the reconstruction error of the artificial neural network model for the first point cloud data.
[0013] In one embodiment, the step of detecting an anomaly in the first sensor based on inputting the first point cloud data into the artificial neural network model may further include the step of generating user notification information related to the anomaly in the first sensor when the restoration error of the first point cloud data of the first sensor is greater than or equal to a predetermined threshold.
[0014] In one embodiment, the step of determining weight information for the first point cloud data and the second point cloud data may include: determining location-based weight information based on the distance between the first sensor and the object to be measured and the distance between the second sensor and the object to be measured; determining data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data; and determining weight information based on combining the location-based weight information and the data-based weight information.
[0015] In one embodiment, the step of determining the location-based weighting information based on the distance between the first sensor and the object to be measured and the distance between the second sensor and the object to be measured may include: assigning a higher location-based weighting as the distance between the object to be measured and each sensor becomes closer.
[0016] In one embodiment, the step of determining data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data may include: assigning a higher data-based weight as the number of points included in each point cloud data increases.
[0017] In one embodiment, prior to the step of generating 3D shape data corresponding to the object to be measured based on the first point cloud data, the second point cloud data, and the weight information, the method may further include: a step of determining a first offset value corresponding to the first sensor; a step of determining a second offset value corresponding to the second sensor; a step of correcting the first point cloud data based on the first offset value; and a step of correcting the second point cloud data based on the second offset value.
[0018] In one embodiment, the step of determining a first offset value corresponding to the first sensor may include: a step of generating a measurement value corresponding to a first point of the object to be measured using the first sensor; a step of generating a measurement value corresponding to the first point using a third sensor which is a contact sensor; and a step of determining the first offset value based on comparing the measurement values generated using the first sensor and the third sensor corresponding to the first point.
[0019] According to an embodiment of the present disclosure for realizing the aforementioned task, a computer program stored in a computer-readable storage medium is disclosed, which enables a computing device to perform operations for extracting a 3D shape of an object using a plurality of non-contact sensors. The operations may include: an operation of generating positional relationship information between the first sensor and the second sensor based on the positions of the first sensor and the second sensor; an operation of generating first point cloud data corresponding to at least a part of the object to be measured based on the first sensor; an operation of generating second point cloud data corresponding to at least a part of the object to be measured based on the second sensor; an operation of determining weight information for the first point cloud data and the second point cloud data; and an operation of generating 3D shape data corresponding to the object to be measured based on the first point cloud data, the second point cloud data, and the weight information.
[0020] According to an embodiment of the present disclosure for realizing the aforementioned objectives, a computing device for extracting a 3D shape of an object using a plurality of non-contact sensors is disclosed. The computing device includes one or more processors and a memory, and the one or more processors may be configured to generate positional relationship information between the first sensor and the second sensor based on the positions of the first sensor and the second sensor, generate first point cloud data corresponding to at least a part of the object to be measured based on the first sensor, generate second point cloud data corresponding to at least a part of the object to be measured based on the second sensor, determine weight information for the first point cloud data and the second point cloud data, and generate 3D shape data corresponding to the object to be measured based on the first point cloud data, the second point cloud data, and the weight information. Effects of the invention
[0021] In the present disclosure, the 3D shape of an object can be measured using a plurality of non-contact sensors, and more precise 3D shape information can be generated by aligning point cloud data acquired from each sensor. In particular, by integrating data by assigning weights according to the reliability or characteristics of each sensor, the influence of errors or noise from specific sensors can be minimized and the overall measurement accuracy can be improved. Therefore, according to the present disclosure, even when using non-contact sensors, it is possible to stably obtain 3D shape data of high accuracy even for objects with complex shapes or various surface characteristics, and a significant effect occurs in that the overall scan quality and reliability are improved due to the multi-sensor configuration.
[0022] Meanwhile, the effects of the present disclosure are not limited to those mentioned above, and various effects may be included within the scope obvious to a person skilled in the art from the contents described below. Brief explanation of the drawing
[0023] FIG. 1 is a block diagram of a computing device for extracting the 3D shape of an object using a plurality of non-contact sensors according to one embodiment of the present disclosure. FIG. 2 illustrates an exemplary structure of an artificial intelligence-based model according to one embodiment of the present disclosure. FIG. 3 is a flowchart illustrating the process of extracting the 3D shape of an object using a plurality of non-contact sensors according to one embodiment of the present disclosure. FIG. 4 is a conceptual diagram illustrating the process of generating 3D shape data of an object using a plurality of non-contact sensors according to one embodiment of the present disclosure. FIG. 5 is a conceptual diagram illustrating the process of generating 3D shape data of an object based on synthesizing point cloud data generated by each sensor according to one embodiment of the present disclosure. FIG. 6 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented. Specific details for implementing the invention
[0024] The present disclosure discloses a method for determining the weights of data generated by a plurality of non-contact sensors including a first sensor and a second sensor, and for matching point cloud data corresponding to at least a part of the object to be measured generated by each sensor data according to the positional relationship of the sensors, thereby finally generating 3D shape data corresponding to the object to be measured.
[0025] Various embodiments are now described with reference to the drawings. In this specification, various descriptions are provided to provide an understanding of the present disclosure. However, it is evident that these embodiments can be practiced without such specific descriptions.
[0026] As used herein, terms such as “component,” “module,” “system,” etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be, but is not limited to, a procedure executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and / or an execution thread. A component may be localized within a single computer. A component may be distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from a component interacting with another component in a local system or distributed system, and / or data transmitted through signals to other systems and networks such as the Internet).
[0027] Furthermore, the term "or" is intended to mean an implicit "or" rather than an exclusive "or." That is, unless otherwise specified or evident from the context, "X uses A or B" is intended to mean one of the natural implicit substitutions. In other words, if X uses A; if X uses B; or if X uses both A and B, "X uses A or B" may apply to any of these cases. Additionally, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the enumerated related items.
[0028] Additionally, the terms “comprising” and / or “comprising” should be understood to mean that such features and / or components are present. However, the terms “comprising” and / or “comprising” should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Furthermore, unless otherwise specified or clearly evident from the context to indicate a singular form, the singular in this specification and claims should generally be interpreted to mean “one or more.”
[0029] And, the term “at least one of A or B” should be interpreted to mean “a case including only A,” “a case including only B,” or “a combination of A and B.”
[0030] Those skilled in the art should recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly exemplify the interchangeability of hardware and software, various exemplary components, blocks, configurations, means, logics, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled technicians may implement the described functionality in various ways for each specific application. However, such decisions regarding implementation should not be construed as going beyond the scope of this disclosure.
[0031] Description of the presented embodiments is provided to enable those skilled in the art to use or practice the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein. The present disclosure should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.
[0032] In the present disclosure, "point cloud data" refers to a set of three-dimensional coordinate points obtained as a result of scanning the surface of an object. For example, when an object is scanned using a non-contact sensor, X, Y, and Z coordinate information corresponding to each point on the surface of the object is generated, and these points come together to form a shape. Specifically, multiple points calculated based on distance and direction information collected by each sensor are organized into a single dataset, which can serve as a basis for extracting information such as the shape, size, and curvature of the object.
[0033] FIG. 1 is a block diagram of a computing device for extracting the 3D shape of an object using a plurality of non-contact sensors according to one embodiment of the present disclosure.
[0034] The configuration of the computing device (100) illustrated in FIG. 1 is merely a simplified example. In one embodiment of the present disclosure, the computing device (100) may include other configurations for performing the computing environment of the computing device (100), and only some of the disclosed configurations may constitute the computing device (100).
[0035] The computing device (100) may include a processor (110), memory (130), and a network unit (150).
[0036] The processor (110) may be composed of one or more cores and may include a processor for data analysis and deep learning, such as a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU). The processor (110) may read a computer program stored in memory (130) and perform data processing for machine learning according to one embodiment of the present disclosure. According to one embodiment of the present disclosure, the processor (110) may perform calculations for learning a neural network. The processor (110) may perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation.
[0037] At least one of the CPU, GPGPU, and TPU of the processor (110) can process the learning of a network function. For example, the CPU and GPGPU can together process the learning of a network function and data classification using the network function. In addition, in one embodiment of the present disclosure, processors of a plurality of computing devices can be used together to process the learning of a network function and data classification using the network function. In addition, a computer program executed in a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.
[0038] In the present disclosure, the processor (110) may generate positional relationship information between the first sensor and the second sensor for a plurality of sensors including the first sensor and the second sensor. The positional relationship information may be information regarding the coordinates of each sensor in an orthogonal coordinate system around the object to be measured. The first sensor and the second sensor may be sensors of the same type, but different types of sensors may be used, and the center coordinates that serve as the reference for the position of each sensor may be the center of gravity of each sensor or predefined coordinates.
[0039] In addition to the positional relationship information between the first sensor and the second sensor, the positional relationship information between the sensors may also include information about the position between each sensor and the object to be measured and the sensor. For example, the processor (110) may determine the center coordinates of the object to be measured as (0,0,0), and then determine that the first sensor is located at (3,-4, 5) and the second sensor is located at (-3, -4, -5).
[0040] However, specific expressions for generating sensor position information are not limited to the above disclosure, and it is obvious to a person skilled in the art that various techniques may be used to appropriately generate position relationship information for aligning data from multiple sensors, including cameras.
[0041] The processor (110) can generate first point cloud data corresponding to at least a part of the object to be measured based on the first sensor, and generate second point cloud data corresponding to at least a part of the object to be measured based on the second sensor. In the present disclosure, each sensor may be placed around the object to be measured and may be set to face different directions to generate more accurate 3D shape data according to the shape of the object to be measured.
[0042] Each sensor generates point cloud data corresponding to the same object being measured. Since the object being measured is a 3D object, there may be blind spots that are not included in the measurement range of the fixed sensor. Therefore, the point cloud data generated using multiple sensors corresponds to at least a part of the object being measured. The point cloud data generated using each sensor contains scan data for a part of the object being measured, and may also contain data corresponding to overlapping areas.
[0043] The processor (110) can generate a validation result for the first point cloud data and the second point cloud data. Specifically, the processor (110) can use an artificial neural network model to determine whether the first point cloud data and the second point cloud data are data that was properly acquired by a normal sensor.
[0044] To this end, the processor (110) can detect whether there is an abnormality in the first sensor based on inputting the first point cloud data into the artificial neural network model, and detect whether there is an abnormality in the second sensor based on inputting the second point cloud data into the artificial neural network model.
[0045] In the present disclosure, an artificial neural network model for identifying abnormalities in sensors may be an autoencoder-based artificial neural network model. The encoder part of an autoencoder-type artificial neural network model maps important features representing input data into a low-dimensional feature space, and the decoder part is trained to restore the feature vectors mapped to the feature space back to the original data. When data with features not learned by the model, i.e., anomalies, is input to a well-trained autoencoder model, the autoencoder has a characteristic of failing to perform data restoration well.
[0046] In the present disclosure, the processor (110) inputs point cloud data generated based on each sensor into a trained autoencoder model, calculates a reconstruction error from the value output by the model, and can detect whether the sensor is abnormal based on the reconstruction error. For example, the processor (110) calculates a reconstruction error for first point cloud data generated based on a first sensor, and if the reconstruction error is greater than or equal to a predetermined threshold, the autoencoder model can determine that the first point cloud data received contains anomalies and is in an abnormal state.
[0047] The fact that the point cloud data generated by each sensor contains anomalies may mean that the state of the sensor is not normal. Accordingly, in the present disclosure, if the restoration error of the first point cloud data generated based on the first sensor is greater than or equal to a predetermined threshold, the processor (110) may notify the user that repair or replacement of the first sensor is necessary by generating user notification information related to the abnormality of the first sensor. If the processor (110) identifies that all sensors are in a normal state, the processor (110) may generate a validity verification result that determines that each data is valid for the first point cloud data and the second point cloud data.
[0049] The processor (110) can determine weight information for the first point cloud data and the second point cloud data. In the present disclosure, the weight information may be a real value between 0 and 1 assigned according to each sensor, and may be a value such that the sum of all weights assigned to each sensor equals 1. For example, the processor (110) may determine a weight value such as 0.6 for the first point cloud data and 0.4 for the second point cloud data. At this time, the processor (110) may determine the weight using various criteria that can determine the reliability of the data.
[0050] Specifically, the processor (110) can determine weight information for the first point cloud data and the second point cloud data by determining location-based weight information based on the distance between each sensor and the object to be measured, and then determine data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data, and finally determine weight information based on combining the location-based weight information and the data-based weight information.
[0051] In the present disclosure, the closer the distance between the sensor and the object to be measured, the lower the possibility of errors caused by interference or reflection. Accordingly, to determine location-based weight information, the processor (110) may assign a location-based weight with a higher value as the distance between the object to be measured and each sensor becomes closer. For example, if the distance between the center of the object to be measured and the center of the first sensor is 4 cm and the distance between the center of the object to be measured and the center of the first sensor is 1 cm, the processor (110) may determine the location-based weight for the first point cloud data to be 0.2 and the location-based weight for the second point cloud data to be 0.8.
[0052] Meanwhile, since it can be expected that the accuracy of the data in the area will be high when point cloud data is generated more densely, the processor (110) can assign a higher data-based weight as the number of points included in the point cloud data generated by each sensor increases. For example, if the first point cloud data contains 13,000 points and the second point cloud data contains 7,000 points, the processor (110) can determine the data-based weight for the first point cloud data to be 0.65 and the data-based weight for the second point cloud data to be 0.35.
[0053] The processor (110) can determine the final weight information based on combining location-based weight information and data-based weight information. For example, if the location-based weight of the first point cloud data is 0.2 and the data-based weight is 0.65, and the location-based weight of the second point cloud data is 0.8 and the data-based weight is 0.35, the processor (110) can determine the weight of the first point cloud data as (0.2+0.65) / 2=0.425 and the weight of the second point cloud data as (0.8+0.35) / 2=0.575.
[0055] In the process of generating 3D shape data using non-contact sensors, each sensor may have a certain offset. Since it is difficult to properly align point cloud data generated from each sensor if this offset is left unchecked, the present disclosure includes a configuration for identifying and correcting the exact value of this offset.
[0056] The processor (110) can determine a first offset value corresponding to the first sensor and a second offset value corresponding to the second sensor.
[0057] To determine the first offset value, the processor (110) can determine a specific point of the object to be measured as the first point and generate a measurement value corresponding to the first point of the object to be measured using the first sensor. Then, the processor (110) can generate a measurement value corresponding to the same first point using the third sensor, which is a contact sensor. At this time, since the third sensor scans only a specific point rather than scanning the entire object to be measured, the process of determining the first offset value does not take much time. The processor (110) can determine the first offset value corresponding to the first sensor based on comparing the measurement value generated using the first sensor corresponding to the first point with the measurement value generated using the third sensor. The same process as above can be performed by the processor (110) for other sensors including the second sensor.
[0058] After that, the processor (110) can correct the first point cloud data based on the first offset value and correct the second point cloud data based on the second offset value. For example, if the first offset value is measured as (0, 0, 0.1), the processor (110) can perform a correction of (0, 0, -0.1) on all data included in the first point cloud data. The same process as above can be performed by the processor (110) on another sensor including the second sensor.
[0059] After that, the processor (110) can generate 3D shape data corresponding to the object to be measured based on the corrected first point cloud data, the corrected second point cloud data, and weight information. Specifically, the processor (110) can generate 3D shape data corresponding to the object to be measured by utilizing positional relationship information between the first sensor and the second sensor to align the first point cloud data and the second point cloud data, and applying weights to each point cloud data.
[0060] In the present disclosure, the 3D shape of an object can be measured using a plurality of non-contact sensors, and more precise 3D shape information can be generated by aligning point cloud data acquired from each sensor. In particular, by utilizing contact sensors as auxiliary tools to perform calibration of each non-contact sensor and integrating data by assigning weights according to the reliability or characteristics of each sensor, the influence of errors or noise from specific sensors can be minimized and the overall measurement accuracy can be improved. Therefore, according to the present disclosure, it is possible to stably obtain 3D shape data with high accuracy even for objects with complex shapes or various surface characteristics while using non-contact sensors, and a significant effect of improving overall scan quality and reliability occurs due to the multi-sensor configuration.
[0062] According to one embodiment of the present disclosure, the memory (130) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may operate in conjunction with web storage that performs the storage function of the memory (130) on the internet. The description of the memory described above is merely an example and the present disclosure is not limited thereto.
[0063] A network unit (150) according to one embodiment of the present disclosure can use various wired communication systems such as a public switched telephone network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and a local area network (LAN).
[0064] In addition, the network unit (150) presented in this specification may use various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA), and other systems.
[0065] In the present disclosure, the network unit (150) may use any type of wired or wireless communication system.
[0066] The technologies described in this specification can be used not only in the networks mentioned above but also in other networks.
[0068] FIG. 2 illustrates an exemplary structure of an artificial intelligence-based model according to one embodiment of the present disclosure.
[0069] Throughout this specification, artificial intelligence model, artificial intelligence-based model, computational model, neural network, network function, and neural network may be used interchangeably.
[0070] A neural network can be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node. The nodes (or neurons) constituting neural networks may be interconnected by one or more links.
[0071] In a neural network, one or more nodes connected via links can form relative input and output node relationships. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As described above, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.
[0072] In a relationship between an input node and an output node connected through a single link, the value of the output node's data can be determined based on the data input to the input node. Here, the link interconnecting the input node and the output node may have a weight. The weight can be variable and can be varied by the user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node's value can be determined based on the values input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.
[0073] As described above, a neural network consists of one or more nodes interconnected through one or more links, forming input-output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values for the links, the two neural networks may be recognized as different from each other.
[0074] A neural network can be composed of a set of one or more nodes. A subset of nodes constituting a neural network can form a layer. Some of the nodes constituting a neural network can form a layer based on their distances from an initial input node. For example, a set of nodes with a distance of n from an initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within a neural network can be defined in a way different from that described above. For example, a layer of nodes may be defined by its distance from a final output node.
[0075] In one embodiment of the present disclosure, a set of neurons or nodes may be defined by the expression a layer.
[0076] Initial input nodes may refer to one or more nodes within a neural network to which data is directly input without passing through links in their relationships with other nodes. Alternatively, in terms of link-based relationships between nodes within the neural network, they may refer to nodes that do not have other input nodes connected by links. Similarly, final output nodes may refer to one or more nodes within a neural network that do not have output nodes in their relationships with other nodes. Furthermore, hidden nodes may refer to nodes constituting the neural network that are neither initial input nodes nor final output nodes.
[0077] A neural network according to one embodiment of the present disclosure may have the number of nodes in the input layer equal to the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. Additionally, a neural network according to another embodiment of the present disclosure may have the number of nodes in the input layer less than the number of nodes in the output layer, and may be a neural network in which the number of nodes increases as it progresses from the input layer to the hidden layer. Additionally, a neural network according to yet another embodiment of the present disclosure may have the number of nodes in the input layer greater than the number of nodes in the output layer, and may be a neural network in which the number of nodes decreases as it progresses from the input layer to the hidden layer. A neural network according to yet another embodiment of the present disclosure may be a neural network in which the above-described neural networks are combined.
[0078] An artificial intelligence-based model according to one embodiment of the present disclosure may include a deep neural network (DNN). A deep neural network may refer to a neural network that includes a plurality of hidden layers in addition to an input layer and an output layer. By using a deep neural network, latent structures of data can be identified. That is, latent structures of photos, text, video, voice, protein sequence structures, gene sequence structures, peptide sequence structures, music (e.g., what objects are in the photo, what is the content and emotion of the text, what is the content and emotion of the voice, etc.), and / or binding affinity between peptides and MHCs can be identified. Deep neural networks may include convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, Siamese networks, Generative Adversarial Networks (GAN), Transformers, etc. The description of deep neural networks described above is merely illustrative and the present disclosure is not limited thereto.
[0079] The artificial intelligence-based model of the present disclosure may be represented by a network structure of any structure described above, including an input layer, a hidden layer, and an output layer.
[0080] A neural network that can be used in an artificial intelligence-based model of the present disclosure may be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, transfer learning, active learning, or reinforcement learning. Training of a neural network may be a process of applying knowledge to the neural network to perform a specific operation.
[0081] Neural networks can be trained to minimize the error in their output. The training process involves repeatedly inputting training data into the network, calculating the error between the network's output and the target for the training data, and updating the weights of each node by backpropagating the error from the output layer to the input layer in a direction that reduces the error. In supervised learning, training data is used where the correct answer is labeled for each data point (i.e., labeled training data), whereas in unsupervised learning, the correct answer may not be labeled for each training data point. For instance, in the case of supervised learning for data classification, the training data may consist of data where each training point is labeled with a category. Labeled training data is input into the neural network, and the error can be calculated by comparing the network's output (category) with the labels of the training data. As another example, in the case of unsupervised learning for data classification, the error can be calculated by comparing the input training data with the neural network's output. The calculated error is backpropagated in the neural network (i.e., from the output layer to the input layer), and through backpropagation, the connection weights of each node in each layer of the neural network can be updated. The amount of change in the connection weights of each node being updated can be determined by the learning rate. The neural network's calculation of the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network's learning cycle. For example, a high learning rate can be used in the early stages of training to quickly achieve a certain level of performance and increase efficiency, while a low learning rate can be used in the later stages to improve accuracy.
[0082] In the training of neural networks, the training data is generally a subset of the real-world data (i.e., the data intended to be processed using the trained neural network). Consequently, a training cycle may exist where errors decrease on the training data but increase on the real-world data. Overfitting is a phenomenon where the network learns excessively on the training data, leading to increased errors on the real-world data. For example, a neural network trained on yellow cats might fail to recognize cats when seeing anything other than yellow, which can be considered a type of overfitting. Overfitting can act as a cause for increased errors in machine learning algorithms. Various optimization methods can be used to prevent this overfitting. To prevent overfitting, methods such as increasing the training data, regularization, dropout (which disables some nodes in the network during training), and the use of batch normalization layers can be applied.
[0083] An AI-based model according to one embodiment of the present disclosure may include a Large Language Model (LLM). In the present disclosure, a Large Language Model may refer to an AI-based model trained using a vast amount of training data to perform Natural Language Processing. The Large Language Model may include a transformer, a transformer encoder family model, and / or a transformer decoder family model. A transformer encoder family model may correspond to an AI model using a transformer encoder structure. A transformer decoder family model may correspond to an AI model using a transformer decoder structure.
[0084] In one embodiment, the transformer may be composed of an encoder that encodes input data and a decoder that decodes the encoded data. The transformer may have a structure that takes a series of input data as input, undergoes encoding and decoding steps, and outputs a series of output data. In one embodiment, the series of input data may be processed into a form that the transformer can compute. The process of processing the series of input data into a form that the transformer can compute may include a tokenizing process and an embedding process. The tokenizing process may refer to the process of dividing the series of input data into tokens of a certain unit. For example, the certain unit may include word units. The embedding process may refer to the process of converting at least one token tokenized from the series of input data into an embedding vector.
[0085] In one embodiment, the transformer can obtain an embedding vector to be input to an encoder by combining a token embedding vector that embeds at least one token corresponding to a series of input data, a segment embedding vector that distinguishes sentences containing the tokens for each token, and a position embedding vector that reflects the position of the token. The encoder family model and the decoder family model of the transformer can also obtain embedding vectors by performing the same method.
[0086] In one embodiment, for the transformer to encode and decode a series of input data, the encoder and decoder within the transformer may utilize an attention algorithm. An attention algorithm may refer to an algorithm that, for a given query, calculates similarity by applying a softmax function to an attention score obtained by matrix multiplying the query with a key, and calculates an attention value for the query by matrix multiplying the calculated similarity with a value.
[0087] In one embodiment, the self-attention algorithm may refer to an attention algorithm that uses a query, a key, and a value generated by multiplying the same embedding vector by a query weight, a key weight, and a value weight, respectively. The cross-attention algorithm may refer to an attention algorithm that uses a query generated by multiplying a first embedding vector by a query weight, and a key and a value generated by multiplying a second embedding vector by a key weight and a value weight, respectively. The query weight, the key weight, and the value weight may be trainable parameters that are updated through the training process of a large-scale language model.
[0088] In one embodiment, the encoder of the transformer may include an embedding layer, a self-attention layer that applies a self-attention algorithm to an embedding vector, a normalization layer, and a feed-forward neural network (FFN). Additionally, the encoder may have a form in which N unit structures including a self-attention layer, a normalization layer, and a feed-forward neural network are connected. The decoder of the transformer may include an embedding layer, a masked self-attention layer, a normalization layer, a cross-attention layer that applies a cross-attention algorithm, and a feed-forward neural network. Additionally, the decoder may have a form in which N unit structures including a masked self-attention layer, a normalization layer, a cross-attention layer, and a feed-forward neural network are connected. The masked self-attention layer may correspond to a layer that calculates an attention value for each of the sequences in which words are sequentially included among a plurality of words included in a series of input data.
[0089] A transformer may include additional components such as a linear layer and a softmax layer, in addition to an encoder and a decoder. The encoder family model and the decoder family model of a transformer may also each include the aforementioned additional components in addition to the encoder and decoder. A method for constructing a transformer using an attention algorithm may include the method disclosed in Vaswani et al., Attention Is All You Need, 2017 NIPS, which is incorporated herein by reference.
[0090] In one embodiment, attention layers such as a self-attention layer, a masked self-attention layer, and a cross-attention layer may correspond to a multi-head attention layer that includes a plurality of attention layers in parallel. The multi-head attention layer may concatenate the attention values output from each of the plurality of attention layers and output an output attention value by matrix multiplying the concatenated matrix by an output weight. The output attention value output from the multi-head attention layer may have the same size as the attention value output from a single attention layer.
[0091] In one embodiment, the transformer may be trained through a Masked Language Model (MLM) process, a Next Sentence Prediction (NSP) process, etc. The MLM process may refer to a training process that predicts a masked word through a series of training data in which some words are masked. The NSP process may refer to a training process that determines whether two sentences are connected sentences in a series of training data containing any two sentences.
[0092] In one embodiment, a large-scale language model can process various data formats, such as natural language text as well as image data, audio data, and video data. The large-scale language model can embed data to convert data of various data formats into a series of computationally operable data. The large-scale language model can process additional data that represents the relative positional or topological relationships between a series of input data. Alternatively, a series of input data may be embedded by additionally reflecting vectors that represent the relative positional or topological relationships between the input data. In one example, the relative positional relationships between a series of input data may include, but are not limited to, word order within a natural language sentence, the relative positional relationships of each segmented image, and the temporal order of segmented audio waveforms. The process of adding information that represents the relative positional or topological relationships between a series of input data may be referred to as positional encoding.
[0093] An example of a large-scale language model for processing image data (Vision Transformer, ViT) is disclosed in Dosovitskiy, et al., AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE, which is incorporated herein by reference.
[0094] An artificial intelligence model according to one embodiment of the present disclosure may include a multimodal large-scale language model. A multimodal large-scale language model may refer to a large-scale language model capable of understanding and processing relationships between different data formats, such as natural language text data, image data, audio data, and video data. A multimodal language model may include a plurality of encoders that encode input data corresponding to each data format. A multimodal language model may be trained to calculate similarity between embedding vectors encoded by encoders of each data format through training data containing data of different data formats, such that similarity between identical pairs is calculated to be higher and similarity between different pairs is calculated to be lower.
[0095] An example of a large-scale multimodal language model that understands and processes the relationship between image data and natural language text data (Contrastive Language-Image Pre-training, CLIP) is disclosed in Alec Radford, et al., *LEARNING TRANSFERABLE VISUAL MODELS FROM NATURAL LANGUAGE SUPERVISION*, which is incorporated herein by reference.
[0097] FIG. 3 is a flowchart illustrating the process of extracting the 3D shape of an object using a plurality of non-contact sensors according to one embodiment of the present disclosure.
[0098] According to FIG. 3, the process of extracting a 3D shape of an object using a plurality of non-contact sensors in the present disclosure may include: a step (S110) of generating positional relationship information between a first sensor and a second sensor based on the positions of a first sensor and a second sensor; a step (S130) of generating first point cloud data corresponding to at least a part of the object to be measured based on the first sensor and generating second point cloud data corresponding to at least a part of the object to be measured based on the second sensor; a step (S150) of determining weight information for the first point cloud data and the second point cloud data; and a step (S170) of generating 3D shape data corresponding to the object to be measured based on the first point cloud data, the second point cloud data, and the weight information.
[0099] In step S110, the processor (110) can generate positional relationship information between the first sensor and the second sensor based on the positions of the first sensor and the second sensor. The specific process for generating positional relationship information between the first sensor and the second sensor has been described above with reference to FIG. 1.
[0100] In step S130, the processor (110) can generate first point cloud data corresponding to at least a part of the object to be measured based on the first sensor, and generate second point cloud data corresponding to at least a part of the object to be measured based on the second sensor. To determine the first offset value, the processor (110) can determine a specific point of the object to be measured as the first point and generate a measurement value corresponding to the first point of the object to be measured using the first sensor. After that, the processor (110) can generate a measurement value corresponding to the same first point using a third sensor, which is a contact sensor. At this time, since the third sensor scans only a specific point rather than scanning the entire object to be measured, the process of determining the first offset value does not take much time. The processor (110) can determine the first offset value corresponding to the first sensor based on comparing the measurement value generated using the first sensor and the measurement value generated using the third sensor corresponding to the first point. The same process as above can be performed by the processor (110) on another sensor including a second sensor.
[0101] After that, the processor (110) can correct the first point cloud data based on the first offset value and correct the second point cloud data based on the second offset value. For example, if the first offset value is measured as (0, 0, 0.1), the processor (110) can perform a correction of (0, 0, -0.1) on all data included in the first point cloud data. The same process as above can be performed by the processor (110) on another sensor including the second sensor.
[0102] In step S150, the processor (110) can determine weight information for the first point cloud data and the second point cloud data. Various factors for determining weight information may include location-based weights and data-based weights. As described above through FIG. 1, the processor (110) can determine weight information for the first point cloud data and the second point cloud data by determining location-based weight information based on the distance between each sensor and the object to be measured, and then determining data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data, and finally determining weight information based on combining the location-based weight information and the data-based weight information.
[0103] In step S170, the processor (110) can generate 3D shape data corresponding to the object to be measured based on the first point cloud data, the second point cloud data, and weight information. By performing calibration based on offset values for each non-contact sensor and synthesizing data in a way that gives higher weight to more accurate data, accurate 3D shape data corresponding to the object to be measured can be obtained quickly.
[0105] FIG. 4 is a conceptual diagram illustrating the process of generating 3D shape data of an object using a plurality of non-contact sensors according to one embodiment of the present disclosure.
[0106] Referring to FIG. 4, eight non-contact sensors (420) are positioned at respective locations to generate 3D shape data corresponding to a C-shaped object to be measured (410). In the present disclosure, the processor (110) can generate positional relationship information between the first sensor and the second sensor for a plurality of sensors including the first sensor and the second sensor. The positional relationship information may be information regarding the coordinates of each sensor in an orthogonal coordinate system around the object to be measured. The first sensor and the second sensor may be sensors of the same type, but different types of sensors may be used, and the center coordinates that serve as the reference for the position of each sensor may be the center of gravity of each sensor or predefined coordinates.
[0107] In addition to the positional relationship information between the first sensor and the second sensor, the positional relationship information between the sensors may also include information about the position between each sensor and the object to be measured and the sensor. For example, the processor (110) may determine the center coordinates of the object to be measured as (0,0,0), and then determine that the first sensor is located at (3,-4, 5) and the second sensor is located at (-3, -4, -5).
[0108] However, specific expressions for generating sensor position information are not limited to the above disclosure, and it is obvious to a person skilled in the art that various techniques may be used to appropriately generate position relationship information for aligning data from multiple sensors, including cameras.
[0109] The processor (110) can generate first point cloud data corresponding to at least a part of the object to be measured based on the first sensor, and generate second point cloud data corresponding to at least a part of the object to be measured based on the second sensor. In the present disclosure, each sensor may be placed around the object to be measured and may be set to face different directions to generate more accurate 3D shape data according to the shape of the object to be measured.
[0110] Each sensor generates point cloud data corresponding to the same object being measured. Since the object being measured is a 3D object, there may be blind spots that are not included in the measurement range of the fixed sensor. Therefore, the point cloud data generated using multiple sensors corresponds to at least a part of the object being measured. The point cloud data generated using each sensor contains scan data for a part of the object being measured, and may also contain data corresponding to overlapping areas.
[0111] Extracting the 3D shape of an object with a complex shape as described above using a contact sensor requires a significant amount of time for path design and execution, but when using a non-contact sensor, 3D shape data can be easily generated by adjusting the position of the sensor and changing the adjusted position coordinates.
[0113] FIG. 5 is a conceptual diagram illustrating the process of generating 3D shape data of an object based on synthesizing point cloud data generated by each sensor according to one embodiment of the present disclosure.
[0114] Each sensor generates point cloud data, and correction using an offset value can be performed on each point cloud data. Then, the processor (110) can generate 3D shape data (530) corresponding to the object to be measured by aligning the first point cloud data (510) generated by the first sensor and the second point cloud data (520) generated by the second sensor within a single coordinate system.
[0115] In the process of generating 3D shape data (530), the processor (110) may determine weight information for the first point cloud data (510) and the second point cloud data (520). In the present disclosure, the weight information may be a real value between 0 and 1 assigned according to each sensor, and may be a value such that the sum of all weights assigned to each sensor equals 1. For example, the processor (110) may determine a weight value such as 0.6 for the first point cloud data (510) and 0.4 for the second point cloud data (520). At this time, the processor (110) may determine the weight using various criteria that can determine the reliability of the data.
[0116] Specifically, the processor (110) can determine weight information for the first point cloud (510) data and the second point cloud data (520) by determining location-based weight information based on the distance between each sensor and the object to be measured, and then determine data-based weight information based on comparing the number of points of the first point cloud data (510) and the second point cloud data (520), and finally determine weight information based on combining the location-based weight information and the data-based weight information.
[0117] In the present disclosure, the closer the distance between the sensor and the object to be measured, the lower the possibility of errors caused by interference or reflection. Accordingly, to determine location-based weight information, the processor (110) may assign a location-based weight with a higher value as the distance between the object to be measured and each sensor becomes closer. For example, if the distance between the center of the object to be measured and the center of the first sensor is 4 cm and the distance between the center of the object to be measured and the center of the first sensor is 1 cm, the processor (110) may determine the location-based weight for the first point cloud data (510) to be 0.2 and the location-based weight for the second point cloud data (520) to be 0.8.
[0118] Meanwhile, since it can be expected that the accuracy of the data in the area will be higher when point cloud data is generated more densely, the processor (110) can assign a higher data-based weight as the number of points included in the point cloud data generated by each sensor increases. For example, if the first point cloud data (510) contains 13,000 points and the second point cloud data (520) contains 7,000 points, the processor (110) can determine the data-based weight for the first point cloud data (510) to be 0.65 and the data-based weight for the second point cloud data to be 0.35.
[0119] The processor (110) can determine the final weight information based on combining location-based weight information and data-based weight information. For example, if the location-based weight of the first point cloud data (510) is 0.2 and the data-based weight is 0.65, and the location-based weight of the second point cloud data (520) is 0.8 and the data-based weight is 0.35, the processor (110) can determine the weight of the first point cloud data (510) as (0.2+0.65) / 2=0.425 and the weight of the second point cloud data (520) as (0.8+0.35) / 2=0.575.
[0120] After that, the processor (110) can generate 3D shape data corresponding to the object to be measured based on the first point cloud data (510), the second point cloud data (520), and weight information. Specifically, the processor (110) can generate 3D shape data corresponding to the object to be measured by aligning the first point cloud data (510) and the second point cloud data (520) using positional relationship information of the first sensor and the second sensor, and applying weights to each point cloud data.
[0124] Meanwhile, a computer-readable medium storing a data structure is disclosed according to an embodiment of the present disclosure.
[0125] A data structure can refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure can refer to the organization of data to solve specific problems (e.g., data retrieval, data storage, data modification in the shortest possible time). A data structure may also be defined by physical or logical relationships between data elements designed to support specific data processing functions. Logical relationships between data elements may include connections between user-defined data elements. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (e.g., a permanent storage device). Specifically, a data structure may include sets of data, relationships between data, and functions or instructions applicable to the data. Through an effectively designed data structure, a computing device can perform operations while minimizing the use of the device's resources. Specifically, through an effectively designed data structure, a computing device can increase the efficiency of operations, reading, insertion, deletion, comparison, exchange, and retrieval.
[0126] Data structures can be classified into linear and non-linear data structures based on their form. A linear data structure is one where only one piece of data is connected to the next. Linear data structures can include lists, stacks, queues, and deques. A list can refer to a set of data that maintains an internal order. Lists can include linked lists. A linked list is a data structure where data is connected in a line, with each piece of data possessing a pointer. In a linked list, the pointer can contain information regarding the connection to the next or previous data. Depending on its form, a linked list can be represented as a singly linked list, a doubly linked list, or a circular linked list. A stack is a data arrangement structure that allows for restricted access to data. A stack can be a linear data structure where data can be processed (e.g., insertion or deletion) only at one end of the structure. Data stored in a stack can be a Last-In, First-Out (LIFO) data structure, meaning that the later an item is entered, the sooner it is retrieved. A queue is a data sequence structure that allows for limited access to data; unlike a stack, it can be a FIFO (First in First Out) data structure where data stored later is retrieved later. A deque is a data structure that can process data at both ends.
[0127] Non-linear data structures can be structures where multiple data are connected after a single piece of data. Non-linear data structures may include graph data structures. A graph data structure can be defined by vertices and edges, and an edge may include a line connecting two different vertices. Graph data structures may include tree data structures. A tree data structure may be a data structure where there is only one path connecting two different vertices among the multiple vertices included in the tree. In other words, it may be a data structure that does not form a loop in a graph data structure.
[0128] Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. Hereinafter, the term neural network will be used consistently. A data structure may include a neural network. Furthermore, a data structure including a neural network may be stored on a computer-readable medium. A data structure including a neural network may also include data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, loss functions for learning the neural network, etc. A data structure including a neural network may include any of the components disclosed above. That is, a data structure including a neural network may be configured to include all or any combination thereof, such as data preprocessed for processing by the neural network, data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. In addition to the configurations described above, a data structure including a neural network may include any other information that determines the characteristics of the neural network. Furthermore, the data structure may include any form of data used or generated during the computational process of the neural network, and is not limited to the foregoing. A computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. A neural network may be composed of a set of interconnected computational units that may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is composed of at least one node.
[0129] A data structure may include data input to a neural network. A data structure including data input to a neural network may be stored on a computer-readable medium. Data input to a neural network may include training data input during the neural network learning process and / or input data input to a neural network after training is complete. Data input to a neural network may include pre-processed data and / or data subject to pre-processing. Pre-processing may include a data processing process for inputting data into a neural network. Accordingly, a data structure may include data subject to pre-processing and data generated by pre-processing. The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.
[0130] The data structure may include weights of the neural network. (In this specification, weights and parameters may be used interchangeably.) The data structure including the weights of the neural network may be stored on a computer-readable medium. The neural network may include multiple weights. The weights may be variable and may be varied by a user or an algorithm to enable the neural network to perform a desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine the data value output from the output node based on values input to the input nodes connected to the output node and weights set on the links corresponding to each input node. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.
[0131] As an example rather than a limitation, weights may include weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Weights that vary during the neural network learning process may include weights at the start of the learning cycle and / or weights that vary during the learning cycle. Weights for which neural network learning is completed may include weights for which the learning cycle is completed. Accordingly, a data structure containing the weights of a neural network may include a data structure containing weights that vary during the neural network learning process and / or weights for which neural network learning is completed. Therefore, the weights and / or combinations of each weight described above are included in the data structure containing the weights of a neural network. The aforementioned data structure is merely an example and the present disclosure is not limited thereto.
[0132] Data structures containing the weights of a neural network may be stored on a computer-readable storage medium (e.g., memory, hard disk) after undergoing a serialization process. Serialization may be a process of converting a data structure into a form that can be stored on the same or different computing devices and later reconstructed for use. A computing device may serialize the data structure to transmit and receive data over a network. A serialized data structure containing the weights of a neural network may be reconstructed on the same or different computing devices through deserialization. Data structures containing the weights of a neural network are not limited to serialization. Furthermore, data structures containing the weights of a neural network may include data structures designed to increase computational efficiency while minimizing the use of computing device resources (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree in non-linear data structures). The foregoing is merely an example and the present disclosure is not limited thereto.
[0133] The data structure may include hyperparameters of the neural network. The data structure including the neural network hyperparameters may be stored on a computer-readable medium. The hyperparameters may be variables that are varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle iterations, weight initialization (e.g., setting the range of weight values subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layers). The aforementioned data structure is merely an example, and the present disclosure is not limited thereto.
[0134] FIG. 6 is a brief and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.
[0135] Although the present disclosure has been described as generally being implementable by a computing device, a person skilled in the art will be well aware that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or as a combination of hardware and software.
[0136] Generally, a program module includes routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Furthermore, a person skilled in the art will be well aware that the method of the present disclosure can be implemented in other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which may be connected to and operated with one or more associated devices).
[0137] The embodiments described in this disclosure may also be implemented in a distributed computing environment in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
[0138] Computers typically include various computer-readable media. Any medium accessible by a computer may be a computer-readable medium, and such computer-readable media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media. By example, but not limiting, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be accessed by a computer and used to store desired information.
[0139] Computer-readable transmission media typically include all information transmission media that implement computer-readable instructions, data structures, program modules, or other data, etc., on a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal in which one or more of the characteristics of the signal are set or modified to encode information within the signal. By example, not limiting, computer-readable transmission media include wired media, such as wired networks or direct-wired connections, and wireless media, such as acoustic, RF, infrared, and other wireless media. Any combination of the media described above is also considered to be within the scope of computer-readable transmission media.
[0140] An exemplary environment for implementing various aspects of the present disclosure, including a computer (1102), is shown, wherein the computer (1102) includes a processing unit (1104), system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including system memory (1106) (but not limited thereto), to the processing unit (1104). The processing unit (1104) may be any processor among various commercial processors. Dual processors and other multiprocessor architectures may also be used as the processing unit (1104).
[0141] The system bus (1108) may be any of several types of bus structures that can be additionally interconnected to a local bus using any of the memory bus, peripheral bus, and various commercial bus architectures. System memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). The basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, EEPROM, etc., and this BIOS includes basic routines that help transfer information between components within the computer (1102) at times such as during startup. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.
[0142] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA)—this internal hard disk drive (1114) may also be configured for external use within a suitable chassis (not shown)—a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from a CD-ROM disk (1122) or reading from or writing to other high-capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may each be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128). The interface (1124) for implementing an external drive includes at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.
[0143] These drives and associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. Although the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, a person skilled in the art will know that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, etc., may also be used in exemplary operating environments and that any of these media may contain computer-executable instructions for performing the methods of the present disclosure.
[0144] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or part of the operating system, application, module and / or data may also be cached in RAM (1112). It will be well known that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.
[0145] The user can input commands and information into the computer (1102) through one or more wired / wireless input devices, such as a pointing device like a keyboard (1138) and a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, etc. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) connected to the system bus (1108), but may also be connected via other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc.
[0146] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface such as a video adapter (1146). In addition to the monitor (1144), the computer generally includes other peripheral output devices (not shown), such as speakers, a printer, and so on.
[0147] The computer (1102) may operate in a networked environment using a logical connection to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communication. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), but for brevity, only the memory storage device (1150) is illustrated. The illustrated logical connection includes a wired / wireless connection to a local area network (LAN) (1152) and / or a larger network, e.g., a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a global computer network, e.g., the Internet.
[0148] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communication to the LAN (1152), and the LAN (1152) may also include a wireless access point installed therein to communicate with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communication computing device on the WAN (1154), or have other means to establish communication through the WAN (1154), such as through the Internet. The modem (1158), which may be an internal or external and a wired or wireless device, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, the program modules described for the computer (1102) or parts thereof may be stored in a remote memory / storage device (1150). It will be well known that the illustrated network connection is exemplary and that other means of establishing a communication link between computers may be used.
[0149] The computer (1102) operates to communicate with any wireless device or object that is deployed and operated via wireless communication, for example, a printer, scanner, desktop and / or portable computer, PDA (portable data assistant), communication satellite, any equipment or place associated with a wireless detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network, or simply ad hoc communication between at least two devices.
[0150] Wi-Fi (Wireless Fidelity) enables connectivity to the Internet and other sources without wires. Wi-Fi is a wireless technology, similar to a cell phone, that allows devices, such as computers, to transmit and receive data indoors and outdoors—that is, anywhere within the coverage area of a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz wireless bands, for example, at data rates of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band).
[0151] Those skilled in the art of the present disclosure will understand that information and signals may be represented using any various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
[0152] Those skilled in the art will understand that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs or design code (referred to herein as software for convenience), or a combination of all such. To clearly illustrate this interoperability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the design constraints imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementation decisions should not be interpreted as being outside the scope of this disclosure.
[0153] The various embodiments presented herein may be implemented as methods, devices, or articles manufactured using standard programming and / or engineering techniques. The term "article manufactured" includes a computer program, a carrier, or a medium accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical discs (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Additionally, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.
[0154] It should be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of this disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but do not imply being limited to the specific order or hierarchy presented.
[0155] Description of the presented embodiments is provided so that a person skilled in the art may use or practice the present disclosure. Various modifications to these embodiments will be apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.
Claims
Claim 1 A method performed on a computing device to extract a 3D shape of an object using a plurality of non-contact sensors, comprising: a step of generating positional relationship information between a first sensor and a second sensor based on the positions of a first sensor and a second sensor; a step of generating first point cloud data corresponding to at least a part of a measurement target object based on the first sensor; a step of generating second point cloud data corresponding to at least a part of the measurement target object based on the second sensor; a step of determining weight information for the first point cloud data and the second point cloud data; and a step of generating 3D shape data corresponding to the measurement target object by aligning the first point cloud data and the second point cloud data based on the weight information; wherein the step of determining weight information for the first point cloud data and the second point cloud data comprises: a step of determining location-based weight information based on the distance between the first sensor and the measurement target object and the distance between the second sensor and the measurement target object; and a step of determining data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data. A method comprising the step of determining the weight information based on combining the location-based weight information and the data-based weight information. Claim 2 A method according to claim 1, further comprising the step of generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model prior to the step of determining weight information for the first point cloud data and the second point cloud data. Claim 3 In claim 2, the step of generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model comprises: a step of detecting whether the first sensor is abnormal based on inputting the first point cloud data into the artificial neural network model; a step of detecting whether the second sensor is abnormal based on inputting the second point cloud data into the artificial neural network model; and a step of generating the validation result based on whether the first sensor is abnormal and whether the second sensor is abnormal. Claim 4 In claim 3, the artificial neural network model is an autoencoder-based artificial neural network model, and the step of detecting whether the first sensor is abnormal based on inputting the first point cloud data into the artificial neural network model comprises: the step of detecting whether the first sensor is abnormal based on the reconstruction error of the artificial neural network model for the first point cloud data. Claim 5 In claim 4, the step of detecting an anomaly in the first sensor based on inputting the first point cloud data into the artificial neural network model further comprises: the step of generating user notification information related to the anomaly in the first sensor when the restoration error of the first point cloud data of the first sensor is greater than or equal to a predetermined threshold. Claim 6 delete Claim 7 In claim 1, the step of determining the location-based weighting information based on the distance between the first sensor and the object to be measured and the distance between the second sensor and the object to be measured comprises: assigning a higher location-based weighting as the distance between the object to be measured and each sensor becomes closer; a method. Claim 8 In claim 1, the step of determining data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data comprises: a step of assigning a higher data-based weight the greater the number of points included in each point cloud data; a method. Claim 9 A method according to claim 1, further comprising, prior to the step of generating 3D shape data corresponding to the object to be measured by aligning the first point cloud data and the second point cloud data based on the weighting information, the step of determining a first offset value corresponding to the first sensor; the step of determining a second offset value corresponding to the second sensor; the step of correcting the first point cloud data based on the first offset value; and the step of correcting the second point cloud data based on the second offset value. Claim 10 In claim 9, the step of determining a first offset value corresponding to the first sensor comprises: a step of generating a measurement value corresponding to a first point of the object to be measured using the first sensor; a step of generating a measurement value corresponding to the first point using a third sensor which is a contact sensor; and a step of determining the first offset value based on comparing the measurement values generated using the first sensor and the third sensor corresponding to the first point. Claim 11 A computer program stored on a computer-readable storage medium, which enables a computing device to perform operations for extracting a 3D shape of an object using a plurality of non-contact sensors, wherein the operations include: an operation of generating positional relationship information between a first sensor and a second sensor based on the positions of a first sensor and a second sensor; an operation of generating first point cloud data corresponding to at least a part of a measurement target object based on the first sensor; an operation of generating second point cloud data corresponding to at least a part of the measurement target object based on the second sensor; and an operation of determining weight information for the first point cloud data and the second point cloud data. A computer program stored on a computer-readable storage medium, comprising: an operation to generate 3D shape data corresponding to the object to be measured by aligning the first point cloud data and the second point cloud data based on the weight information; wherein the operation to determine weight information for the first point cloud data and the second point cloud data comprises: an operation to determine location-based weight information based on the distance between the first sensor and the object to be measured and the distance between the second sensor and the object to be measured; an operation to determine data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data; and an operation to determine the weight information based on combining the location-based weight information and the data-based weight information. Claim 12 A computer program stored on a computer-readable storage medium, further comprising, in claim 11, an operation of generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model prior to the operation of determining weight information for the first point cloud data and the second point cloud data. Claim 13 A computer program stored on a computer-readable storage medium, wherein, in claim 12, the operation of generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model comprises: an operation of detecting whether the first sensor is abnormal based on inputting the first point cloud data into the artificial neural network model; an operation of detecting whether the second sensor is abnormal based on inputting the second point cloud data into the artificial neural network model; and an operation of generating the validation result based on whether the first sensor is abnormal and whether the second sensor is abnormal. Claim 14 A computer program stored on a computer-readable storage medium, wherein, in claim 13, the artificial neural network model is an autoencoder-based artificial neural network model, and the operation of detecting an abnormality of the first sensor based on inputting the first point cloud data into the artificial neural network model comprises: the operation of detecting an abnormality of the first sensor based on the reconstruction error of the artificial neural network model for the first point cloud data. Claim 15 A computer program stored on a computer-readable storage medium, wherein, in claim 14, the operation of detecting an abnormality of the first sensor based on inputting the first point cloud data into the artificial neural network model further comprises: the operation of generating user notification information related to the abnormality of the first sensor when the restoration error of the first point cloud data of the first sensor is greater than or equal to a predetermined threshold. Claim 16 A computing device for extracting the 3D shape of an object using multiple non-contact sensors, comprising one or more processors; The apparatus includes memory; wherein the one or more processors are configured to generate positional relationship information between the first sensor and the second sensor based on the positions of the first sensor and the second sensor, generate first point cloud data corresponding to at least a part of the object to be measured based on the first sensor, generate second point cloud data corresponding to at least a part of the object to be measured based on the second sensor, determine weight information for the first point cloud data and the second point cloud data, and generate 3D shape data corresponding to the object to be measured by aligning the first point cloud data and the second point cloud data based on the weight information, and determining the weight information for the first point cloud data and the second point cloud data comprises: determining position-based weight information based on the distance between the first sensor and the object to be measured and the distance between the second sensor and the object to be measured; and determining data-based weight information based on comparing the number of points of the first point cloud data and the second point cloud data. A computing device comprising: determining the weight information based on combining the location-based weight information and the data-based weight information. Claim 17 A computing device according to claim 16, further comprising, prior to determining weight information for the first point cloud data and the second point cloud data, generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model. Claim 18 In claim 17, generating a validation result for the first point cloud data and the second point cloud data by utilizing an artificial neural network model comprises: detecting whether the first sensor is abnormal based on inputting the first point cloud data into the artificial neural network model; detecting whether the second sensor is abnormal based on inputting the second point cloud data into the artificial neural network model; and generating the validation result based on whether the first sensor is abnormal and whether the second sensor is abnormal. Claim 19 In claim 18, the artificial neural network model is an autoencoder-based artificial neural network model, and based on inputting the first point cloud data into the artificial neural network model, detecting whether the first sensor is abnormal is: detecting whether the first sensor is abnormal based on the reconstruction error of the artificial neural network model for the first point cloud data; comprising a computing device. Claim 20 A computing device according to claim 19, wherein detecting whether the first sensor is abnormal based on inputting the first point cloud data into the artificial neural network model further comprises: generating user notification information related to the abnormality of the first sensor when the restoration error of the first sensor for the first point cloud data is greater than or equal to a predetermined threshold.
Citation Information
Patent Citations
Surface shape measurement method and surface shape measurement device
JP2017015653A
Object monitoring device equipped with multiple sensors
JP2019078725A
System for monitoring of structural and method ithereof
KR1020220170122A
Facility maintenance system using 3D scanning equipment, and method thereof
KR102293526B1
