Semi-Supervised Framework for Efficient Time-Series Order Classification

Abstract

A computer-implemented method for ordinal prediction is provided. The method includes encoding time series data with a temporal encoder to obtain a latent space representation. The method includes optimizing the temporal encoder using semi-supervised learning to distinguish different classes in the labeled space with the labeled data and augmenting the latent space representation with unlabeled training data to obtain a semi-supervised representation. The method further includes discarding a linear layer after the temporal encoder and modifying the temporal encoder. The method also includes training k-1 binary classifiers on the semi-supervised representation to obtain k-1 binary predictions. The method further includes identifying and correcting inconsistent ones of the k-1 binary predictions by matching them with non-inconsistent ones of the k-1 binary predictions. The method further includes aggregating the k-1 binary predictions to obtain an ordinal prediction.

Description

Technical Field

[0001] The present invention relates to a classifier, and more particularly to a semi-supervised framework for efficient time-series order classification.

Background Art

[0002] Description of Related Art Real-world classification problems often involve labels with an inherent order. Consider an example where a patient's health status is classified into three label categories: "good", "stable", and "critical". Since "stable" is close to "critical", misclassifying "critical" as "good" would be a disaster. In such a setting, formal classification methods such as classifiers using cross-entropy loss with one-hot encoded labels are prone to making such errors. This undesirable behavior is a result of the loss function that equally penalizes classification errors. In contrast, order classification methods (sometimes called ordinal regression) suppress errors that violate the order inherent in the labels. Therefore, order classification methods are more suitable for classification problems involving ordered labels. Order classification can be considered to lie between formal classification and regression. Unlike regression tasks that map inputs to a continuous and ordered space, in order classification, the inputs need to be classified into ordered discrete categories.

[0003] Conventional order classification methods have two weaknesses and are not practical in actual business. First, they assume that all training data is labeled, which is not always correct, especially when it is costly and time-consuming to request annotations from human experts. Second, the predictions of a collection of binary classifiers can be inconsistent. In the above example, the first binary classifier can indicate that the patient's health status is critical, while the second classifier indicates that it is good, making it difficult to aggregate the two classifications.

Summary of the Invention

[0004] According to an aspect of the present invention, there is provided a computer-implemented method for order prediction. The method includes encoding time series data with a time encoder to obtain a latent space representation. The method includes optimizing the time encoder using semi-supervised learning to distinguish different classes in a labeled space using labeled data, reinforcing the latent space representation using unlabeled training data, and obtaining a semi-supervised representation. The method further includes discarding a linear layer after the time encoder and modifying the time encoder. The method also includes training k-1 binary classifiers on the semi-supervised representation to obtain k-1 binary prediction values. The method further includes identifying and correcting conflicting ones of the k-1 binary prediction values by matching them with non-conflicting ones of the k-1 binary prediction values. The method further includes aggregating the k-1 binary prediction values to obtain an order prediction.

[0005] According to another aspect of the present invention, there is provided a computer program product for order prediction. The computer program product includes a non-transitory computer-readable storage medium having program instructions embodied therein. The program instructions are executable by a computer to perform a method. The method includes encoding time series data with a time encoder by a hardware processor of the computer to obtain a latent space representation. The method further includes optimizing the time encoder using semi-supervised learning by the hardware processor to distinguish different classes in a labeled space using labeled data and reinforce the latent space representation using unlabeled training data to obtain a semi-supervised representation. The method also includes discarding a linear layer after the time encoder and modifying the time encoder by the hardware processor. The method further includes training k-1 binary classifiers on the semi-supervised representation by the hardware processor to obtain k-1 binary prediction values. The method further includes identifying and correcting conflicting ones of the k-1 binary prediction values by matching them with non-conflicting ones of the k-1 binary prediction values by the hardware processor. The method also includes aggregating the k-1 binary prediction values to obtain an order prediction by the hardware processor.

[0006] According to yet another aspect of the present invention, there is provided a computer processing system for order prediction. The system includes a memory device for storing program code. The system further includes a processor device operatively coupled to the memory device and executing program code for encoding time series data with a time encoder to obtain a latent space representation. The processor device also executes program code for optimizing the time encoder using semi-supervised learning to distinguish different classes in a labeled space using labeled data and reinforcing the latent space representation using unlabeled training data to obtain a semi-supervised representation. The processor device further executes program code for discarding a linear layer after the time encoder and modifying the time encoder. The processor device further executes program code for training k - 1 binary classifiers on the semi-supervised representation to obtain k - 1 binary prediction values. The processor device also executes program code for identifying and correcting conflicting ones of the k - 1 binary prediction values by matching them with non-conflicting ones of the k - 1 binary prediction values. The processor device further executes program code for aggregating the k - 1 binary prediction values to obtain an order prediction.

[0007] These and other features and advantages will become apparent from the following detailed description of its exemplary embodiments, read in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0008] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0009]

Figure 1

[0010]

Figure 2

[0011]

Figure 3

[0012]

Figure 4

[0013]

Figure 5

Figure 6

[0014]

Figure 7

[0015] Embodiments of the present invention are directed to a semi-supervised framework for efficient time series ordinal classification.

[0016] As described above, many state-of-the-art techniques assume that ideally all training data is labeled, so performance may degrade if only a portion of the training data is annotated. Additionally, conflicts in the results obtained from a collection of binary classifiers can also contribute to unsatisfactory performance.

[0017] One or more embodiments provide a semi-supervised representation learning module. The semi-supervised representation learning module aims to learn excellent representations by training with both labeled data and unlabeled data so as to achieve high data efficiency. In one or more embodiments, the semi-supervised representation learning module uses a semi-supervised loss that enables learning representations from labeled data, and the representations can also be reinforced by unlabeled data.

[0018] One or more embodiments provide an ensemble binary classification module. The ensemble binary classification module decomposes a semi-supervised ordinal k-class learning problem into k - 1 binary learning problems. The ensemble binary classification module operates on a representation module and returns a collection of binary prediction results. In one or more embodiments, the ensemble binary classification model converts an ordinal classification, which is a complex problem, into a controller binary classification sub-problem.

[0019] One or more embodiments provide a robust aggregation module. The robust aggregation module aims to detect and correct conflicting prediction results among a collection of binary classifiers and aggregate the results for a final decision. In one or more embodiments, the robust aggregation module employs a conflict detector to identify and correct conflicts between binary classifications. The aggregation further enforces that order information is retained in the final output.

[0020] FIG. 1 is a block diagram showing an exemplary computing device 100 according to an embodiment of the present invention. The computing device 100 is configured to perform a semi-supervised ordinal classification in a time series.

[0021] The computing device 100 can be embodied as any type of computing or computer device capable of performing the functions described herein, including, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network device, a web device, a distributed computing system, a processor-based system, and / or a user electronic device. Additionally or alternatively, the computing device 100 may be embodied as one or more compute threads, memory threads, or other components of a rack, thread, computing chassis, or other physically decomposed computing device. As shown in FIG. 1, the computing device 100 illustratively includes a processor 110, an input / output subsystem 120, a memory 130, a data storage device 140, and a communication subsystem 150, and / or other components and devices commonly found in a server or similar computing device. Of course, the computing device 100 may include other or additional components (e.g., various input / output devices) as commonly found in a server computer in other embodiments. Further, in some embodiments, one or more of the exemplary components may be incorporated into or otherwise form part of another component. For example, the memory 130, or a portion thereof, may be incorporated into the processor 110 in some embodiments.

[0022] The processor 110 can be embodied as any type of processor capable of performing the functions described herein. The processor 110 may be embodied as a single processor, a multi-processor, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0023] Memory 130 may be embodied as any type of volatile or non-volatile memory or data storage that can execute the functions described herein. During operation, memory 130 can store various data and software used during the operation of arithmetic unit 100, such as an operating system, applications, programs, libraries, and drivers. Memory 130 is communicatively coupled to processor 110 via I / O subsystem 120 and may be embodied as circuitry and / or components for facilitating input / output operations between processor 110, memory 130, and other components of arithmetic unit 100. For example, I / O subsystem 120 may be embodied as a memory controller hub, an input / output control hub, a platform controller hub, an integrated control circuit, a firmware device, a communication link (e.g., a point-to-point link, a bus link, a wire, a cable, a light guide, a printed circuit board trace, etc.) and / or other components and subsystems for facilitating input / output operations, or alternatively, may include these. In some embodiments, I / O subsystem 120 forms part of a system-on-chip (SOC) and may be incorporated into a single integrated circuit chip together with processor 110, memory 130, and other components of arithmetic unit 100.

[0024] The data storage device 140 can be embodied as any type of device or apparatus configured for short-term or long-term storage of data, such as, for example, a memory device and circuits, a memory card, a hard disk drive, a solid state drive, or other data storage devices. The data storage device 140 can store program code for time-series semi-supervised sequential classification. The communication subsystem 150 of the computing device 100 can be embodied as any network interface controller or other communication circuit, device, or combination thereof that can enable communication between the computing device 100 and other remote devices via a network. The communication subsystem 150 can be configured to implement such communication using any one or more communication technologies (e.g., wired or wireless communication) and associated protocols (e.g., Ethernet, InfiniBand®, Bluetooth®, Wi-Fi®, WiMAX®, etc.).

[0025] As shown, the computing device 100 can also include one or more peripheral devices 160. The peripheral devices 160 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, the peripheral devices 160 can include a display, a touch screen, a graphics circuit, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0026] Of course, the computing device 100 can also include other elements (not shown), which can be easily conceived by those skilled in the art, and certain elements can also be omitted. For example, various other input devices and / or output devices can be included in the computing device 100 depending on the specific implementation of the same, as can be easily understood by those skilled in the art. For example, various types of wireless and / or wired input and / or output devices can be used. Further, a processor, a controller, a memory, etc. can be added and utilized in various configurations. These and other variations of the processing system 100 are easily contemplated by those skilled in the art in view of the teachings of the present invention provided herein.

[0027] As used herein, the term "hardware processor subsystem" or "hardware processor" can refer to a processor, memory (including RAM, cache, etc.), software (including memory management software), or a combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem can include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements can include a central processing unit, an image processing unit, and / or a controller based on a separate processor or computing element (e.g., logic gates, etc.). The hardware processor subsystem can include one or more on-board memories (e.g., cache, dedicated memory arrays, read-only memories, etc.). In some embodiments, the hardware processor subsystem can include one or more memories that can be on-board or off-board, or dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input / output system (BIOS), etc.).

[0028] In one embodiment, the hardware processor subsystem can include and execute one or more software elements. The one or more software elements can include an operating system and / or one or more applications and / or specific code to achieve a particular result.

[0029] In other embodiments, the hardware processor subsystem can include dedicated circuitry that executes one or more electronic processing functions to achieve a specified result. Such circuitry can include one or more application-specific integrated circuits (ASICs), FPGAs, and / or PLAs.

[0030] These and other variations of the hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0031] FIG. 2 is a block diagram showing an exemplary semi-supervised k-class ordinal learning 200 according to an embodiment of the present invention.

[0032] The semi-supervised k-class ordinal learning 200 includes a first phase 291 corresponding to semi-supervised learning, a second phase 292 corresponding to binary classification on the semi-supervised representation, and a third phase 293 corresponding to robust aggregation.

[0033] Given a collection 201 of labeled and unlabeled data, a time encoder (usually an LSTM neural network) 210 is trained by semi-supervised learning 291. By doing so, the original input 201 can be converted into a latent space representation 220. The present invention utilizes the latent space representation 220 and constructs k - 1 binary classifiers 230 thereon. Then, the present invention uses an error identifier 240 to check whether there are contradictions during the binary predictions 231 and correct them when such contradictions are identified. Finally, the present invention aggregates the corrected binary predictions 231 as ordinal predictions 251.

[0034] Figure 3 shows an exemplary method 300 of semi-supervised k-class ordinal learning according to an embodiment of the present invention.

[0035] In block 310, a time series is encoded by a time encoder.

[0036] In block 320, the encoder is optimized by semi-supervised learning.

[0037] In block 330, the last layer is discarded and the encoder is modified.

[0038] In block 340, k-1 binary classifiers are trained on top of the semi-supervised representation.

[0039] In block 350, it is determined whether the binary classifier provides conflicting binary predictions. If so, proceed to block 260. Otherwise, proceed to block 270.

[0040] In block 360, conflicting binary predictions are identified and corrected.

[0041] In block 370, the binary predictions are aggregated as ordinal predictions.

[0042] Next, block 310 according to an embodiment of the present invention will be further described.

[0043] First, using a neural network, time series segments are transformed from a high dimension to a low-dimensional latent space. These segments are sliced from very long sequences included in the training data. The sequence is all recorded values in consecutive time periods. To capture the time-dependency of the time series, LSTM 210 is used as an encoder, and by optimizing LSTM 210 with several loss functions described below, the time series is transformed into a latent space representation. Note that other encoders such as GRU can also be used while maintaining the spirit of the present invention.

[0044] Next, the block 320 according to the embodiment of the present invention will be further described.

[0045] Our goal is to obtain the following expressions using the encoder of block 310. (a) Input data belonging to different classes can be well distinguished in the latent space (labeled data is used to achieve this), (b) the learned representation can be further reinforced by the unlabeled data of the training set.

[0046] To achieve these objectives, semi-supervised learning is used to learn these representations as follows.

Number

[0047] For the labeled data, cross-entropy loss is used for learning the representation. Specifically, a linear layer is added after the LSTM210, and the linear layer serves as a classifier for the latent space representation. Here, both the LSTM210 and the linear layer are trainable. Then, for each data X and its label Y, the training loss is as follows.

Number

Number

[0048] For unlabeled data, self-supervised learning is used to reinforce the representations learned from the labeled data. Specifically, the N-pair contrastive loss is used for self-supervised learning as follows.

Number

[0049] Once the formal loss and the self-supervised loss are identified, the neural network is optimized by finding the optimal θ that minimizes the training loss L(θ).

[0050] Next, block 330 according to an embodiment of the present invention will be further described.

[0051] When the LSTM210 and its subsequent linear layer are trained, the linear layer is removed and the encoder is modified. Thereafter, when input time series data is given, the final hidden state of the LSTM210 becomes a semi-supervised latent space representation, which becomes the input feature vector used to train the K-1 binary classifiers 230.

[0052] Next, block 340 according to an embodiment of the present invention will be further described.

[0053] When the expressive learning encoder network is trained, the latent space representation 220 is used as a feature vector for training K-1 binary classifiers 230 (K is the number of classes). This can be achieved through the following procedure.

[0054] First, for the k-th binary classifier 230, re-label each data that was previously labeled in the training set. Specifically, for the i-th data of the labeled training data and its corresponding label, let them be (x i , y i ).

Number

Number

[0055] By doing so, the training data

Number

[0056] Next, when the training data corresponding to each binary classifier is re-labeled, train the K-1 classifiers 230 using a formal loss. Specifically

Number

Number

Number

[0057] Next, block 350 according to an embodiment of the present invention will be further described.

[0058] When the LSTM 210 and the binary classifier 230 are trained, for the given test data (x, y), the binary prediction value

Number

Number

Number

Number

[0059] Next, block 360 according to an embodiment of the present invention will be further described.

[0060] As introduced in block 350, when certain patterns appear, conflicting binary predictions can be detected. And these conflicting binary predictions can be reversed, and since the prediction is different from the adjacent prediction, the position can be identified. Then, the conflicting prediction is reversed so that the corrected prediction is the same as its neighboring prediction. For example, when K = 7 and k = 4, the correct binary prediction is [1, 1, 1, 1, 0, 0, 0, 0], and [1, 0, 1, 1, 0, 0, 0, 0] indicates a conflicting prediction (the second prediction).

[0061] Next, block 370 according to an embodiment of the present invention will be further described.

[0062] When the binary prediction is checked and a conflict is confirmed, it is corrected and then aggregated as follows to obtain the final ordered prediction.

Number

[0063] That is, the final predicted value is the count of the predicted values indicating that the label is greater than the current index k.

[0064] FIG. 4 is a block diagram showing an exemplary processing flow 400 including alternatives according to an embodiment of the present invention.

[0065] In block 410, time series encoding is performed. The encoding can include any of a recurrent neural network (RNN) 410A, a gated recurrent network (GRU) 410B, and / or a long short-term memory 410C.

[0066] In block 420, optimization of the encoder is performed. The optimization can include any of a supervised loss 420A, a supervised triplet loss 420A1, a supervised cross-entropy loss 420A2, an unsupervised loss 420B, and an unsupervised N-pair contrastive loss 420B1.

[0067] In block 430, semi-supervised learning is performed to obtain a representation.

[0068] In block 440, binary classification is performed. The classification can include any of cross-entropy loss 440A, binary cross-entropy (BCE) loss 440B, and focal loss 440C.

[0069] In block 450, error identification / correction is performed.

[0070] In block 460, binary prediction aggregation is performed.

[0071] FIGS. 5 and 6 are flowcharts showing another exemplary method 500 for ordinal classification according to an embodiment of the present invention. Method 500 provides additional details to the higher-level method 300.

[0072] In block 510, time series data is encoded by a time encoder to obtain a latent space representation.

[0073] In one embodiment, block 510 can include block 510A.

[0074] In block 510A, an LSTM (Long Short-Term Memory) is configured as a time encoder that encodes time series data from a high-dimensional space exceeding x dimensions to a low-dimensional space below y dimensions (where x and y are integers and x > y).

[0075] In block 520, labeled data is used to distinguish different classes within the labeled space, unlabeled training data is used to reinforce the latent space representation, and semi-supervised learning is used to optimize the time encoder to obtain a semi-supervised representation.

[0076] In block 530, discard the linear layer after the time encoder and modify the time encoder. In one embodiment, the linear layer can be a classifier for the latent space representation. In one embodiment, both the time encoder and the linear layer are trainable.

[0077] In block 540, train k - 1 binary classifiers on the semi-supervised representation to obtain k - 1 binary predictions.

[0078] In one embodiment, block 540 can include one or more of blocks 540A to 540C.

[0079] In block 540A, train the k - 1 classifiers using a formal loss.

[0080] In block 540B, train the k - 1 classifiers using a softmax cross-entropy loss.

[0081] In block 540C, train the k - 1 classifiers using a binary cross-entropy loss.

[0082] In block 550, identify and correct conflicting ones among the k - 1 binary predictions by matching them with non-conflicting ones among the k - 1 binary predictions.

[0083] In one embodiment, block 550 can include one or more of blocks 550A and 550B.

[0084] In block 550A, search for a sequence of 1s containing unexpected 0s and a sequence of 0s containing unexpected 1s.

[0085] In block 550B, remove unexpected 0s and 1s respectively from the sequence of 1s and the sequence of 0s.

[0086] In block 560, aggregate the k - 1 binary predictions to obtain an ordered prediction value.

[0087] FIG. 7 is a block diagram showing an exemplary environment 700 to which the present invention can be applied according to an embodiment of the present invention.

[0088] In environment 700, user 788 is located in a scene with a plurality of objects 799, each having its own position and trajectory. User 788 is driving a vehicle 772 (e.g., a car, truck, motorcycle, etc.) having ADAS 777.

[0089] ADAS 777 receives an order prediction value.

[0090] In response to the order prediction value, a vehicle control decision is made. To that end, ADAS 777 can control, for example, but not limited to, steering, braking, and acceleration systems as operations corresponding to the decision.

[0091] Thus, in the situation of ADAS, all of steering, acceleration / braking, friction (or lack of friction), yaw rate, lighting (hazard, high beam flashing, etc.), tire pressure, turn signaling, etc. can be efficiently utilized in the optimization determination according to the present invention.

[0092] The system of the present invention (e.g., system 777) can interface with a user through one or more systems of a vehicle 772 that the user is operating. For example, the system of the present invention can provide user information via a system 772A of the vehicle 772 (e.g., a display system, a speaker system, and / or any other system). Further, the system of the present invention (e.g., system 777) can interface with the vehicle 772 itself (e.g., through one or more systems of the vehicle 772 including, but not limited to, a steering system, a braking system, an acceleration system, a steering system, a lighting (turn signal, headlight) system, etc.) to control the vehicle and cause one or more operations to be performed on the vehicle 772. In this way, the user or the vehicle 772 itself can navigate around these objects 799 and avoid potential collisions therebetween. The provision of information and / or the control of the vehicle can be considered as operations determined according to embodiments of the present invention.

[0093] Although described with respect to ADAS, the present invention can be applied to numerous applications including, for example, trajectories. For example, navigation including, for example, autonomous agents, robots, assistive technologies for visually impaired persons, etc. can be utilized according to embodiments of the present invention.

[0094] The present invention can be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product can include a computer-readable storage medium (or media) having thereon computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0095] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes the following. Portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards and raised structures in grooves having instructions recorded thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein should not be construed to be a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted via a wire.

[0096] The computer-readable program instructions described herein can be downloaded to respective computing / processing devices from a computer-readable storage medium or can be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may be composed of copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in the computer-readable storage medium within each respective computing / processing device.

[0097] The computer-readable program instructions for carrying out the operations of the present invention may be source code or object code written in any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages such as SMALLTALK®, C++, conventional procedural programming languages such as the "C" programming language, or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), and the connection may be made through an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may utilize the state information of the computer-readable program instructions to personalize the electronic circuit and execute the computer-readable program instructions to carry out aspects of the present invention.

[0098] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0099] These computer-readable program instructions are provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus create means for implementing the functions / operations specified in the flowchart and / or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the instructions stored in the computer-readable storage medium constitute an article of manufacture including instructions for implementing the aspects of the functions / operations specified in the flowchart and / or block diagram block or blocks.

[0100] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / operations specified in the flowchart and / or block diagram block or blocks.

[0101] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0102] In the specification, references to "one embodiment" or "an embodiment" of the present invention and other variations thereof mean that the specific features, structures, characteristics, etc. described in connection with the embodiment are included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout this specification and any other variations thereof do not necessarily all refer to the same embodiment.

[0103] For example, in the case of "A / B", the use of any of the following, such as " / ", "and / or", "at least one of A and B", is intended to include the selection of only the first-listed option (A), or only the second-listed option (B), or the selection of both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such expressions are intended to include the selection of only the first-listed option (A), or only the second-listed option (B), or only the third-listed option (C), or the selection of only the first and second-listed options (A and B), the selection of only the first and third-listed options (A and C), the selection of only the second and third-listed options (B and C), or the selection of all three options (A and B and C). This can be extended for as many items as are described, as will be readily understood by one of ordinary skill in the art in the present technology and related arts.

[0104] The above is to be understood as illustrative and exemplary in all respects and not restrictive, and the scope of the invention disclosed herein is determined from the claims construed in accordance with the full breadth permitted by patent law, rather than from the detailed description. The embodiments shown and described herein are merely illustrative of the invention, and it is to be understood that those skilled in the art can make various modifications without departing from the scope and spirit of the invention. Those skilled in the art can implement various other combinations of features without departing from the scope and spirit of the invention. Thus, while the aspects of the invention have been described with the particularity and detail required by patent law, those who desire to be claimed and protected by a patent are as set forth in the appended claims.

Claims

**Claim 1** A computer-implemented method for sequential prediction, comprising: encoding time series data with a time encoder to obtain a latent space representation; optimizing the time encoder using semi-supervised learning to distinguish different classes in a labeled space using labeled data and reinforce the latent space representation using unlabeled training data to obtain a semi-supervised representation; discarding a linear layer after the time encoder and modifying the time encoder; training k-1 binary classifiers on the semi-supervised representation to obtain k-1 binary prediction values; identifying and correcting conflicting ones among the k-1 binary prediction values by matching them with non-conflicting ones among the k-1 binary prediction values; aggregating the k-1 binary prediction values to obtain a sequential prediction. **Claim 2** The computer-implemented method according to claim 1, wherein the time encoder consists of an LSTM (Long Short-Term Memory) that encodes the time series data from a high-dimensional space with more than x dimensions to a low-dimensional space with less than y dimensions, where x and y are integers and x > y. **Claim 3** The computer-implemented method according to claim 1, wherein the linear layer is a classifier for the latent space representation. **Claim 4** The computer-implemented method according to claim 1, wherein the time encoder and the linear layer are both trainable. **Claim 5** The computer-implemented method according to claim 1, further comprising training the k-1 classifiers using a formal loss. **Claim 6** The computer-implemented method according to claim 5, wherein the formal loss is selected from the group consisting of softmax cross-entropy loss and binary cross-entropy loss. **Claim 7** The computer-implemented method according to claim 1, wherein the identifying step comprises searching for a sequence of 1s having unexpected 0s therein and a sequence of 0s having unexpected 1s therein. **Claim 8** The computer-implemented method according to claim 7, wherein the correcting step comprises removing unexpected 0s and unexpected 1s from each of the sequence of 1s and the sequence of 0s. **Claim 9** The computer-implemented method according to claim 1, further comprising automatically controlling a vehicle system for collision avoidance in response to an order prediction that predicts an imminent collision.

10. A computer program for causing a computer to execute for order prediction, the computer program causing the computer to encode time series data with a time encoder by a hardware processor of the computer to obtain a latent space representation; optimize the time encoder using semi-supervised learning by the hardware processor to distinguish different classes in a labeled space using labeled data and reinforce the latent space representation using unlabeled training data to obtain a semi-supervised representation; discard a linear layer after the time encoder and modify the time encoder by the hardware processor; train k-1 binary classifiers on the semi-supervised representation by the hardware processor to obtain k-1 binary prediction values; identify and correct conflicting ones among the k-1 binary prediction values by matching them with non-conflicting ones among the k-1 binary prediction values by the hardware processor; A computer program that causes a computer to execute a method including aggregating the k-1 binary prediction values to obtain an order prediction by the hardware processor.

11. The computer program according to claim 10, wherein the time encoder consists of an LSTM (Long Short-Term Memory) that encodes the time series data from a high-dimensional space exceeding x dimensions to a low-dimensional space below y dimensions, where x and y are integers and x > y.

12. The computer program according to claim 10, wherein the linear layer is a classifier for the latent space representation.

13. The computer program according to claim 10, wherein the time encoder and the linear layer are both trainable.

14. The computer program according to claim 10, wherein the method further includes training the k-1 classifiers using a formal loss.

15. The computer program according to claim 14, wherein the formal loss is selected from the group consisting of softmax cross-entropy loss and binary cross-entropy loss.

16. The computer program according to claim 10, wherein the specifying step searches for a sequence of 1s having unexpected zeros therein and a sequence of 0s having unexpected 1s therein.

17. The computer program according to claim 16, wherein the correcting step removes unexpected zeros and unexpected 1s from each of the sequence of 1s and the sequence of 0s.

18. The computer program according to claim 10, wherein the method further includes automatically controlling a vehicle system for collision avoidance in response to a sequential prediction that predicts an imminent collision.

19. A computer processing system for sequential prediction, a memory device for storing program code, and a processor device operably coupled to the memory device, the processor device encoding time series data with a time encoder to obtain a latent space representation, optimizing the time encoder using semi-supervised learning to distinguish different classes in a labeled space using labeled data and reinforcing the latent space representation using unlabeled training data to obtain a semi-supervised representation, discarding a linear layer after the time encoder and modifying the time encoder, training k - 1 binary classifiers on the semi-supervised representation to obtain k - 1 binary prediction values, identifying and correcting conflicting ones of the k - 1 binary prediction values by matching them with non-conflicting ones of the k - 1 binary prediction values, a processor device for executing program code for aggregating the k - 1 binary prediction values to obtain a sequential prediction.

20. The computer processing system according to claim 19, wherein the time encoder consists of an LSTM (Long Short-Term Memory) that encodes the time series data from a high-dimensional space of more than x dimensions to a low-dimensional space of less than y dimensions, where x and y are integers and x > y.

Images