Prediction Method, Device, Equipment and Medium Based on Multimodal Extreme Learning Machine

Abstract

The present application discloses a prediction method, device, equipment and medium based on a multi-modal extreme learning machine. The method includes: obtaining training data and neuron weight parameters under a probability distribution, and constructing a label vector corresponding to the training data; constructing a number of intermediate neurons under each probability distribution according to each neuron weight parameter; constructing a composite feature of the training data under each probability distribution according to each number of intermediate neurons; calculating a second-order sample feature corresponding to the composite feature, and constructing a kernel matrix corresponding to the second-order sample feature; obtaining a sample to be predicted, and predicting the sample to be predicted according to an extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result. The present invention can be applied to solve data-driven modeling problems, such as image classification, sequence prediction, geophysics, credit evaluation, etc. The present application solves the technical problem of unstable accuracy of the extreme learning machine model caused by random mapping in the prior art.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a prediction method, device, equipment and medium based on a multi-modal extreme learning machine. Background Art

[0002] With the continuous development of machine learning, various machine learning methods have emerged at present. Among them, the extreme learning machine model can be used to solve classification and regression problems. However, the random mapping of the extreme learning machine model has always been a difficult problem to solve, that is, the hidden layers obtained by sampling under different probability distributions during the training process are different, which in turn leads to different prediction performances. In the training process of the current extreme learning machine model, the probability distribution usually relies on experience for manual selection, which has great uncertainty, thus affecting the stability of the accuracy of the extreme learning machine model. The present invention can be applied to solve data-driven modeling problems, such as image classification, sequence prediction, geophysics, credit evaluation, and so on.

[0003] In the invention patent with the application number CN201810811670.X, a computer-aided reference system and method for fusing multi-modal breast images are disclosed. It concatenates and fuses the extracted GLCM, LBP, HIS texture features and CNN depth features to construct a texture-depth fusion feature model. Here, the multiple texture features are multi-modal, which is suitable for fields that require prior feature extraction such as image classification. However, in many fields, such as well logging interpretation, the input is features composed of well logging curve values with actual physical meanings, so it is impossible to extract multiple features. In the invention patent with the application number CN202110418142.X, a dynamic service demand prediction method and system based on multi-modal machine learning are disclosed. It respectively extracts features from text data and image data and shares the features to obtain a user service expression vector. Here, the multi-modal is generated by two data forms of text and image. In the related invention patents disclosed in the application numbers CN202110470704.5 and CN202011191238.9, there are also technical features related to multi-modal. However, from the publicly available technical literature, its multi-modal machine learning is aimed at multi-source heterogeneous data itself. In fact, the uncertainty of feature mapping will also bring multiple modalities. If a wrong modality is selected, it will lead to a decrease in learning accuracy. Therefore, it is necessary to fully fuse multi-modal to eliminate the instability of accuracy. Summary of the Invention

[0004] The main purpose of this application is to provide a prediction method, device, equipment and medium based on a multi-modal extreme learning machine, aiming to solve the technical problem of unstable accuracy caused by random mapping in the existing extreme learning machine model.

[0005] To achieve the above object, the present application provides a prediction method based on a multi-modal extreme learning machine, and the prediction method based on the multi-modal extreme learning machine includes:

[0006] Obtain training data and neuron weight parameters under each probability distribution, and construct a label vector corresponding to the training data;

[0007] According to each of the neuron weight parameters, construct a number of intermediate neurons under each probability distribution;

[0008] According to each of the number of intermediate neurons, construct composite features of the training data under each probability distribution;

[0009] Calculate second-order sample features corresponding to the composite features, and construct a kernel matrix corresponding to the second-order sample features;

[0010] Obtain a sample to be predicted, and predict the sample to be predicted according to an extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result.

[0011] The present application also provides a prediction device based on a multi-modal extreme learning machine, and the prediction device based on the multi-modal extreme learning machine includes:

[0012] An acquisition module, configured to obtain training data and neuron weight parameters under each probability distribution, and construct a label vector corresponding to the training data;

[0013] An intermediate neuron construction module, configured to construct a number of intermediate neurons under each probability distribution according to each of the neuron weight parameters;

[0014] A composite feature construction module, configured to construct composite features of the training data under each probability distribution according to each of the number of intermediate neurons;

[0015] A kernel matrix construction module, configured to calculate second-order sample features corresponding to the composite features, and construct a kernel matrix corresponding to the second-order sample features;

[0016] A prediction module, configured to obtain a sample to be predicted, and predict the sample to be predicted according to an extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result.

[0017] The present application also provides an electronic device, and the electronic device includes: a memory, a processor, and a program of the prediction method based on the multi-modal extreme learning machine stored on the memory and executable on the processor, and when the program of the prediction method based on the multi-modal extreme learning machine is executed by the processor, the steps of the prediction method based on the multi-modal extreme learning machine as described above can be implemented.

[0018] The present application also provides a computer-readable storage medium, on which a program for implementing a prediction method based on a multi-modal extreme learning machine is stored. When the program for the prediction method based on the multi-modal extreme learning machine is executed by a processor, the steps of the prediction method based on the multi-modal extreme learning machine as described above are implemented.

[0019] The present application also provides a computer program product, including a computer program, which implements the steps of the prediction method based on the multi-modal extreme learning machine as described above when executed by a processor.

[0020] The present application provides a prediction method, device, equipment and medium based on a multi-modal extreme learning machine. Compared with the prior art in which the probability distribution is usually manually selected based on experience during the training process of the current extreme learning machine model, the present application obtains training data and neuron weight parameters under each probability distribution, and constructs a label vector corresponding to the training data; according to each of the neuron weight parameters, a number of intermediate neurons under each probability distribution are constructed, achieving the purpose of constructing hidden layers under different probability distributions. Furthermore, according to each of the number of intermediate neurons, composite features of the training data under each probability distribution are constructed; second-order sample features corresponding to the composite features are calculated, and a kernel matrix corresponding to the second-order sample features is constructed, achieving the purpose of fusing a number of neurons (hidden layers) under different probability distributions into a kernel matrix by constructing composite features, making the kernel matrix more stable and reliable, and making the extreme learning machine model jointly constructed based on the kernel matrix and the label vector more stable and reliable. Furthermore, a sample to be predicted is obtained, and the sample to be predicted is predicted according to the extreme learning machine model jointly constructed based on the kernel matrix and the label vector to obtain a prediction result. The purpose of prediction based on a more stable and reliable extreme learning machine model can be achieved, overcoming the technical defect in the prior art that the probability distribution usually depends on manual selection based on experience during the training process of the current extreme learning machine model, which has great uncertainty and thus affects the stability of the accuracy of the extreme learning machine model, and solving the technical problem of unstable accuracy caused by random mapping in the extreme learning machine model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 This is a schematic flowchart of the first embodiment of the prediction method based on a multi-modal extreme learning machine according to the present application;

[0024] Figure 2 This is a schematic diagram of the device structure of the hardware operating environment involved in the prediction method based on a multi-modal extreme learning machine in the embodiments of the present application.

[0025] The implementation, functional features, and advantages of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific Embodiments

[0026] To make the above objects, features, and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] The embodiments of the present application provide a prediction method based on a multi-modal extreme learning machine, which is applied to a federal participant. In the first embodiment of the prediction method based on a multi-modal extreme learning machine according to the present application, with reference to Figure 1 , the prediction method based on a multi-modal extreme learning machine includes:

[0028] Step S10: Obtain training data and neuron weight parameters under each probability distribution, and construct a label vector corresponding to the training data;

[0029] Step S20: Construct a number of intermediate neurons under each probability distribution according to each neuron weight parameter;

[0030] Step S30: Construct composite features of the training data under each probability distribution according to each of the intermediate neurons;

[0031] Step S40: Calculate second-order sample features corresponding to the composite features, and construct a kernel matrix corresponding to the second-order sample features;

[0032] Step S50: Obtain a sample to be predicted, and predict the sample to be predicted according to the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result.

[0033] In this embodiment, it should be noted that the training data is used to construct an extreme learning machine model, and the extreme learning machine model is a neural network model. The neural network model includes at least one neuron. Among them, the neuron is the basic unit that composes the neural network; the neuron weight parameter can be the input weight vector corresponding to the neuron and the input bias coefficient corresponding to the neuron. The training data includes at least one training sample, and one training sample corresponds to a sample label. The label training is constructed based on the sample labels corresponding to each training sample; the probability distribution is the probability distribution of the neuron weight parameter. Among them, there are multiple groups of input weight vectors and input bias coefficients, and the input weight vectors and input bias coefficients in each group respectively conform to the corresponding probability distribution. The probability distribution can be a normal distribution.

[0034] Steps S10 to S50 include: obtaining each training sample and the input weight vectors and input bias coefficients under each probability distribution, constructing a corresponding label vector according to the sample label corresponding to each training sample; according to the definition formula of the neuron, the input weight vectors and input bias coefficients under each probability distribution, constructing a number of intermediate neurons under each probability distribution, where the number of intermediate neurons constitutes the hidden layer of the neural network; by inputting the training samples into each of the neurons respectively, calculating the output features of each training sample under each probability distribution, and fusing the output features into a composite feature; calculating the second-order feature corresponding to the composite feature to obtain the second-order sample feature corresponding to the composite feature; according to the second-order sample feature, calculating the kernel matrix of the extreme learning machine model, and constructing the extreme learning machine model according to the kernel matrix and the label vector; obtaining the sample to be predicted, inputting the sample to be predicted into the extreme learning machine model, and outputting the classification label corresponding to the sample to be predicted to predict the sample to be predicted, and using the classification label as the prediction result.

[0035] As an example, the specific implementation process of steps S10 to S50 is as follows:

[0036] S1: Collect a training data set. The training data set includes at least one training sample, and the training sample d is the initial feature dimension of the training sample, and the sample label corresponding to the training sample is represents the real number field. Suppose there are n training samples collected, then the training data set is x i is the i-th training sample, and y i is the label corresponding to x i , and the label vector is defined as y = [y1,..., y n T ; ​

[0037] The following is an example to illustrate the actual meaning of the dataset:

[0038] Taking image classification as an example, the dataset has n images. That is, the feature vectors of the images can be extracted as samples through common feature extraction methods such as SIFT (Scale-invariant feature transform). Suppose there are c classification targets, then the sample x i corresponds to the label y i ∈ {1, …, c}. If one-hot encoding is adopted, y i can be written as a c-dimensional vector y i , and the corresponding label vector y needs to be written in the form of a label matrix, that is, Y = [y1; …; y n ; If operations need to be performed in the form of one-hot encoding, y can be replaced by Y in the relevant operations involved later. This task is a classification task.

[0039] Taking well logging interpretation as an example, the well logging curve values at each depth form a feature vector, that is, a sample is constructed. If there are n depth points in a well, then n samples are obtained, and their corresponding labels are geological information, such as porosity. This task is a regression task.

[0040] S2: Set m different probability distributions, that is, m different probability distributions. Set the number of intermediate neurons z > 0, and set the dimension r > 0 of the sample after dimensionality reduction; Here, the m probability distributions can be set as normal distributions with different means and variances, or can be set as various probability distributions such as uniform distribution, Bernoulli distribution, binomial distribution, geometric distribution, etc.;

[0041] S3: Randomly generate z input weight vectors and input bias coefficients denoted as w1, …, w z and b1, …, b z , and then according to the m different probability distributions set in advance, m groups

[0042] {(w i , b i ), i = 1, ..., z}

[0043] Furthermore, define

[0044]

[0045] Then the i-th intermediate neuron excited by the training sample x under the j-th probability distribution is denoted as

[0046]

[0047] where and respectively represent the i-th input weight vector and the input bias coefficient randomly generated under the j-th distribution; φ(x) represents the intermediate neuron;

[0048] S4: Calculate the composite feature of the sample x as:

[0049]

[0050] where the i-th column vector of A(x) Furthermore, calculate the second-order feature of the sample x, that is:

[0051]

[0052] where ‖·‖2 represents the 2-norm;

[0053] S5: Calculate the kernel matrix The element in its i-th row and j-th column

[0054]

[0055] where represents the Hadamard product, and are randomly generated matrices, whose elements are 1 or -1, and the probabilities of generating positive and negative 1 are the same;

[0056] S6: Input the sample to be predicted into:

[0057]

[0058] where represents the Moore-Penrose generalized inverse of, where the i-th element of is:

[0059]

[0060] That is the prediction result of, and f is the extreme learning machine model.

[0061] Among them, it should be noted that the extreme learning machine model can be a machine learning model for image classification, that is, an image classification model. Steps S10 to S50 include: obtaining training image data and neuron weight parameters under each probability distribution, and constructing a label vector corresponding to the training image data; constructing a number of intermediate neurons under each probability distribution according to each of the neuron weight parameters; constructing composite image features of the training image data under each probability distribution according to each of the number of intermediate neurons; calculating second-order image features corresponding to the composite image features, and constructing a kernel matrix corresponding to the second-order image features; obtaining an image to be predicted, and performing image classification on the image to be predicted according to the image classification model jointly constructed by the kernel matrix and the label vector to obtain an image classification result. Among them, the training image data includes at least one image as a training sample. Furthermore, in the embodiments of the present application, the purpose of constructing the hidden layer of the image classification model under different probability distributions is to fuse a number of neurons (hidden layers) under different probability distributions into the kernel matrix of the image classification model by constructing composite image features, making the kernel matrix more stable and reliable. Furthermore, performing image classification according to the image classification model constructed based on the more stable and reliable kernel matrix can avoid the situation that in the training process of the current extreme learning machine model, the probability distribution usually relies on experience for manual selection, which has great uncertainty and thus affects the stability of the image classification accuracy of the extreme learning machine model, and can improve the accuracy stability of image classification using the extreme learning machine model.

[0062] The embodiment of the present application provides a prediction method based on a multi-modal extreme learning machine. Compared with the prior art in which the probability distribution is usually manually selected based on experience during the training process of the current extreme learning machine model, the embodiment of the present application obtains training data and neuron weight parameters under each probability distribution, and constructs a label vector corresponding to the training data; according to each neuron weight parameter, a number of intermediate neurons are constructed under each probability distribution, achieving the purpose of constructing hidden layers under different probability distributions. Furthermore, according to each of the number of intermediate neurons, composite features of the training data under each probability distribution are constructed; second-order sample features corresponding to the composite features are calculated, and a kernel matrix corresponding to the second-order sample features is constructed, achieving the purpose of fusing a number of neurons (hidden layers) under different probability distributions into a kernel matrix by constructing composite features, making the kernel matrix more stable and reliable, and making the extreme learning machine model jointly constructed based on the kernel matrix and the label vector more stable and reliable. Then, a sample to be predicted is obtained, and the sample to be predicted is predicted based on the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result. It can achieve the purpose of prediction based on a more stable and reliable extreme learning machine model, overcome the technical defect in the prior art that the probability distribution usually relies on experience for manual selection during the training process of the current extreme learning machine model, which has great uncertainty and thus affects the stability of the accuracy of the extreme learning machine model, and solves the technical problem of unstable accuracy caused by random mapping in the extreme learning machine model.

[0063] The embodiment of the present application also provides a prediction device based on a multi-modal extreme learning machine. The prediction device based on a multi-modal extreme learning machine includes:

[0064] An acquisition module, configured to acquire training data and neuron weight parameters under each probability distribution, and construct a label vector corresponding to the training data;

[0065] An intermediate neuron construction module, configured to construct a number of intermediate neurons under each probability distribution according to each neuron weight parameter;

[0066] A composite feature construction module, configured to construct composite features of the training data under each probability distribution according to each of the number of intermediate neurons;

[0067] A kernel matrix construction module, configured to calculate second-order sample features corresponding to the composite features, and construct a kernel matrix corresponding to the second-order sample features;

[0068] A prediction module, configured to obtain a sample to be predicted, and predict the sample to be predicted based on the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result.

[0069] Optionally, the training data includes at least one training sample, and the intermediate neuron construction module is further configured to:

[0070] Use the following formula to construct a number of intermediate neurons under each of the probability distributions according to the neuron weight parameters:

[0071]

[0072] where φ(x) is the intermediate neuron, x is the training sample, w is the input weight vector, b is the input bias coefficient, d is the feature dimension of the training sample;

[0073] The i-th neuron excited by the training sample x under the j-th probability distribution is denoted as

[0074]

[0075] where, and respectively represent the i-th input weight vector generated under the j-th probability distribution and the i-th input bias coefficient generated.

[0076] Optionally, the training data includes at least one training sample, and the composite feature construction module is further configured to:

[0077] Use the following formula to construct the composite feature of the training data under each of the probability distributions according to the intermediate neurons:

[0078]

[0079] where A(x) is the composite feature, is the i-th neuron excited by the training sample x under the m-th probability distribution, and z is the number of neuron weight parameters under the probability distribution.

[0080] Optionally, the training data includes at least one training sample, and the kernel matrix construction module is further configured to:

[0081] Use the following formula to calculate the second-order sample feature corresponding to the composite feature:

[0082]

[0083] where h(x) is the second-order sample feature, A(x) is the composite feature, m is the number of probability distributions, and x is the training sample.

[0084] Optionally, the training data includes at least one training sample, and the kernel matrix construction module is further configured to:

[0085] Construct the kernel matrix corresponding to the second-order sample features by using the following formula:

[0086] The kernel matrix The element in the i-th row and j-th column

[0087]

[0088] where represents the Hadamard product, and are randomly generated matrices with elements being 1 or -1, and the probabilities of generating positive and negative 1 are the same, h(x) is the second-order sample feature, x is the training sample, r << m 2 , and r is the dimension after dimensionality reduction of the training sample.

[0089] Optionally, the prediction module is further configured to:

[0090] Predict the sample to be predicted by using the following formula based on the extreme learning machine model jointly constructed by the kernel matrix and the label vector, and obtain a prediction result:

[0091]

[0092] where represents the Moore-Penrose generalized inverse of the kernel matrix , y is the label vector, is the sample to be predicted, f is the extreme learning machine model, where

[0093]

[0094] where is the prediction result corresponding to the sample to be predicted x n The n-th sample feature in the sample to be predicted, is expressed as the second-order sample feature corresponding to the sample to be predicted, h(x i ) is the second-order feature corresponding to the i-th feature in the sample to be predicted, represents the Hadamard product, and are randomly generated matrices with elements being 1 or -1, and the probabilities of generating positive and negative 1 are the same, r << m 2 , and r is the dimension after dimensionality reduction of the training sample.

[0095] Optionally, the training data includes at least one training sample, and the obtaining module is further configured to:

[0096] obtain the sample label corresponding to each of the training samples;

[0097] transpose the vector formed by each of the sample labels to obtain the label vector, where the formula for calculating the label vector is as follows:

[0098] y = [y1,..., y n T

[0099] where y is the label vector, and y1 to y n are all the sample labels.

[0100] The prediction device based on the multi-modal extreme learning machine provided by the present invention adopts the prediction method based on the multi-modal extreme learning machine in the above embodiment, and solves the technical problem of unstable accuracy of the extreme learning machine model caused by random mapping. Compared with the prior art, the beneficial effects of the prediction device based on the multi-modal extreme learning machine provided by the embodiment of the present invention are the same as those of the prediction method based on the multi-modal extreme learning machine provided by the above embodiment, and other technical features in the prediction device based on the multi-modal extreme learning machine are the same as the features disclosed in the method of the above embodiment, and will not be described in detail here.

[0101] An embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the prediction method based on the multi-modal extreme learning machine in the first embodiment above.

[0102] Next, refer to Figure 2 , which shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 2 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0103] As Figure 2 ​As shown, an electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0104] Generally, the following systems may be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and a communication device. The communication device may allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an electronic device having various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be alternatively implemented or had.

[0105] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from a storage device, or installed from the ROM. When the computer program is executed by the processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0106] The electronic device provided by the present invention adopts the prediction method based on a multi-modal extreme learning machine in the above embodiment, and solves the technical problem of unstable accuracy of the extreme learning machine model caused by random mapping. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present invention are the same as those of the prediction method based on the multi-modal extreme learning machine provided in the above embodiment, and other technical features in the electronic device are the same as those disclosed in the method of the above embodiment, which will not be elaborated herein.

[0107] It should be understood that each part of the present disclosure may be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0108] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims described above.

[0109] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the method for prediction based on multi-modal extreme learning in the first embodiment above.

[0110] The computer-readable storage medium provided by the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device or component. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0111] The above computer-readable storage medium may be included in an electronic device; or it may exist separately without being assembled into the electronic device.

[0112] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device is enabled to: obtain training data and neuron weight parameters under each probability distribution, and construct a label vector corresponding to the training data; construct a number of intermediate neurons under each probability distribution according to each neuron weight parameter; construct a composite feature of the training data under each probability distribution according to each of the number of intermediate neurons; calculate a second-order sample feature corresponding to the composite feature, and construct a kernel matrix corresponding to the second-order sample feature; obtain a sample to be predicted, and predict the sample to be predicted according to the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0115] The modules described in the embodiments of the present disclosure may be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0116] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for performing the above-mentioned prediction method based on a multi-modal extreme learning machine, and solves the technical problem of unstable accuracy of the extreme learning machine model due to random mapping. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiments of the present invention are the same as those of the above-mentioned prediction method based on a multi-modal extreme learning machine, and will not be elaborated herein.

[0117] The present application also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the prediction method based on a multi-modal extreme learning machine as described above.

[0118] The computer program product provided by the present application solves the technical problem of unstable accuracy caused by random mapping in the extreme learning machine model. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiments of the present invention are the same as those of the prediction method based on a multi-modal extreme learning machine provided in the above embodiments, and will not be elaborated here.

[0119] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent scope of the present application.

Claims

1. A prediction method based on a multi-modal extreme learning machine, characterized in that, The prediction method based on the multi-modal extreme learning machine includes: Obtain training data and neuron weight parameters under each probability distribution, and construct a label vector corresponding to the training data, where the training data includes at least training samples for image classification; Use the following formula to construct a number of intermediate neurons under each probability distribution according to each neuron weight parameter, where the neuron weight parameters include an input weight vector and an input bias coefficient: Among them, φ(x) is the intermediate neuron, x is the training sample, w is the input weight vector, and b is the input bias coefficient. d is the feature dimension of the training sample. The i-th neuron excited by the training sample x under the j-th probability distribution is denoted as wherein, and respectively represent the i-th input weight vector and the i-th input bias coefficient generated under the j-th probability distribution; Use the following formula to construct composite features of the training data under each probability distribution according to each of the number of intermediate neurons: Among them, A(x) is the composite feature, is the i-th neuron activated under the m-th probability distribution of the training sample x, and z is the number of the neuron weight parameters under the probability distribution; Use the following formula to calculate the second-order sample features corresponding to the composite features: where h(x) is the second-order sample feature, A(x) is the composite feature, m is the number of the probability distributions, x is the training sample, and the kernel matrix corresponding to the second-order sample feature is constructed by using the following formula: The nuclear matrix The element in the i-th row and j-th column where, ° represents the Hadamard product, and are randomly generated matrices, the elements of which are 1 or -1, and the probabilities of generating positive and negative 1 are the same, h(x) is the second-order sample feature, x is the training sample, r << m 2 , and r is the dimension after dimensionality reduction of the training sample; Obtain a sample to be predicted, and predict the sample to be predicted based on the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result, where the extreme learning machine model is a machine learning model for image classification.

2. The prediction method based on a multi-modal extreme learning machine according to claim 1, characterized in that, Use the following formula to predict the sample to be predicted based on the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result: Among them, represents the Moore-Penrose generalized inverse of the nuclear matrix , y is the label vector, is the sample to be predicted, f is the extreme learning machine model, where Among them, is the prediction result corresponding to the sample to be predicted x, n the nth sample feature in the sample to be predicted, is expressed as the second-order sample feature corresponding to the sample to be predicted, h(x i ) is the second-order feature corresponding to the ith feature in the sample to be predicted, represents the Hadamard product, and are randomly generated matrices, the elements of which are 1 or -1, and the probabilities of generating positive and negative 1 are the same, r << m 2 , and r is the dimension after dimensionality reduction of the training samples.

3. The prediction method based on a multi-modal extreme learning machine according to claim 1, characterized in that, The training data includes at least one training sample, and the step of constructing a label vector corresponding to the training data includes: Obtain the sample labels corresponding to each training sample; Transpose the vector composed of each sample label to obtain the label vector, where the formula for calculating the label vector is as follows: where y is the label vector, and y1 to y b are all the sample labels.

4. A prediction device based on a multi-modal extreme learning machine, characterized in that, The prediction device based on the multi-modal extreme learning machine includes: An acquisition module for obtaining training data and neuron weight parameters under each probability distribution, and constructing a label vector corresponding to the training data, where the training data includes at least training samples for image classification; An intermediate neuron construction module for using the following formula to construct a number of intermediate neurons under each probability distribution according to each neuron weight parameter, where the neuron weight parameters include an input weight vector and an input bias coefficient: where φ(x) is the intermediate neuron, x is the training sample, w is the input weight vector, b is the input bias coefficient, d is the feature dimension of the training sample; The i-th neuron excited by the training sample x under the j-th probability distribution is denoted as Among them, and respectively represent the i-th input weight vector and the i-th input bias coefficient generated under the j-th probability distribution; A composite feature construction module for using the following formula to construct composite features of the training data under each probability distribution according to each of the number of intermediate neurons: where A(x) is the composite feature, is the i-th neuron activated under the m-th probability distribution of the training sample x, and z is the number of the neuron weight parameters under the probability distribution; A kernel matrix construction module for using the following formula to calculate the second-order sample features corresponding to the composite features: where h(x) is the second-order sample feature, A(x) is the composite feature, m is the number of the probability distributions, x is the training sample, and the kernel matrix corresponding to the second-order sample feature is constructed by using the following formula: The said kernel matrix The element in the i-th row and j-th column Among them, represents the Hadamard product, and are randomly generated matrices, whose elements are 1 or -1, and the probabilities of generating positive and negative 1 are the same. h(x) is the second-order sample feature, x is the training sample, and r << m 2 , and r is the dimension after dimensionality reduction of the training sample; A prediction module for obtaining a sample to be predicted, and predicting the sample to be predicted based on the extreme learning machine model jointly constructed by the kernel matrix and the label vector to obtain a prediction result, where the extreme learning machine model is a machine learning model for image classification.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; where The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the prediction method based on the multi-modal extreme learning machine according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, A program for implementing a prediction method based on a multi-modal extreme learning machine is stored on the computer-readable storage medium, and the program for implementing the prediction method based on the multi-modal extreme learning machine is executed by a processor to implement the steps of the prediction method based on the multi-modal extreme learning machine according to any one of claims 1 to 3.

Images