Long text analysis method, device and equipment based on hybrid expert model

Abstract

The invention discloses a long text analysis method, device and equipment based on a hybrid expert model. The method comprises the following steps: acquiring a long text to be analyzed; inputting the long text into a target analysis model to obtain a target attention result; the target analysis model comprises a gated attention unit improved based on a hybrid expert model; and on the basis of the target attention result, generating an analysis result corresponding to the long text, so that the model analysis accuracy can be effectively improved on the basis of improving the calculation efficiency.

Description

Technical Field

[0001] The present disclosure generally relates to the technical field of long text analysis, and specifically relates to a long text analysis method, device and equipment based on a hybrid expert model. Background Art

[0002] Currently, Transformer-based language models are widely used in the field of text analysis, such as text classification, relationship extraction, intent recognition and other natural language processing related tasks. However, since the computational complexity of Transformer-based language models is usually related to the square of the input text length, processing very long texts will result in low computational efficiency.

[0003] In related technologies, in order to solve the problem of low computational efficiency when processing ultra-long texts with Transformer-based language models, for example, linear attention is added to the model. However, although this method can reduce model complexity and improve computational efficiency, it will cause certain information loss and affect the final analysis results. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a long text analysis method, device and equipment based on a hybrid expert model, which can effectively improve the accuracy of model analysis while improving computational efficiency.

[0005] In a first aspect, an embodiment of the present application provides a long text analysis method based on a hybrid expert model, comprising:

[0006] Get the long text to be analyzed;

[0007] Inputting the long text into a target analysis model to obtain a target attention result; the target analysis model includes a gated attention unit improved based on a hybrid expert model;

[0008] Based on the target attention result, an analysis result corresponding to the long text is generated.

[0009] In some embodiments, inputting the long text into a target analysis model to obtain a target attention result includes:

[0010] Performing a radial transformation on the long text to obtain local linear transformation matrix parameters, global linear transformation parameters and text features;

[0011] Determine a local attention result based on the local linear transformation matrix and the text feature, and determine a global attention result based on the global linear transformation parameter and the text feature;

[0012] Based on the local attention result and the global attention result, the target attention result is determined.

[0013] In some embodiments, it also includes:

[0014] Acquire a plurality of expert text features determined by the hybrid expert model;

[0015] Based on the multiple expert text features, the text features used to determine the local attention result and the global attention result are determined.

[0016] In some embodiments, it also includes:

[0017] Respectively obtaining a plurality of expert attention results determined by a plurality of hybrid expert models based on the local attention results and the global attention results;

[0018] Based on the multiple expert attention results, the target attention result is determined.

[0019] In some embodiments, it also includes:

[0020] Based on the number of experts in the hybrid expert model, multiple results obtained based on the hybrid expert model are averaged.

[0021] In some embodiments, it also includes:

[0022] The target analysis model is trained using mixed precision.

[0023] In a second aspect, an embodiment of the present application provides a long text analysis device based on a hybrid expert model, comprising:

[0024] The acquisition module is used to obtain the long text to be analyzed;

[0025] An analysis module, used for inputting the long text into a target analysis model to obtain a target attention result; the target analysis model includes a gated attention unit improved based on a hybrid expert model;

[0026] A generation module is used to generate an analysis result corresponding to the long text based on the target attention result.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the program.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the embodiment of the present application.

[0029] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the method described in the embodiment of the present application.

[0030] The long text analysis method, device and equipment based on the hybrid expert model proposed in the embodiments of the present application introduce a hybrid expert model for optimization on the basis of the analysis model based on the gated attention unit. While utilizing the gated attention unit to ensure the computational efficiency of the analysis model, it can speed up the training speed and improve the analysis ability of the model in long text related tasks.

[0031] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0033] Figure 1 A schematic diagram of a process of a long text analysis method based on a hybrid expert model provided in an embodiment of the present application is shown;

[0034] Figure 2 A schematic diagram of a process of a long text analysis method based on a hybrid expert model provided in another embodiment of the present application is shown;

[0035] Figure 3 A schematic diagram showing a partial structure of a target analysis model provided by an embodiment of the present application is shown;

[0036] Figure 4 A schematic diagram showing a partial structure of a target analysis model provided by another embodiment of the present application is shown;

[0037] Figure 5 A schematic diagram of the structure of a long text analysis device based on a hybrid expert model provided in an embodiment of the present application is shown;

[0038] Figure 6 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] In order to further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation instruction steps shown in the following embodiments or drawings, more or less operation instruction steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method may be executed in the order of the method shown in the embodiment or drawings or in parallel during the actual processing process or when the device is executed.

[0042] Please refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a long text analysis method based on a hybrid expert model provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0043] Step 101, obtaining a long text to be analyzed.

[0044] It should be noted that before the long text is input into the target analysis model, the long text needs to be represented by a vector so that the subsequent target analysis model can perform analysis based on the feature vector of the long text.

[0045] Specifically, before analyzing the long text to be analyzed, an input vector corresponding to the original long text is obtained based on the original long text, wherein the input vector is a vector concatenation of three vectors: a word vector, a position vector, and a paragraph vector. For example, the word vector, the position vector, and the paragraph vector are added together to obtain the final input vector.

[0046] Step 102, input the long text into the target analysis model to obtain the target attention result; the target analysis model includes a gated attention unit improved based on the hybrid expert model.

[0047] It should be noted that, in an embodiment of the present application, the target analysis model with a gated attention unit can be a FLASH model including a gated attention unit and a linear attention. Among them, the FLASH model uses global attention and local attention to analyze the text. Specifically, when calculating the local attention, the input vector of the long text is calculated within the range of a predetermined block size, and when calculating the global attention, a method similar to linear attention is used for calculation. Since the vector block used when calculating the local attention is much smaller than the actual input length of the long text, the FLASH model can greatly reduce the computational complexity of the model by using a combination of global attention and local attention. However, although the FLASH model can reduce the complexity of the model by using global attention and local attention to analyze long texts, it will also bring about certain information loss that affects the final analysis results, such as the lack of information between blocks and the incoherence of information within blocks caused by the block mechanism when calculating the local attention, which affects the final analysis accuracy.

[0048] Therefore, the embodiment of the present application, based on the use of the FLASH model, increases the number of trainable parameters of the analysis model by introducing a hybrid expert model, and then obtains the target analysis model, so that the target analysis model can be trained faster at the same computing cost and improve the analysis capabilities of long text related tasks.

[0049] Step 103, based on the target attention result, generate the analysis result corresponding to the long text.

[0050] It should be understood that after obtaining the target attention results, the corresponding steps can be executed according to the preset rules based on the target attention results to obtain the analysis results corresponding to the long text, including but not limited to text classification, relationship extraction, intent recognition and other natural language processing related tasks.

[0051] Therefore, the long text analysis method based on the hybrid expert model proposed in the embodiment of the present application introduces a hybrid expert model for optimization on the basis of the analysis model based on the gated attention unit. It can speed up the training while utilizing the gated attention unit to ensure the computational efficiency of the analysis model, thereby improving the model's analysis capabilities in long text related tasks.

[0052] It should also be noted that the FLASH model uses gated attention units to calculate attention and obtains attention results, that is, it combines gated linear units with attention calculation. Compared with traditional BERT models that use multi-head attention, gated attention units can capture a wider range of contextual information and are more conducive to improving the effect of global attention.

[0053] In a feasible example, Figure 2As shown, in step 102, the long text is input into the target analysis model to obtain the target attention result, including:

[0054] Step 1021, perform radial transformation on the long text to obtain local linear transformation matrix parameters, global linear transformation parameters and text features.

[0055] Among them, the local linear transformation matrix parameters include Q part and k part , the global linear transformation parameters include Q whole and k whole , the text feature is V.

[0056] Specifically, the input vector corresponding to the long text is divided into blocks according to the preset text block size, and the radial transformation is performed based on the divided long text to obtain the local linear transformation matrix parameter Q corresponding to each text block. part and k part .

[0057] Preferably, in an embodiment of the present application, the text block is smaller than the maximum input length of the long text and is divisible by the maximum input length. In an embodiment of the present application, the maximum input length set by the target analysis model can be set to 9216, which can better meet the requirement of dividing most long texts into integer multiples. It should be understood that when the length of the long text exceeds the maximum input length set by the target analysis model, the long text can be truncated to meet the processing capacity of the target analysis model and ensure that the target analysis model can effectively analyze the long text. In addition, when the maximum length of the long text does not meet the maximum input length set by the target analysis model, the long text can be padded (padding) and then input into the target analysis model.

[0058] Step 1022, determining the local attention result based on the local linear transformation matrix and the text features, and determining the global attention result based on the global linear transformation parameters and the text features.

[0059] When calculating the local attention result, the local attention calculation is performed based on the above-mentioned block results. Preferably, relu is used 2 (·) is used as the activation function. Specifically, the following formula is used to calculate the local attention result A part :

[0060]

[0061] Among them, b is a trainable parameter.

[0062] When calculating the global attention result, the target analysis model adopts the linearized attention calculation method, first multiplying the global linear transformation parameters and the text features, so that the calculation complexity of the global attention is proportional to the model input length, so as to reduce the model complexity and improve the calculation efficiency, so that the calculation effect is close to the calculation effect of the standard attention. Specifically, the global attention result A is calculated using the following formula whole :

[0063]

[0064] Step 1023, determine the target attention result based on the local attention result and the global attention result.

[0065] Furthermore, the present application utilizes a hybrid expert model to optimize the above process.

[0066] In some embodiments, a hybrid expert model is used to obtain the most effective multiple text features involved in the calculation of global attention results and local attention results, so as to improve the reliability and accuracy of text features by increasing the number of parameters in the process of obtaining text features.

[0067] Specifically, the model structure is improved as follows: Figure 3 As shown, multiple expert text features determined by the hybrid expert model are obtained, and then based on the multiple expert text features, text features used to determine the local attention results and the global attention results are determined. Specifically, the following expression can be used:

[0068] V expert1 , V expert2 , ..., V expertn =ExpertNet(x)

[0069] V=Average(V expert1 , V expert2 , ..., V expertn )

[0070] Where x is the input vector of the target analysis model, and Average(·) is used to calculate n V expert The average value of .

[0071] In other instances, a hybrid expert model is used to obtain the attention result A, that is, the reliability and accuracy of the final target attention result are improved by increasing the number of parameters in the process of obtaining the attention result A as a whole.

[0072] Specifically, the model structure is improved as follows: Figure 4 As shown, multiple hybrid expert models are used to obtain multiple expert attention results based on local attention results and global attention results, and then a target attention result is determined based on the multiple expert attention results.

[0073] Specifically, the following expression can be used:

[0074] A exper , A expert2 , ..., A expertn =ExpertNet(A)

[0075] A'=Average(A expert1 , A expert2 , ..., A expertn )

[0076] Among them, Average(·) is used to calculate n A expert The average value of A' is the target attention result.

[0077] It should be noted that, in the embodiment of the present application, the number corresponding to n is the number of expert models in the hybrid expert model. Preferably, the number n is the number of multiple expert models selected for feature processing based on the gating network.

[0078] It should be understood that the above two improvement methods using the hybrid expert model can be used alone or in combination, and this application does not make specific limitations.

[0079] It should also be noted that since the hybrid expert model uses a gated network to input the input vector into a suitable expert model, it is prone to training instability. Based on this, the embodiment of the present application further improves the training strategy and introduces a new loss function to ensure the training stability of the target analysis model.

[0080] Specifically, the target analysis model in the embodiment of the present application is trained using a mixed precision training method to solve the instability problem caused by traditional half-precision training, and at the same time uses smaller initialization parameters for training.

[0081] In addition, the embodiment of the present application further adopts a new loss function to encourage the gated network to generate a smaller original output value ExpertNet(), avoiding the problem of unstable training caused by a large gradient in the traditional softmax activation function due to the original output value being too large. The specific loss function can be expressed as follows:

[0082]

[0083] Among them, B is the number of long text blocks processed in batches, N is the number of experts, is the original output value of the i-th expert.

[0084] Therefore, the embodiment of the present application combines the hybrid expert model on the basis of the FLASH model, which can accelerate training at the same computing cost and bring more training parameters to the model. At the same time, combined with mixed precision training and a new loss function, it can alleviate the overfitting problem when fine-tuning downstream tasks on the basis of ensuring the stable training of the target analysis model, and improve the modeling effect of long text tasks.

[0085] To summarize, the long text analysis method based on the hybrid expert model proposed in the embodiment of the present application introduces a hybrid expert model for optimization on the basis of the analysis model based on the gated attention unit. It can speed up the training while utilizing the gated attention unit to ensure the computational efficiency of the analysis model, thereby improving the analysis capability of the model in long text related tasks.

[0086] It should be noted that although the operations of the method of the present invention are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order or that all illustrated operations must be performed to achieve desired results.

[0087] Figure 5 A schematic diagram of the structure of a long text analysis device based on a hybrid expert model provided in an embodiment of the present application is shown.

[0088] like Figure 5 As shown, the long text analysis device 10 based on the hybrid expert model proposed in the embodiment of the present application includes:

[0089] An acquisition module 11 is used to acquire a long text to be analyzed;

[0090] An analysis module 12 is used to input the long text into a target analysis model to obtain a target attention result; the target analysis model includes a gated attention unit improved based on a hybrid expert model;

[0091] The generation module 13 is used to generate the analysis result corresponding to the long text based on the target attention result.

[0092] In some embodiments, the analysis module 12 is further configured to:

[0093] Performing a radial transformation on the long text to obtain local linear transformation matrix parameters, global linear transformation parameters and text features;

[0094] Determine a local attention result based on the local linear transformation matrix and the text feature, and determine a global attention result based on the global linear transformation parameter and the text feature;

[0095] Based on the local attention result and the global attention result, the target attention result is determined.

[0096] In some embodiments, the analysis module 12 is further configured to:

[0097] Acquire a plurality of expert text features determined by the hybrid expert model;

[0098] Based on the multiple expert text features, the text features used to determine the local attention result and the global attention result are determined.

[0099] In some embodiments, the analysis module 12 is further configured to:

[0100] Respectively obtaining a plurality of expert attention results determined by a plurality of hybrid expert models based on the local attention results and the global attention results;

[0101] Based on the multiple expert attention results, the target attention result is determined.

[0102] In some embodiments, the analysis module 12 is further configured to:

[0103] Based on the number of experts in the hybrid expert model, multiple results obtained based on the hybrid expert model are averaged.

[0104] In some embodiments, the target analysis model is trained using mixed precision.

[0105] It should be understood that the modules or modules described in the long text analysis device 10 based on the hybrid expert model are similar to those in the reference Figure 1 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the long text analysis device 10 based on the hybrid expert model and the modules contained therein, and will not be repeated here. The long text analysis device 10 based on the hybrid expert model can be pre-implemented in the browser or other security applications of the electronic device, and can also be loaded into the browser or its security application of the electronic device by downloading or the like. The corresponding modules in the long text analysis device 10 based on the hybrid expert model can cooperate with the modules in the electronic device to implement the solution of the embodiment of the present application.

[0106] For the several modules or units mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0107] Reference below Figure 6 , Figure 6A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown.

[0108] like Figure 6 As shown, the computer system includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation instructions of the system are also stored. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0109] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0110] In particular, according to an embodiment of the present application, the above reference flow chart Figure 2 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0111] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units or modules involved in the embodiments described in the present application may be implemented by software or by hardware. The units or modules described may also be arranged in a processor, for example, they may be described as: a processor includes an acquisition module, an analysis module, and a generation module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves, for example, the acquisition module may also be described as "acquiring a long text to be analyzed".

[0114] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the long text analysis method based on the hybrid expert model described in the present application.

[0115] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A long text analysis method based on a hybrid expert model, characterized in that: include: Get the long text to be analyzed; Inputting the long text into a target analysis model to obtain a target attention result; The target analysis model includes a gated attention unit improved based on a hybrid expert model; Based on the target attention result, an analysis result corresponding to the long text is generated.

2. The long text analysis method based on the hybrid expert model according to claim 1 is characterized in that: The step of inputting the long text into the target analysis model to obtain the target attention result includes: Performing a radial transformation on the long text to obtain local linear transformation matrix parameters, global linear transformation parameters and text features; Determine a local attention result based on the local linear transformation matrix and the text feature, and determine a global attention result based on the global linear transformation parameter and the text feature; Based on the local attention result and the global attention result, the target attention result is determined.

3. The long text analysis method based on the hybrid expert model according to claim 2 is characterized in that: Also includes: Acquire a plurality of expert text features determined by the hybrid expert model; Based on the multiple expert text features, the text features used to determine the local attention result and the global attention result are determined.

4. The long text analysis method based on the hybrid expert model according to claim 2 is characterized in that: Also includes: Respectively obtaining a plurality of expert attention results determined by a plurality of hybrid expert models based on the local attention results and the global attention results; Based on the multiple expert attention results, the target attention result is determined.

5. The long text analysis method based on the hybrid expert model according to claim 3 or 4, characterized in that: Also includes: Based on the number of experts in the hybrid expert model, multiple results obtained based on the hybrid expert model are averaged.

6. The long text analysis method based on hybrid expert model according to claim 1 is characterized in that: Also includes: The target analysis model is trained using mixed precision.

7. A long text analysis device based on a hybrid expert model, characterized in that: include: The acquisition module is used to obtain the long text to be analyzed; An analysis module, used for inputting the long text into a target analysis model to obtain a target attention result; The target analysis model includes a gated attention unit improved based on a hybrid expert model; A generation module is used to generate an analysis result corresponding to the long text based on the target attention result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the long text analysis method based on the hybrid expert model as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the long text analysis method based on the hybrid expert model as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the long text analysis method based on a hybrid expert model described in any one of claims 1 to 6 is implemented.

Images