User terminal, data processing method, and data processing program

Abstract

To provide a user terminal, a data processing method and a data processing program which can properly compress data according to the need of a user.SOLUTION: A user terminal according to the present disclosure comprises: an acquisition unit which acquires processing object data including image data; a measurement unit which measures biological information including sight line information of a user; a calculation unit which calculates the importance of the processing object data on the basis of the sight line information included in the biological information and other biological information; a decision unit which decides compressing contents of the processing object data according to the importance; and a compression unit which compresses the processing object data according to the compressing contents decided by the decision unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a user terminal, a data processing method, and a data processing program.

Background Art

[0002] In recent years, in order to provide users with high-definition and high-quality image information and audio information, there has been an increasing need to handle data of huge capacity. On the other hand, there are limits to the amount of data that can be stored and transferred. Therefore, further improvement of image compression technology and audio compression technology is required.

[0003] For example, in Patent Document 1, a line-of-sight position of an observer on a display unit for displaying an image is derived, and a first region surrounding the derived line-of-sight position, a second region surrounding the first region and having a different magnification from the first region, and a third region arranged between the first and second regions and having a continuously changing magnification are provided. Corresponding to these regions, an image display device that converts image data and displays the converted image data on the display unit is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document No. 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the image display device disclosed in Patent Document 1, although wide-angle conversion of image data is performed based on the line-of-sight position of an observer, compression content is not changed and compressed according to the needs of the user. In view of the above problems, an object of the present invention is to provide a user terminal, a data processing method, and a data processing program that can appropriately compress data according to the needs of the user.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objectives, the user terminal relating to this disclosure comprises: an acquisition unit that acquires data to be processed, including image data; a measurement unit that measures biometric information, including the user's gaze information; a calculation unit that calculates the importance of the data to be processed based on the gaze information and other biometric information included in the biometric information; a determination unit that determines the compression content of the data to be processed according to the importance; and a compression unit that compresses the data to be processed according to the compression content determined by the determination unit.

[0007] To solve the above-mentioned problems and achieve the objectives, the data processing method relating to this disclosure includes the steps of: acquiring data to be processed, including image data; measuring biometric information, including the user's gaze information; calculating the importance of the data to be processed based on the gaze information and other biometric information; determining the compression content of the data to be processed according to the importance; and compressing the data to be processed according to the compression content.

[0008] To solve the above-mentioned problems and achieve the objectives, the data processing program relating to this disclosure causes a computer to perform the following steps: acquiring data to be processed, including image data; measuring biometric information, including the user's gaze information; calculating the importance of the data to be processed based on the gaze information and other biometric information; determining the compression content of the data to be processed according to the importance; and compressing the data to be processed according to the compression content. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a user terminal, a data processing method, and a data processing program that can appropriately compress data according to the user's needs. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic diagram of a data processing system according to an embodiment. [Figure 2] Figure 2 shows an example of the configuration of a data processing system according to an embodiment. [Figure 3] Figure 3 shows an example of the configuration of a user terminal according to the first embodiment. [Figure 4] Figure 4 is a schematic diagram illustrating the concentration level of users. [Figure 5] Figure 5 is a flowchart showing an example of data processing according to the embodiment. [Figure 6] Figure 6 shows an example of the configuration of a user terminal according to the second embodiment. [Figure 7] Figure 7 is a schematic diagram illustrating the transition in the user's situation. [Figure 8] Figure 8 shows an example of the configuration of an information processing device according to the embodiment. [Figure 9] Figure 9 is a flowchart showing an example of data processing according to the embodiment as a process of the data processing system. [Modes for carrying out the invention]

[0011] The following describes in detail, with reference to the drawings, the user terminal, data processing method, and data processing program according to this application, and the forms for implementing them (hereinafter referred to as "embodiments"). However, these embodiments do not limit the user terminal, data processing method, and data processing program according to this application.

[0012] [First Embodiment] The configuration of the data processing system 1 according to the embodiment will be briefly explained using Figure 1. A detailed explanation of the configuration of the data processing system 1 will be given later. As shown in Figure 1, the data processing system 1 comprises a user terminal 100 and an information processing device 200, and the user terminal 100 and the information processing device 200 are connected to each other via a network N. The user terminal 100 comprises a communication unit 110, a storage unit 120, a control unit 130, an acquisition unit 140, and a measurement unit 150. The acquisition unit 140 acquires data to be processed, including image data, and the measurement unit 150 measures biometric information, including the user's gaze information.

[0013] [Data Processing System] Next, the configuration of the data processing system according to the embodiment will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the data processing system according to the embodiment. As shown in Figure 2, the data processing system 1 includes a user terminal 100 and an information processing device 200. Note that the data processing system 1 shown in Figure 2 may be configured to include multiple user terminals 100 and multiple information processing devices 200. Here, the user terminal 100 and the information processing device 200 are connected to each other via a predetermined communication network (network N) by wired or wireless means.

[0014] The user terminal 100 is an information processing terminal used by the user. The user terminal 100 may be, for example, a wearable computer, a smartphone, a tablet device, a desktop PC, a notebook PC, a mobile phone, a PDA (Personal Digital Assistant), or other information processing terminal. In the example shown in Figure 1, the user terminal 100 is shown as a wearable computer.

[0015] The information processing device 200 may be, for example, a personal computer (PC), a workstation (WS), or a computer equipped with server functions. The information processing device 200 processes information transmitted from the user terminal 100 via the network N.

[0016] Next, the configuration of the user terminal 100 according to the first embodiment will be described with reference to FIG. 3. FIG. 3 is a diagram showing a configuration example of the user terminal according to the first embodiment. As shown in FIG. 3, the user terminal 100 includes a communication unit 110, a storage unit 120, a control unit 130, an acquisition unit 140, and a measurement unit 150. Although not shown in FIG. 3, the user terminal 100 may include an input unit (for example, a keyboard, a mouse, etc.) for receiving various operations from the user and a display unit (for example, a liquid crystal display, etc.) for displaying various information.

[0017] (Communication Unit 110) The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. And the communication unit 110 is connected to the network N by wire or wirelessly, and performs information transmission and reception with the information processing device 200.

[0018] (Storage Unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, or a storage device such as a hard disk, an SSD (Solid State Drive), or an optical disk. As shown in FIG. 3, the storage unit 120 includes a processing target data storage unit 121, a measurement data storage unit 122, and a compressed data storage unit 123.

[0019] (Processing Target Data Storage Unit 121) The processing target data storage unit 121 stores the processing target data acquired by the acquisition unit 140 described later. For example, when the acquisition unit 140 is a camera, the processing target data storage unit 121 stores the image data acquired by the acquisition unit 140. Also, when the acquisition unit 140 is a microphone, the processing target data storage unit 121 stores the audio data acquired by the acquisition unit 140.

[0020] (Measurement Data Storage Unit 122) The measurement data storage unit 122 stores the data measured and analyzed by the measurement unit 150, which will be described later. For example, if the measurement unit 150 is an electrooculography sensor, it stores the measured value of the user's electrooculography measured by the electrooculography sensor and gaze information based on the analysis of the measured value. Also, for example, if the measurement unit 150 is a gaze detection camera, it stores the image data of the user's gaze captured by the gaze detection camera and gaze information based on the analysis of the image data. Also, for example, if the measurement unit 150 is a skin electrical activity sensor, it stores the user's skin electrical activity state measured by the skin electrical activity sensor. Also, for example, if the measurement unit 150 is a pulse wave sensor, it stores the user's pulse wave measured by the pulse wave sensor.

[0021] (Compressed data storage unit 123) The compressed data storage unit 123 stores the data to be processed that has been compressed by the compression unit 134, which will be described later. For example, if the data to be processed is image data, it stores the image data compressed by MPEG. If the data to be processed is audio data, it stores the audio data compressed by MP3 (MPEG-1 Audio Layer-3).

[0022] (Acquisition part 140) The acquisition unit 140 acquires data to be processed, including image data. For example, the acquisition unit 140 may be a camera that acquires image data of the user's surroundings. The camera includes optical elements and an image sensor. Optical elements are elements that constitute an optical system, such as lenses, mirrors, prisms, and filters. An image sensor is an element that converts light incident through the optical elements into an image signal, which is an electrical signal. Examples of image sensors include CCD (Charge Coupled Device) sensors and CMOS (Complementary Metal Oxide Semiconductor) sensors.

[0023] Furthermore, the acquisition unit 140 may be a microphone that acquires audio data from the user's surroundings. A microphone is an acoustic device that receives sound, which is air vibration, with a diaphragm or the like, and converts the vibration into an electrical signal.

[0024] (Measurement unit 150) The measurement unit 150 measures biological information. The measurement unit 150 measures gaze information and other biological information other than gaze information as biological information. The measurement unit 150 that detects gaze information may be, for example, an electrooculography sensor that measures the user's electrooculography. An electrooculography sensor is a sensor that measures the potential difference that occurs between the corneal side and the retinal side of the eyeball. The corneal side of the human eyeball is generally positively charged, and the retinal side is negatively charged, so a potential difference is generated between the two. By attaching electrodes to the skin around the eye and measuring the potential difference, this potential difference changes with eye movement and blinking, so by measuring the potential difference, it is possible to determine eye movement, i.e., the direction of gaze.

[0025] Furthermore, the measurement unit 150 that detects gaze information may be a gaze detection camera. A gaze detection camera is a camera that captures images of the user's eyes. The gaze detection camera may also include an analysis unit that performs gaze detection by image analysis of the user's eye image data. For example, the analysis unit of the gaze detection camera may detect the direction of the gaze from the user's eye image data, using characteristic parts of the face and eyes (e.g., the inner corner or outer corner of the eye) as a reference and their positional relationship with a moving point (outer edge of the iris).

[0026] Furthermore, the measurement unit 150 that detects other biological information may be an electrodermal activity (EDA) sensor that detects the electrical activity state of the skin as other biological information. The electrodermal activity sensor measures the electrical activity state of the skin due to sweat secreted from the sweat glands (e.g., eccrine glands) of the skin. Electrical activity of the skin is broadly classified into skin potential and skin conductance. Skin potential is further classified into skin potential level and skin potential reflex. Skin potential level is the direct current component of skin potential, showing a negatively high value when the level of arousal is high, and a positive value when feeling sleepy or relaxed. Skin potential reflex is the alternating current component of skin potential, and skin potential reflex frequently occurs when stimuli such as pain, touch, hearing, and sight occur, or when taking deep breaths, moving the body, or doing mental arithmetic or thinking. Therefore, the user's level of concentration can be estimated from the electrical activity state of the skin.

[0027] Furthermore, the measurement unit 150 that detects other biological information may be a pulse wave sensor that detects pulse wave information as other biological information. A pulse wave sensor is a sensor that captures the volume change of blood vessels that occurs as the heart pumps blood as a waveform. The pulse wave sensor irradiates the body with infrared light, red light, or green light with a wavelength of around 550 nm, and measures the light reflected from inside the body using a photodiode or phototransistor. Oxyhemoglobin is present in arterial blood and has the property of absorbing incident light, so a pulse wave signal can be obtained by measuring the blood volume (change in blood vessel volume) that changes with the heart's pulsation in a time series.

[0028] (Control unit 130) The control unit 130 is implemented by a CPU (Central Processing Unit) or MPU (Micro Processing Unit), etc., which executes various programs stored in the memory of the user terminal 100 using RAM as the working area. Alternatively, the control unit 130 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0029] As shown in Figure 3, the control unit 130 includes a calculation unit 131, an assignment unit 132, a determination unit 133, and a compression unit 134. Note that the calculation unit 131, assignment unit 132, determination unit 133, and compression unit 134 of the control unit 130 are not essential components of the control unit 130; it is sufficient if at least one of the control unit 130 of the user terminal 100 or the control unit 230 of the information processing device 200 has them.

[0030] (Calculation section 131) The calculation unit 131 controls the measurement unit 150 to detect gaze information and other biological information, and stores the gaze information and other biological information detected by the measurement unit 150 in the measurement data storage unit 122. The calculation unit 131 reads the gaze information and other biological information from the measurement data storage unit 122 and calculates the importance of the data to be processed based on the gaze information and other biological information. First, the calculation unit 131 calculates the concentration level based on the gaze information and the concentration level based on other biological information other than gaze information as follows.

[0031] First, the level of concentration based on eye-tracking information is determined by calculating the user's usual eye movement patterns (distance traveled and acceleration) and then calculating the change in eye movement patterns during concentration (difference in distance traveled and difference in acceleration). In other words, if the user's eye movement patterns decrease compared to their usual eye movement patterns, the level of concentration is calculated to have increased. Conversely, if the user's eye movement patterns increase compared to their usual eye movement patterns, the level of concentration is calculated to have decreased. Figure 4 is a schematic diagram illustrating the user's level of concentration. As shown on the left side of Figure 4, if the user's eye movement patterns are small and the user's field of view is fixed, the user's level of concentration is calculated to be high. Conversely, as shown on the right side of Figure 4, if the user's eye movement patterns are large and the user's field of view is not fixed, the user's level of concentration is calculated to be low. In other words, for example, the level of concentration may be calculated such that the greater the user's eye movement patterns, the lower the user's level of concentration.

[0032] Next, the concentration level based on other biometric information is calculated by measuring the user's normal biometric information and the changes in biometric information during concentration. If the values ​​increase or decrease compared to normal, the concentration level is calculated to have increased or decreased. For example, if normal biometric information is set as 0%, and biometric information increases by 1%, increases by 3%, decreases by 1%, and decreases by 3% compared to normal, the concentration level can be calculated to increase or decrease in 5 stages, with the middle being the normal level of concentration. For example, if the other biometric information is a pulse wave measured by a pulse wave sensor, the concentration level is calculated based on the RR interval of the pulse wave. That is, in order to calculate the concentration level from the pulse wave, the fluctuation of the RR interval of the pulse wave is calculated. Here, at the cellular level, heartbeat is a repetition of depolarization (action potential) and repolarization (resting potential), and an electrocardiogram can be created by measuring this electrical activity from the body surface. The RR interval is the time width between adjacent peaks R of the electrocardiogram waveform, that is, the interval between R waves of this waveform. Since pulse waves are transmitted throughout the body almost simultaneously with the heart's activity, it can be assumed that the heartbeat is synchronized with the pulse wave. Because pulse waves and the R wave of an electrocardiogram are synchronized, the RR interval of the pulse wave can be considered equivalent to the RR interval of an electrocardiogram. The fluctuation in the RR interval of the pulse wave can also be called the time derivative, so by calculating the derivative and detecting the magnitude of the fluctuation, the degree of activation or deactivation of the autonomic nervous system in the body can be determined. In other words, the degree of concentration can be determined from this derivative. For example, the degree of concentration can be calculated such that the larger this derivative (pulse wave fluctuation), the higher the degree of concentration. Alternatively, if other biological information is the state of electrical activity of the skin measured by a skin electrical activity sensor, the degree of concentration can be calculated based on the positive or negative skin potential level, the frequency of skin potential reflexes, etc.

[0033] The calculation unit 131 calculates the concentration level based on gaze information and the concentration level based on other biometric information, and then calculates the user's concentration level CT based on the following formula. Here, CT1 in formula (1) represents the concentration level based on gaze information, and CT2 represents the concentration level based on other biometric information. Note that CT2 in formula (1) may be the concentration level based on one other biometric information, or it may be the concentration level based on multiple other biometric information.

[0034]

number

[0035] The calculation unit 131 calculates the importance of the data to be processed after calculating the user's concentration level CT based on equation (1). For example, the calculation unit 131 may divide the user's concentration level CT into five stages, associate these five stages with the importance of the data to be processed, and calculate the importance of the data to be processed in five stages according to the user's concentration level CT. Here, the calculation unit 131 calculates the importance such that the higher the user's concentration level CT, the higher the importance of the data to be processed.

[0036] The calculation unit 131 calculates importance for each predetermined unit of the data to be processed, divided into three or more levels, based on gaze information and other biometric information. For example, if the data to be processed is image data, the calculation unit 131 may use MB (Macroblock) as the predetermined unit, divide the image data corresponding to one image into MBs, and calculate importance for each MB. That is, for example, among the MBs, the importance of MBs that the user is expected to be focusing on is calculated to be higher. For example, the calculation unit 131 calculates the importance of MBs located in the direction of the user's gaze based on the user's gaze information. Specifically, importance is calculated based on the following equation (2). Here, IMP in equation (2) represents importance and K represents the weighting coefficient.

[0037]

number

[0038] In other words, the calculation unit 131 calculates the importance of MBs located in the direction of the user's line of sight by increasing the weighting coefficient K. For example, the weighting coefficient can be set to 5 for MBs located within ±0.25° of the direction of sight, and to 4 for MBs located between -0.5° and -0.25°, and between 0.25° and 0.5° of the direction of sight. The weighting coefficient can be set so that it decreases as the MB deviates from the direction of sight. Note that the weighting coefficient values ​​mentioned above are examples, and other arbitrary values ​​can be set. Also, although equation (2) sets a weighting coefficient K common to concentration levels CT1 and CT2, it is not limited to this. A weighting coefficient K can be set for each concentration level CT1 and CT2 by setting a weighting coefficient K1 for concentration level CT1 and a weighting coefficient K2 for concentration level CT2. Furthermore, the calculation unit 131 may change the weighting coefficient K (including the cases of K1 and K2) based on the operation received by the input unit. Also, if the data to be processed is audio data, the calculation unit 131 may calculate the importance at predetermined time intervals.

[0039] (Granting section 132) The assignment unit 132 assigns the importance calculated by the calculation unit 131 to each predetermined unit of data to be processed. For example, the assignment unit 132 may assign the importance calculated by the calculation unit 131 as tag information to the user data that can be set for each picture in the data compressed using the MPEG compression method. That is, since there is an area in the MPEG syntax where a start code called user_start_code, which can be uniquely determined by the Huffman code, can be stored as 32 bits of data, the importance may be assigned as tag information to that area.

[0040] Alternatively, the tagging unit 132 may also multiplex only the tag information as an independent elementary stream using the syntax of the "MPEG system layer," which packs elementary streams such as MPEG video and audio, adds headers and IDs, and multiplexes them in predetermined units.

[0041] (Decision Section 133) The determination unit 133 determines the compression ratio (compression content) of the data to be processed according to its importance. The determination unit 133 determines the compression ratio such that the higher the importance, the lower the compression ratio. For example, if the calculation unit 131 calculates the importance in five stages from highest to lowest: "Level 5", "Level 4", "Level 3", "Level 2", and "Level 1", the determination unit 133 may determine the compression ratios to be 5%, 10%, 15%, 20%, and 30% in order from highest to lowest importance. Note that the compression ratio values ​​corresponding to the importance levels mentioned here are examples, and any compression ratio value may be determined according to the importance level. Furthermore, although the determination unit 133 determines the compression ratio of the data to be processed according to its importance, it is not limited to this, and may also determine the compression ratio, the amount of compressed data, distortion, and other compression content.

[0042] Furthermore, the determination unit 133 determines the compression ratio for each predetermined unit of the data to be processed based on the importance level assigned to each predetermined unit of the data to be processed. That is, for example, if the importance level of unit A of the data to be processed is "level 5", the determination unit 133 may determine the compression ratio for unit A to be 5%, and if the importance level of unit B of the data to be processed is "level 2", the compression ratio for unit B to be processed may be determined to be 20%.

[0043] (Compression section 134) The compression unit 134 compresses the data to be processed according to the compression ratio determined by the determination unit 133. The compression unit 134 may compress the data to be processed using internationally standardized compression technologies such as MPEG. Note that MPEG is a technology that combines several technologies. The input image of the compression unit 134 reduces time redundancy by taking the difference between the input image and the image encoded by the motion compensation predictor. There are three modes for prediction direction: past, future, and both, and these are switched and used for each 16x16 pixel MB. Motion compensation is performed by pattern matching of the motion region for each MB, detecting the motion vector with half-pixel (half-pel) accuracy, shifting by the amount of motion, and then predicting. The difference image is subjected to a Discrete Cosine Transform (DCT), and the integral transform with the cosine function as the integral kernel is discretely transformed into a finite space. In MPEG, the MB is divided into four parts, and a 2D DCT is performed on 8x8 DCT blocks. Generally, video signals have many low-frequency components and few high-frequency components, so when DCT is performed, the coefficients concentrate on the low-frequency components. The DCT-processed image data (DCT coefficients) is then quantized. Quantization is performed by multiplying a quantization matrix, which is an 8x8 two-dimensional frequency weighted by visual characteristics, by a quantization scale, which is a scalar multiplier of the entire matrix, to obtain a quantized value, and then dividing the DCT coefficient by this quantized value.

[0044] The quantized data is then encoded using variable-length coding. The DC component of the quantized value is coded using differential pulse code modulation (DPCM), a type of predictive coding. DPCM is an encoding method that modulates the difference between signals using pulse code modulation (PCM). The AC component is coded using Huffman coding, which involves a zigzag scan from low to high frequencies, treating the run length and effectiveness coefficient value of zero as a single event, and assigning codes with shorter code lengths in order of highest probability of occurrence. The variable-length coded data is temporarily stored in a buffer and output as coded data. The generated code amount per MB of data is controlled by a scalar multiplier during quantization. This value can be set for each MB of 16x16 pixels of the image. Therefore, the code amount can be controlled by adjusting the quantization scale while observing the generated code amount relative to the target code amount.

[0045] At the same time, the image quality for each MB of the image may be controlled by reducing the quantization scale for the MB of the image corresponding to the position of the gaze, thereby improving image quality, and increasing the quantization scale for other parts.

[0046] Alternatively, before image compression, a low-pass filter with adjustable bandwidth across the entire frame can be used. In scenes where the gaze is relatively still, the overall image quality can be preserved, while in other scenes, the low-pass filter setting can be adjusted to limit bandwidth (reducing resolution) to lower the overall information entropy of the image before MPEG compression.

[0047] Next, using Figure 5, the data processing according to the embodiment will be explained in accordance with the flowchart. Figure 5 is a flowchart of an example of data processing according to the embodiment. The user terminal 100 acquires data to be processed, including image information (step S1). The user terminal 100 measures the user's gaze information (step S2). The user terminal 100 measures the user's biometric information (step S3). The user terminal 100 calculates the user's concentration level from the gaze information and other biometric information (step S4). The user terminal 100 ranks the importance of the data to be processed according to the calculated concentration level (step S5). The user terminal 100 determines the compression content according to the importance of the data to be processed and compresses the data to be processed according to the determined compression content (step S6). The user terminal 100 stores the compressed data to be processed (step S7). The user terminal 100 transmits the compressed data to be processed to the information processing device 200 (step S8). The information processing device 200 stores the received compressed data to be processed.

[0048] As described above, the user terminal 100 according to the first embodiment estimates the importance of the data to be processed based on gaze information and other biometric information such as pulse waves, and determines the compression content based on that importance. In other words, the user terminal 100 estimates whether the user perceives something as important based on biometric information, and determines the compression content accordingly, thereby appropriately compressing the data to suit the user's needs.

[0049] [Second Embodiment] Next, the configuration of the user terminal 100 according to the second embodiment will be described using Figure 6. Figure 6 is a diagram showing an example of the configuration of the user terminal according to the second embodiment. As shown in Figure 6, the user terminal 100 according to the second embodiment has a discrimination unit 135 added to the configuration of the control unit 130 of the user terminal 100 according to the first embodiment. Note that the parts of the configuration of the user terminal 100 according to the second embodiment that are common with the configuration of the user terminal 100 according to the first embodiment will not be explained. The user terminal 100 according to the second embodiment is the same as the user terminal 100 according to the first embodiment, except that the processing of the calculation unit 131 is different and a discrimination unit 135 has been added. For this reason, the processing of the calculation unit 131 and the discrimination unit 135 will be explained.

[0050] The discrimination unit 135 analyzes the data to be processed to determine the situation in which the user is located. For example, the discrimination unit 135 may analyze the data to be processed using a learning model trained on training data to which annotation labels have been added for each situation in which the user is located, in order to determine the situation in which the user is located. Here, the situation in which the user is located refers to various situations that the user may encounter in their daily life, such as the user looking at a mountain landscape or the user negotiating a business deal with a trading partner.

[0051] The calculation unit 131 sets the importance level according to the situation determined by the discrimination unit 135. The calculation unit 131 objectively determines whether the determined situation is important to the user, and if it determines that it is important, it calculates the importance level so that it becomes higher. Specifically, the calculation unit 131 changes the weighting coefficient used to calculate the importance level of the data to be processed according to the situation determined by the discrimination unit 135, and calculates the importance level of the data to be processed. For example, if the situation determined by the discrimination unit 135 is that the user is negotiating with a trading partner, the calculation unit 131 changes the weighting coefficient to a higher value and calculates the importance level of the data to be processed. Also, for example, if the situation determined by the discrimination unit 135 is that the user is looking at a mountain view, the calculation unit 131 changes the weighting coefficient to a lower value and calculates the importance level of the data to be processed. In this way, the calculation unit 131 changes the weighting coefficient according to the situation determined by the discrimination unit 135 and calculates the importance level of the data to be processed.

[0052] Thus, in the second embodiment, importance is calculated based not only on biometric information but also on the user's circumstances. Therefore, data can be compressed more appropriately to suit the user's needs.

[0053] [Third Embodiment] Next, the configuration of the user terminal according to the third embodiment will be described with reference to Figure 6. Note that the parts of the configuration of the user terminal 100 according to the third embodiment that are common with the configuration of the user terminal 100 according to the first embodiment will not be explained. The user terminal 100 according to the third embodiment is the same as the user terminal 100 according to the first embodiment, except that the processing of the calculation unit 131 differs and a discrimination unit 135 is added. Furthermore, the third embodiment can be combined with the second embodiment.

[0054] The discrimination unit 135 analyzes the data to be processed to determine a change in the user's situation. For example, the discrimination unit 135 may analyze the data to be processed using a learning model trained on training data to which annotation labels have been added for each situation the user is in, in order to determine a change in the user's situation. Here, the change in situation will be explained using Figure 7. Figure 7 is a schematic diagram illustrating a change in the user's situation. That is, a change in the user's situation means, for example, a change from situation ST1 where user U1 is looking at a mountain landscape to situation ST2 where user U1 is negotiating with a business partner, as shown in Figure 7.

[0055] The calculation unit 131 calculates the importance of the data to be processed based on eye-tracking information and other biometric information, as well as the transition of circumstances in the data to be processed. Specifically, the calculation unit 131 calculates the importance of the data to be processed in such a way that the importance of the data to be processed is higher when the data to be processed is in a transitional area. In particular, when the data to be processed is in a transitional area, the calculation unit 131 changes the weighting coefficient used to calculate the importance of the transitional area to a higher value, and calculates the importance of the data to be processed in such a way that the importance is calculated to be higher.

[0056] Thus, in the third embodiment, the importance of the data to be processed in the portion where the user's situation changes is calculated to be higher, and the compression ratio of that portion of the data to be processed can be lowered. Therefore, when the user checks the data to be processed, the data quality of the starting point, the portion where the situation changes, is increased, and the compression ratio of the other portions is determined according to the importance based on the degree of concentration, thereby achieving both data quality assurance and data volume reduction.

[0057] [Information Processing Device] In the above description, the user terminal 100 performed the setting of importance levels based on concentration and the compression of image data based on importance levels. However, the information processing device 200 may perform the processing that was performed by the user terminal 100. The configuration of the information processing device 200 in that case will be described below with reference to Figure 8. Figure 8 is a diagram showing an example of the configuration of an information processing device according to the embodiment. As shown in Figure 8, the information processing device 200 has a communication unit 210, a storage unit 220, and a control unit 230. Although not shown in Figure 8, the information processing device 200 may also have an input unit (e.g., a keyboard or mouse) for receiving various operations from the administrator of the information processing device 200, and a display unit (e.g., a liquid crystal display) for displaying various information.

[0058] The communication unit 210 is implemented, for example, by a NIC (Network Interface Card). The communication unit 210 is connected to the network N by wire or wireless connection and transmits and receives various types of information to and from the user terminal 100 via the network N.

[0059] The storage unit 220 is implemented by, for example, semiconductor memory elements such as RAM and flash memory, or storage devices such as hard disks, SSDs, and optical discs. As shown in Figure 8, the storage unit 220 includes a data storage unit 221 for data to be processed, a measurement data storage unit 222, and a compressed data storage unit 223.

[0060] The data to be processed storage unit 221 stores the data to be processed acquired from the user terminal 100 by the acquisition unit 231, which will be described later. Since the information stored in the data to be processed storage unit 221 is the same as the information stored in the data to be processed storage unit 121 of the user terminal 100, a description of the information stored in the data to be processed storage unit 221 will be omitted.

[0061] The measurement data storage unit 222 stores the measurement data acquired from the user terminal 100 by the acquisition unit 231, which will be described later. Since the information stored in the measurement data storage unit 222 is the same as the information stored in the measurement data storage unit 122 of the user terminal 100, a description of the information stored in the measurement data storage unit 222 will be omitted.

[0062] The compressed data storage unit 223 stores compressed data acquired from the user terminal 100 by the acquisition unit 231, which will be described later, or compressed data compressed by the compression unit 236, which will be described later. The information stored in the compressed data storage unit 223 is the same as the information stored in the compressed data storage unit 123 of the user terminal 100, so the explanation of the information stored in the compressed data storage unit 223 will be omitted.

[0063] The control unit 230 is implemented, for example, by a CPU or MPU executing various programs stored in the information processing device 200 using RAM as a working area. Alternatively, the control unit 230 may be implemented by an integrated circuit such as an ASIC or FPGA.

[0064] As shown in Figure 8, the control unit 230 includes an acquisition unit 231, a calculation unit 232, an assignment unit 233, a discrimination unit 234, a determination unit 235, and a compression unit 236. Note that the calculation unit 232, assignment unit 233, discrimination unit 234, determination unit 235, and compression unit 236 included in the control unit 230 are not essential components of the control unit 230; it is sufficient if at least one of the control unit 130 of the user terminal 100 or the control unit 230 of the information processing device 200 includes them.

[0065] The acquisition unit 231 acquires the data to be processed, the measurement data, and the compressed data from the user terminal 100. If all processing by the control unit 130 of the user terminal 100 is executed, the acquisition unit 231 only needs to acquire the compressed data from the user terminal 100.

[0066] Since the calculation unit 232 performs the same processing as the calculation unit 131 located in the control unit 130 of the user terminal 100, a description of the calculation unit 232's processing will be omitted. However, the calculation unit 232 performs the same processing as the calculation unit 131 located in the control unit 130 of the user terminal 100 only when the acquisition unit 231 has acquired the data to be processed and the measurement data from the user terminal 100.

[0067] Since the assignment unit 233 performs the same processing as the assignment unit 132 provided in the control unit 130 of the user terminal 100, the explanation of the processing of the assignment unit 233 is omitted. However, the assignment unit 233 performs the same processing as the assignment unit 132 provided in the control unit 130 of the user terminal 100 only when the acquisition unit 231 has acquired the data to be processed and the measurement data from the user terminal 100.

[0068] Since the discrimination unit 234 performs the same processing as the discrimination unit 135 provided in the control unit 130 of the user terminal 100, the explanation of the processing of the discrimination unit 234 is omitted. However, the discrimination unit 234 performs the same processing as the discrimination unit 135 provided in the control unit 130 of the user terminal 100 only when the acquisition unit 231 has acquired the data to be processed and the measurement data from the user terminal 100.

[0069] Since the decision unit 235 performs the same processing as the decision unit 133 provided in the control unit 130 of the user terminal 100, the explanation of the processing of the decision unit 235 is omitted. However, the decision unit 235 performs the same processing as the decision unit 133 provided in the control unit 130 of the user terminal 100 only when the acquisition unit 231 has acquired the data to be processed and the measurement data from the user terminal 100.

[0070] Since the compression unit 236 performs the same processing as the compression unit 134 provided in the control unit 130 of the user terminal 100, the explanation of the processing of the compression unit 236 is omitted. However, the compression unit 236 performs the same processing as the compression unit 134 provided in the control unit 130 of the user terminal 100 only when the acquisition unit 231 has acquired the data to be processed and the measurement data from the user terminal 100.

[0071] Figure 5 illustrates the case where steps S1 to S8 are executed on the user terminal 100, but steps S4 to S7 can be executed by either the user terminal 100 or the information processing device 200. To illustrate this point, Figure 9 shows a flowchart of the processes to be executed as the data processing system 1.

[0072] Figure 9 is a flowchart showing an example of data processing according to the embodiment as a process of the data processing system. For example, the data processing system 1 acquires data to be processed, including image data (step S101). The data processing system 1 then measures biometric information, including the user's gaze information (step S102). The data processing system 1 then calculates the importance of the data to be processed based on the gaze information and other biometric information (step S103). The data processing system 1 then determines the compression content of the data to be processed according to the calculated importance (step S104). The data processing system 1 then compresses the data to be processed according to the determined compression content (step S105).

[0073] The user terminal 100 relating to this disclosure includes an acquisition unit 140 that acquires data to be processed, including image data; a measurement unit 150 that measures biometric information, including the user's gaze information; a calculation unit 131 that calculates the importance of the data to be processed based on the gaze information and other biometric information included in the biometric information; a determination unit 133 that determines the compression content of the data to be processed according to the importance; and a compression unit 134 that compresses the data to be processed according to the compression content determined by the determination unit 133.

[0074] With this configuration, the user terminal 100 can determine the importance of the data to be processed according to the user's level of concentration and change the compression content of the data to be processed. Therefore, it is possible to achieve both ensuring data quality that meets the user's needs and reducing data volume.

[0075] The calculation unit 131 of the user terminal 100 relating to this disclosure further includes an assignment unit 132 that calculates importance in three or more stages for each predetermined unit of data to be processed based on gaze information and other biometric information, and assigns the importance calculated by the calculation unit 131 to each predetermined unit of data to be processed.

[0076] With this configuration, the user terminal 100 can calculate the importance of each predetermined unit of data to be processed, and compress the data by changing the compression content for each predetermined unit of data to be processed. Therefore, it is possible to precisely balance ensuring data quality according to the user's needs with reducing data volume.

[0077] The determination unit 133 of the user terminal 100 relating to this disclosure determines the compression content for each predetermined unit of data to be processed based on the importance assigned to each predetermined unit of data to be processed, and the compression unit 134 compresses the data to be processed in predetermined units according to the compression content determined by the determination unit 133 for each predetermined unit of data to be processed.

[0078] With this configuration, the user terminal 100 can calculate the importance of each predetermined unit of data to be processed, and compress the data by changing the compression content for each predetermined unit of data to be processed. Therefore, it is possible to precisely balance ensuring data quality according to the user's needs with reducing data volume.

[0079] The user terminal 100 relating to this disclosure further includes a discrimination unit 135 that analyzes the data to be processed to determine the situation in which the user is located, and a calculation unit 131 that, according to the situation determined by the discrimination unit 135, changes the weighting coefficient used to calculate the importance of the data to be processed and calculates the importance of the data to be processed.

[0080] With this configuration, the user terminal 100 can determine the importance of the data to be processed after identifying the situation in which the user is located. Therefore, since the importance can be determined not only based on the user's subjective information, such as concentration level, but also on objective information, it becomes possible to give objectivity to the importance level.

[0081] The discrimination unit 135 of the user terminal 100 relating to this disclosure analyzes the data to be processed to determine a change in the user's situation, and the calculation unit 131 calculates the importance of the data to be processed based on the change in the situation in the data to be processed, in addition to the gaze information and other biometric information.

[0082] With this configuration, the user terminal 100 can calculate the importance of the data to be processed in the portion where the user's situation changes as higher, and reduce the compression ratio of that portion of the data to be processed. Therefore, when the user checks the data to be processed, the data quality of the starting point, the portion where the situation changes, is increased, and the compression ratio of the other portions is determined according to the importance based on the degree of concentration, thereby achieving both data quality assurance and data capacity reduction.

[0083] The data processing method relating to this disclosure includes the steps of: acquiring data to be processed, including image data; measuring biometric information, including the user's gaze information; calculating the importance of the data to be processed based on the gaze information and other biometric information; determining the compression content of the data to be processed according to the importance; and compressing the data to be processed according to the determined compression content.

[0084] This data processing method allows for determining the importance of the data to be processed based on the level of user engagement, and modifying the compression settings for that data. Therefore, it is possible to achieve both data quality tailored to user needs and a reduction in data volume.

[0085] The data processing program relating to this disclosure causes a computer to perform the following steps: acquire data to be processed, including image data; measure biometric information, including the user's gaze information; calculate the importance of the data to be processed based on the gaze information and other biometric information; determine the compression content of the data to be processed according to the importance; and compress the data to be processed according to the determined compression content.

[0086] This data processing program configuration allows for determining the importance of the data to be processed based on the user's level of engagement, and modifying the compression settings for that data. Therefore, it is possible to achieve both data quality tailored to user needs and a reduction in data volume.

[0087] Although embodiments of the present invention have been described above, the embodiments are not limited to those described herein. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the embodiments described above.

[0088] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit." For example, the acquisition unit 140 can be replaced with acquisition means or acquisition circuit. [Explanation of Symbols]

[0089] 1. Data Processing System 100 User Terminals 110 Communications Department 120 Storage section 121 Data storage unit for processing 122 Measurement data storage unit 123 Compressed Data Storage Unit 130 Control Unit 131 Calculation Department 132 Granting section 133 Decision Section 134 Compression section 135 Discrimination part 140 Acquisition Department 150 Measurement Unit 200 Information Processing Devices N Network

Claims

1. An acquisition unit that acquires data to be processed, including image data, A measurement unit that measures biometric information including the user's gaze information, A calculation unit that calculates the importance of the data to be processed based on gaze information and other biometric information included in the aforementioned biometric information, A determination unit that determines the compression content of the data to be processed according to the aforementioned importance, A compression unit compresses the data to be processed according to the compression content determined by the determination unit, A discrimination unit analyzes the data to be processed and determines the situation in which the user is located, Equipped with, The calculation unit calculates the importance of the data to be processed by changing the weighting coefficient used to calculate the importance of the data to be processed according to the situation determined by the discrimination unit. User terminal.

2. The calculation unit calculates the importance of each predetermined unit of the data to be processed in three or more stages based on gaze information and other biometric information. The system further includes an assignment unit that assigns importance calculated by the calculation unit to each predetermined unit of the data to be processed. The user terminal according to claim 1.

3. The determination unit determines the compression content for each predetermined unit of the data to be processed based on the importance assigned to each predetermined unit of the data to be processed, The compression unit compresses the data to be processed in predetermined units according to the compression content determined by the determination unit for each predetermined unit of the data to be processed. The user terminal according to claim 2.

4. The discrimination unit analyzes the data to be processed to determine a change in the user's situation, The calculation unit calculates the importance of the data to be processed based on eye-tracking information and other biometric information, as well as changes in the situation of the data to be processed. The user terminal according to claim 1.

5. The steps include obtaining data to be processed, including image data, The process involves analyzing the data to be processed to determine the user's situation, The steps include changing the weighting coefficient used to calculate the importance of the data to be processed, depending on the determined situation, A step of measuring biometric information including the user's gaze information, A step of calculating the importance of the data to be processed based on the aforementioned gaze information and other biometric information, The steps include determining the compression content of the data to be processed according to the aforementioned importance, The step of compressing the data to be processed according to the determined compression content, Data processing method.

6. The steps include obtaining data to be processed, including image data, The process involves analyzing the data to be processed to determine the user's situation, The steps include changing the weighting coefficient used to calculate the importance of the data to be processed, depending on the determined situation, A step of measuring biometric information including the user's gaze information, A step of calculating the importance of the data to be processed based on the aforementioned gaze information and other biometric information, The steps include determining the compression content of the data to be processed according to the aforementioned importance, The steps include compressing the data to be processed according to the determined compression content, A data processing program that causes a computer to execute a command.

Images