Distraction characteristic ascertaining system

Abstract

This distraction characteristic ascertaining system includes: a property selection unit that selects the properties of an object to be superimposed on video; an object presentation unit that generates an object based on the selected properties and presents, to a user, video on which the object is superimposed; an attention level acquisition unit that acquires the attention level of the user viewing the video; and a model training unit that executes training of a distraction model on the basis of an attention level log including object presentation information and the attention level.

Description

Distraction characteristic identification system

[0001] The present invention relates to a distraction characterization system.

[0002] Diminished reality (DR) technology, which uses augmented reality (AR) technology to hide or highlight objects in an image, is being researched.

[0003] For example, one possible use of distorted reality technology is to make objects that distract a driver while driving a car (such as a conspicuous sign or an oncoming vehicle) disappear, thereby making the driver less likely to become distracted, i.e., improving the driver's concentration.

[0004] For example, by using a model (hereinafter also referred to as a "distraction model") that shows the relationship between the properties of objects displayed in a video (e.g., saturation, brightness, hue, size, texture, speed, or a saliency map) and the likelihood of distraction, it is expected that the obscuration process can be performed relatively easily and automatically.

[0005] Yi Fei Cheng, et al., “Towards Understanding Diminished Reality”, CHI'22, April 29 - May 5, 2022 Association for Computing Machinery; Punit Kumar, et al., “Active Learning Query Strategies for Classification, Regression, and Clustering: A Survey”, JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 35(4), July 2020, pp.913-945; Ryuichi Yoshida and three others, “Measuring Distraction from Sweat on the Hand”, The Japan Society of Mechanical Engineers, No.14-2; Proceeding of the 2014 JSME Conference on Robotics and Mechatronics, May 25-29, 2014, 3A1-W06, pp.1-2; Toshio Mori and two others, “Relationship between Visual Impression of Color Texture and Image Information Content”, Journal of the Japan Research Association for Textile End-Uses, Vol.51, 2010, p.433-440

[0006] For example, information about the properties of objects and the likelihood of distraction is used as training data for the distraction model. However, there are few opportunities to observe a state in which distraction actually occurs, and such a state is highly dangerous.

[0007] The present invention has been made in light of the above-mentioned circumstances, and its purpose is to provide a system for grasping distraction characteristics that can learn a distraction model without observing actual states in which distraction occurs.

[0008] According to one aspect of the present invention, a distraction characteristic understanding system includes a property selection unit that selects the property of an object to be superimposed on an image, an object presentation unit that generates an object based on the selected property and presents the image with the object superimposed to a user, an attention level acquisition unit that acquires the attention level of the user who views the image, and a model learning unit that performs learning of a distraction model based on an attention level log that includes object presentation information and the attention level.

[0009] According to one aspect of the present invention, a system for grasping distraction characteristics is provided that can learn a distraction model without observing a state in which distraction actually occurs by presenting an image on which an object of a selected property is superimposed, obtaining the level of attention of the user who views the image, and using an attention level log containing the object presentation information and the level of attention as training data.

[0010] FIG. 1 is a block diagram illustrating an example of the functional configuration of a distraction characteristic assessment system according to a first embodiment. FIG. 2 is a flowchart illustrating an example of learning a distraction model in the distraction characteristic assessment system according to the first embodiment. FIG. 3 is a diagram illustrating a specific example of object presentation in the distraction characteristic assessment system according to the first embodiment. FIG. 4 is a table illustrating a specific example of an attention level log in the distraction characteristic assessment system according to the first embodiment. FIG. 5 is a table illustrating a specific example of logistic regression coefficients for each parameter when a logistic regression model is used as the distraction model in the distraction characteristic assessment system according to the first embodiment. FIG. 6 is a specific example of a three-dimensional graph displaying the relationship between the likelihood of distraction, saturation, and contrast based on the distraction model in the distraction characteristic assessment system according to the first embodiment. FIG. 7 is a block diagram illustrating an example of the hardware configuration of a distraction characteristic assessment system according to the first embodiment. FIG. 8 is a block diagram illustrating an example of the functional configuration of a distraction characteristic assessment system according to a second embodiment. FIG. 9 is a flowchart illustrating an example of learning a distraction model in the distraction characteristic assessment system according to the second embodiment. FIG. 10 is an explanatory diagram illustrating a specific example of property adjustment in the distraction characteristic assessment system according to the second embodiment. 11 is a table showing a specific example of an adjusted attention level log in the distraction characteristic assessment system according to the second embodiment. FIG. 12 is a table showing a specific example of a logistic regression coefficient of each parameter of the distraction model in the distraction characteristic assessment system according to the second embodiment.

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, components having the same functions and configurations will be given the same reference numerals.

[0012] 1. First Embodiment 1.1 Functional Configuration First, an example of the functional configuration of the distraction characteristic identification system 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the functional configuration of the distraction characteristic identification system 1. Note that in the example shown in Fig. 1, connections between components are indicated by arrows, but connections between components are not limited to this.

[0013] 1 , the distraction characteristic identification system 1 is a system for learning (generating) a distraction model 102. The distraction characteristic identification system 1 includes a property selection unit 11, an object presentation unit 12, an attention level acquisition unit 13, an attention level log storage unit 14, a model learning unit 15, and a distraction model storage unit 16.

[0014] The property selection unit 11 is connected to the object presentation unit 12, the attention log storage unit 14, and the model learning unit 15. The property selection unit 11 acquires an attention log 101 from the attention log storage unit 14. The property selection unit 11 acquires a trained distraction model 102 from the model learning unit 15. The attention log 101 and the distraction model 102 will be described later. The property selection unit 11 selects (sets) the property of an object to be presented on the object presentation unit 12 based on the attention log 101 and the distraction model 102. In this embodiment, the object refers to a virtual object generated by the distraction characteristic assessment system 1. For example, if the attention log 101 and the trained distraction model 102 are not available, the property selection unit 11 selects the property of the object based on a preset initial value. For example, when the training of the distraction model 102 is repeated, the property selection unit 11 selects a property different from the previously selected property based on the attention log 101 and the distraction model 102. The property selection unit 11 may use active learning or the like to select the property of an object. For example, parameters indicating the property of an object include saturation, brightness, hue, size, texture, speed, or a saliency map. The property selection unit 11 may also select the display position of the object in the video (image). The property selection unit 11 transmits information about the property of the selected object (hereinafter also referred to as "object property information") to the object presentation unit 12.

[0015] The object presentation unit 12 is connected to the property selection unit 11, the attention level acquisition unit 13, and the attention level log storage unit 14. The object presentation unit 12 generates an object having the property based on the property information of the object. The object presentation unit 12 presents an image on which the generated object is superimposed to a user (hereinafter simply referred to as "user") of the distraction characteristic identification system 1 via a display device (not shown). The image may be an actually captured image or a virtual reality (VR) image. That is, the image presented to the user may be an augmented reality image or a virtual reality image. The object presentation unit 12 may also generate multiple objects and superimpose them on the image. In this case, the property selection unit 11 may select the properties of multiple objects. The object presentation unit 12 transmits information about the object presented to the user (hereinafter also referred to as "object presentation information") to the attention level acquisition unit 13 and the attention level log storage unit 14.

[0016] The attention level acquisition unit 13 is connected to the object presentation unit 12 and the attention level log storage unit 14. The attention level acquisition unit 13 acquires the level of attention of a user who views a video on which an object is superimposed (hereinafter also referred to as "attention level data"). In other words, the attention level acquisition unit 13 acquires data regarding the likelihood of a user becoming distracted when viewing a video on which an object is superimposed. For example, the more distracted the user's attention is, the lower the value of the user's attention level is. For example, the attention level acquisition unit 13 measures the user's reaction time or the magnitude of the reaction to an object (video) using information such as the user's biosignals (e.g., signals such as electrodermal activity, heart rate, or electroencephalogram) and eye movement measured by a measuring device (not shown). The attention level acquisition unit 13 then acquires (calculates) the attention level data based on the measurement results. Note that the attention level acquisition unit 13 may acquire the attention level data by the user directly inputting the attention level via an input device (not shown). The attention level acquisition unit 13 transmits the attention level data to the attention level log storage unit 14.

[0017] The attention log storage unit 14 is connected to the property selection unit 11, the object presentation unit 12, the attention acquisition unit 13, and the model learning unit 15. The attention log storage unit 14 stores (memorizes) an attention log 101. The attention log 101 is a log including object presentation information and attention data corresponding to the presented object. In other words, the attention log 101 is a log related to the property of an object and the user's attention when the object is presented. The attention log storage unit 14 transmits the attention log 101 to the property selection unit 11 and the model learning unit 15.

[0018] The model learning unit 15 is connected to the property selection unit 11, the attention level log storage unit 14, and the distraction model storage unit 16. The model learning unit 15 uses the attention level log 101 as training data to perform learning (machine learning or deep learning) of the distraction model 102. The distraction model 102 indicates the relationship between an increase or decrease in the value of each parameter indicating the property of an object and the likelihood of the user becoming distracted. The model learning unit 15 uses, for example, a logistic regression model as a learning model for the distraction model 102. Note that the distraction model 102 is not limited to the logistic regression model. The model learning unit 15 may generate a distraction model 102 for each user, or may generate distraction models 102 corresponding to multiple users. The model learning unit 15 transmits the learned distraction model 102 to the property selection unit 11 and the distraction model storage unit 16. Furthermore, for example, the model learning unit 15 outputs the trained distraction model 102 to an external device (such as a video processing device that performs an erasure process). For example, in the video processing device that performs the erasure process, the erasure parameters of each object in the video are estimated based on the distraction model 102. Note that the distraction characteristic identification system 1 may also have the function of a video processing device that performs an erasure process.

[0019] The distraction model storage unit 16 is connected to the model learning unit 15. The distraction model storage unit 16 stores the trained distraction model 102.

[0020] In order to improve the accuracy of learning of the distraction model 102, the distraction characteristic grasping system 1 may repeatedly execute operations including selecting the properties of an object, presenting the object (an image with the object superimposed thereon), acquiring the attention level data, and learning of the distraction model 102. The repeated processing may be executed based on a preset number of repetitions, or may be executed based on a determination result for a preset determination value (e.g., the user's attention level).

[0021] 1.2 Flow of Learning the Distraction Model Next, an example of learning the distraction model 102 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of learning the distraction model 102.

[0022] 2 , first, the property selection unit 11 selects the property of an object to be presented on the object presentation unit 12 (S1). For example, when an acquired attention level log 101 and a trained distraction model 102 are present, the property selection unit 11 selects the property of the object based on the attention level log 101 and the distraction model 102. On the other hand, when an acquired attention level log 101 and a trained distraction model 102 are not present, the property selection unit 11 selects the property of the object based on, for example, a preset initial value. The property selection unit 11 transmits property information of the object to the object presentation unit 12.

[0023] Next, the object presentation unit 12 generates an object having the property based on the property information of the object. Then, the object presentation unit 12 presents the image on which the object is superimposed to the user (S2). The object presentation unit 12 also transmits the presentation information of the object to the attention level acquisition unit 13 and the attention level log storage unit 14.

[0024] Next, the attention level acquisition unit 13 acquires attention level data of the user who viewed the video on which the object is superimposed (S3). The attention level acquisition unit 13 transmits the attention level data to the attention level log storage unit 14.

[0025] The caution level log storage unit 14 stores (memorizes) the caution level log 101 including the presentation information of the object and the corresponding caution level data (S4). The caution level log storage unit 14 transmits the caution level log 101 to the model learning unit 15.

[0026] The model learning unit 15 uses the attention level log 101 as training data to learn the distraction model 102 (S5).

[0027] If the model learning unit 15 repeats learning of the distraction model 102 (S6_Yes), it transmits the learned distraction model 102 to the property selection unit 11 and returns to step S1. The property selection unit 11 acquires the attention level log 101 from the attention level log storage unit 14. Then, the property selection unit 11 selects a property different from the previously selected property based on the attention level log 101 and the learned distraction model 102.

[0028] If the model learning unit 15 does not repeat the learning of the distraction model 102 ( S6 _No), the model learning unit 15 transmits the learned distraction model 102 to the distraction model storage unit 16 .

[0029] The distraction model storage unit 16 stores the distraction model 102 (S7).

[0030] 1.3 Specific Example of Object Presentation Next, a specific example of object presentation will be described with reference to Fig. 3. Fig. 3 is a diagram showing a specific example of object presentation.

[0031] As shown in FIG. 3 , the object presenting unit 12 generates the object 2000 based on property information of the object 2000. Then, the object presenting unit 12 presents the image 1000 on which the object 2000 is superimposed to the user. The user, for example, wears a gaze measurement device 3000 and views the image 1000 on which the object 2000 is superimposed. The gaze measurement device 3000 measures the user's gaze at this time. The attention level acquiring unit 13 calculates attention level data based on the gaze data measured by the gaze measurement device 3000. Note that the gaze measurement device 3000 may also have a function as a head-mounted display. Furthermore, a biological signal measurement device may be used instead of the gaze measurement device 3000.

[0032] 1.4 Specific Example of Caution Log Next, a specific example of the caution log 101 will be described with reference to Fig. 4. Fig. 4 is a table showing a specific example of the caution log 101.

[0033] As shown in FIG. 4 , for example, the attention level log 101 includes information regarding “time,” “object range,” “nature,” and “attention level.” The “object range” and “nature” are based on the presentation information of the object. The “attention level” is based on the attention level data. The attention level log 101 may be generated for each user, or one attention level log 101 may be generated for multiple users.

[0034] The "time" indicates, for example, the time when the attention level acquisition unit 13 acquired the attention level data. Note that the "time" may also be, for example, the time when the object presentation unit 12 presented the object (a video image with the object superimposed) to the user. The attention level log 101 may further include information regarding the date. In the example shown in FIG. 4, 12:34 is registered in the attention level log 101 as the "time."

[0035] The "object range" indicates the display position of an object in the video and the size of the object. For example, "X" and "Y" in the "object range" represent the X and Y coordinates of the reference pixel of the object in the video. Furthermore, for example, "W" and "H" in the "object range" represent the width (W) and height (H) of the object. Note that each parameter of the "object range" can be set appropriately depending on the shape of the object, etc. In the example shown in FIG. 4, "250", "200", "300", and "150" are registered in the attention level log 101 as "X", "Y", "W", and "H" of the "object range", respectively.

[0036] "Quality" indicates the quality of the object being presented. For example, parameters related to "quality" include "type," "saturation," "brightness," "hue," "texture," and "speed."

[0037] "Type" indicates the type of object. In the example shown in Fig. 4, "automobile" is registered as the "type" in the warning level log 101.

[0038] "Saturation," "brightness," and "hue" indicate three attributes of the color of an object. "Saturation," "brightness," and "hue" may each indicate, for example, an average value. In the example shown in FIG. 4, "75," "90," and "270" are registered in the attention log 101 as "saturation," "brightness," and "hue," respectively.

[0039] "Texture" indicates the texture of an object. For example, parameters (features) related to "texture" include "contrast," "correlation," "entropy," and "angular second moment" based on a co-occurrence matrix. Note that the parameters related to "texture" are not limited to these. In the example shown in FIG. 4 , "13.2," "0.95," "5.5," and "0.0125" are registered in the attention level log 101 as "contrast," "correlation," "entropy," and "angular second moment," respectively.

[0040] "Speed" indicates information relating to the moving speed of an object in the video. In the example shown in Fig. 4, "15" is registered in the warning level log 101 as "Speed."

[0041] The "attention level" indicates the user's attention level data acquired by the attention level acquisition unit 13. In the example shown in Fig. 4, "80" is registered in the attention level log 101 as the "attention level."

[0042] 1.5 Specific Examples of Distraction Model Next, two specific examples of the distraction model 102 will be described with reference to Figures 5 and 6. Figure 5 is a table showing specific examples of logistic regression coefficients for each parameter when a logistic regression model is used as the distraction model 102. Figure 6 is a specific example of a three-dimensional graph displaying the relationship between the likelihood of distraction, saturation, and contrast based on the distraction model 102.

[0043] First, a specific example of the logistic regression coefficient will be described with reference to FIG.

[0044] 5, for example, the model learning unit 15 calculates a logistic regression coefficient for each parameter of "object range" and "properties" for each user (user ID) through learning using the attention level log 101 described with reference to Fig. 4 as training data. For example, the magnitude of the logistic regression coefficient indicates the magnitude of the influence on the attention level (i.e., the likelihood of distraction). In the example shown in FIG. 5 , “0.84”, “0.92”, “0.32”, and “0.20” are registered as the logistic regression coefficients of “X”, “Y”, “W”, and “H” of the “object range”, respectively. “0.73”, “0.18”, and “0.57” are registered as the logistic regression coefficients of “saturation”, “lightness”, and “hue”, respectively. “0.73”, “0.18”, “0.57”, and “0.29” are registered as the logistic regression coefficients of “contrast”, “correlation”, “entropy”, and “angular second moment”, respectively. “0.17” is registered as the logistic regression coefficient of “velocity”.

[0045] Next, a specific example of a three-dimensional graph will be described with reference to FIG.

[0046] 6, when focusing on the saturation and contrast of the distraction model 102, the likelihood of distraction increases as the saturation and contrast increase, i.e., as the saturation and contrast increase, the user's attention level decreases.

[0047] 1.6 Hardware Configuration Next, an example of the hardware configuration of the distraction characteristic grasping system 1 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of the hardware configuration of the distraction characteristic grasping system 1.

[0048] As shown in FIG. 7, the distraction characteristic grasping system 1 includes a control device 20, a measuring device 31, a display device 32, an input device 33, and an output device .

[0049] The control device 20 is, for example, a computer, and includes a processor 21, a read-only memory (ROM) 22, a random access memory (RAM) 23, a storage medium 24, and an input / output interface 25.

[0050] The processor 21, ROM 22, RAM 23, storage medium 24, and input / output interface 25 are electrically connected to one another via a bus 26. The processor 21, ROM 22, RAM 23, storage medium 24, and input / output interface 25 transmit and receive data or control signals via the bus 26.

[0051] The processor 21 is configured with a general-purpose hardware processor including, for example, a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit). The processor 21 controls the entire control device 20. The processor 21 controls the entire distraction characteristic identification system 1. The processor 21 interprets and executes programs non-temporarily stored in the ROM 22 and the storage medium 24. For example, the processor 21 executes the distraction characteristic identification program stored in the storage medium 24, thereby causing the components of the distraction characteristic identification system 1, namely, the property selection unit 11, the object presentation unit 12, the attention level acquisition unit 13, and the model learning unit 15, to perform their functions.

[0052] The ROM 22 is a non-volatile memory. The ROM 22 serves as a non-transitory storage medium and stores programs for the processor 21 to execute various processes. For example, the ROM 22 stores an operating system (OS) and various application programs. For example, the processor 21 loads firmware from the ROM 22 into the RAM 23 and executes the firmware.

[0053] The RAM 23 is a volatile memory. The RAM 23 is a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 23 temporarily stores programs used in processing by the processor 21 and data used to execute the programs. The processor 21 executes the programs deployed in the RAM 23 to calculate the data in the RAM 23 and store the calculation results in the RAM 23. The RAM 23 also stores an attention level log 101 and a distraction model 102. That is, the RAM 23 functions as the attention level log storage unit 14 and the distraction model storage unit 16.

[0054] The storage medium 24 is a nonvolatile memory. For example, the storage medium 24 includes a nonvolatile memory such as a hard disk drive (HDD) or a solid state drive (SSD). The storage medium 24 non-temporarily stores a program executed by the processor 21 and data required for executing the program. For example, the storage medium 24 stores, as a non-temporary storage medium, a distraction characteristic identification program that causes the control device 20 to function as the distraction characteristic identification system 1. The storage medium 24 also stores, as non-volatile data, an attention level log 101 and a distraction model 102.

[0055] The program executed by the processor 21 may be provided to the control device 20 via a readable non-transitory storage medium (not shown). Such a storage medium is called a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media include disks such as flexible disks, optical disks (CD-ROM, CD-R, DVD-ROM, DVD-R, etc.), and magneto-optical disks (MO, etc.), as well as semiconductor memories.

[0056] The input / output interface 25 is connected to the measurement device 31, the display device 32, the input device 33, and the output device 34. For example, an external device that outputs the distraction model 102 is connected to the input / output interface 25. The input / output interface 25 enables input of a user's biosignal or gaze data from the measurement device 31, display of an image on the display device 32, input of information from the input device 33, and output of information to the output device 34. For example, the input / output interface 25 enables output of the distraction model 102 to an external device. For example, the input / output interface 25 may be a wired interface or a wireless interface. A wired interface includes a port to which a device is connected. For example, a wireless interface has a function that satisfies a communication standard.

[0057] The measurement device 31 is, for example, a biosignal measurement device or a gaze measurement device. The measurement device 31 transmits the measured biosignal or gaze data to the control device 20.

[0058] The display device 32 includes a display (such as an LCD (Liquid Crystal Display), an EL (Electroluminescence) display, or a cathode ray tube). The display device 32 can display an image on which an object is superimposed, etc.

[0059] The input device 33 may include a keyboard, a mouse, a touch panel, a receiving device, a disk drive, etc. The input device 33 is not limited to these, and may include any other input device.

[0060] The output device 34 may include a transmitting device, a disk drive, etc. The output device 34 is not limited to these and may include any other output device. Note that the input device 33 and the output device 34 may be configured as an input / output device having the functions of both the input device 33 and the output device 34.

[0061] 1.7. Effects of the Present Embodiment With the configuration of the present embodiment, the distraction characteristic identification system 1 can select the properties of an object to be superimposed on a video image based on the attention level log 101 and the trained distraction model 102. The distraction characteristic identification system 1 can generate an object corresponding to the selected properties and present the video image with the object superimposed to the user. The distraction characteristic identification system 1 can acquire attention level data of a user who views the video image with the object superimposed. The distraction characteristic identification system 1 can acquire the attention level log 101 including object presentation information and attention level data. The distraction characteristic identification system 1 can then use the attention level log 101 as training data to train the distraction model 102. This allows the distraction characteristic identification system 1 to learn the distraction model without actually observing a state in which distraction occurs.

[0062] Furthermore, with the configuration according to this embodiment, the distraction characteristic assessment system 1 can select objects with a variety of properties. This allows the distraction characteristic assessment system 1 to acquire a large amount of attention level data. That is, the distraction characteristic assessment system 1 can acquire a large amount of attention level logs 101 as training data. Therefore, the distraction characteristic assessment system 1 can improve the accuracy of learning the distraction model 102.

[0063] Furthermore, with the configuration according to this embodiment, the distraction characteristic assessment system 1 can select the properties of an object based on the attention level log 101 and the trained distraction model 102. Therefore, the distraction characteristic assessment system 1 can efficiently acquire the attention level log 101 that is effective for training the distraction model 102. Therefore, the distraction characteristic assessment system 1 can improve the accuracy of training the distraction model 102.

[0064] 2. Second Embodiment Next, a second embodiment will be described. In the second embodiment, a configuration of the distraction characteristic identification system 1 that is different from that of the first embodiment will be described. The following description will focus on the differences from the first embodiment.

[0065] 2.1 Functional Configuration First, an example of the functional configuration of the distraction characteristic identification system will be described with reference to Fig. 8. Fig. 8 is a block diagram showing an example of the functional configuration of the distraction characteristic identification system 1. Note that in the example shown in Fig. 8, connections between components are indicated by arrows, but connections between components are not limited to this.

[0066] As shown in Figure 8, the distraction characteristic understanding system 1 of this embodiment includes a property selection unit 11, an object presentation unit 12, an attention level acquisition unit 13, an attention level log storage unit 14, a distraction model storage unit 16, a property adjustment unit 17, and a logistic regression unit 18.

[0067] The property selection unit 11 is connected to the object presentation unit 12, the attention log storage unit 14, and the logistic regression unit 18. The property selection unit 11 acquires an attention log 101 from the attention log storage unit 14. The property selection unit 11 acquires a trained distraction model 102 from the logistic regression unit 18. As in the first embodiment, the property selection unit 11 selects (sets) the property of an object to be presented to the object presentation unit 12 based on the attention log 101 and the distraction model 102. The property selection unit 11 transmits property information of the object to the object presentation unit 12.

[0068] The functions and connections of the object presenting unit 12 and the attention level acquiring unit 13 are the same as those described with reference to FIG. 1 in the first embodiment.

[0069] The caution level log storage unit 14 is connected to the property selection unit 11, the object presentation unit 12, the caution level acquisition unit 13, and the property adjustment unit 17. The caution level log storage unit 14 transmits a caution level log 101 to the property adjustment unit 17.

[0070] The property adjustment unit 17 is connected to the attention level log storage unit 14 and the logistic regression unit 18. The property adjustment unit 17 adjusts the attention level log 101, for example, so that the attention level monotonically increases depending on an increase in the value of each parameter of the property (and object range). Note that the property adjustment unit 17 may also adjust the attention level log 101 so that the likelihood of distraction monotonically increases depending on an increase in the value of each parameter of the property (and object range). The property adjustment unit 17 transmits the adjusted attention level log 101 to the logistic regression unit 18.

[0071] The logistic regression unit 18 is connected to the property selection unit 11, the distraction model storage unit 16, and the property adjustment unit 17. As in the first embodiment, the logistic regression unit 18 uses the adjusted attention level log 101 as training data to perform training of the distraction model 102. The logistic regression unit 18 uses a logistic regression model as a training model for the distraction model 102. Therefore, the logistic regression unit 18 is the model training unit 15 when the training model is a logistic regression model. The logistic regression unit 18 transmits the trained distraction model 102 to the property selection unit 11 and the distraction model storage unit 16. Furthermore, for example, the logistic regression unit 18 outputs the trained distraction model 102 to the outside.

[0072] The distraction model storage unit 16 is connected to the logistic regression unit 18. The distraction model storage unit 16 stores a trained distraction model 102.

[0073] 2.2 Flow of Learning the Distraction Model Next, an example of learning the distraction model 102 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of learning the distraction model 102.

[0074] As shown in FIG. 9, the operations in steps S1 to S3 are the same as those explained with reference to FIG. 2 in the first embodiment.

[0075] The caution log storage unit 14 stores the caution log 101 in the same manner as in the first embodiment (S10). The caution log storage unit 14 transmits the caution log 101 to the property adjustment unit 17.

[0076] The property adjustment unit 17 adjusts the attention level log 101 so that the attention level monotonically increases with an increase in the values ​​of the object range and property parameters (S11). That is, the property adjustment unit 17 adjusts the object properties. The property adjustment unit 17 transmits the adjusted attention level log 101 to the logistic regression unit 18.

[0077] The logistic regression unit 18 uses the adjusted attention level log 101 as training data to perform learning of the distraction model 102 (S12).

[0078] If the logistic regression unit 18 is to repeat learning of the distraction model 102 (S13_Yes), it transmits the learned distraction model 102 to the property selection unit 11 and returns to step S1. The property selection unit 11 acquires the attention level log 101 from the attention level log storage unit 14. Then, the property selection unit 11 selects a property different from the previously selected property based on the attention level log 101 and the learned distraction model 102.

[0079] If the logistic regression unit 18 does not repeat the learning of the distraction model 102 (S13_No), the logistic regression unit 18 transmits the learned distraction model 102 to the distraction model storage unit 16.

[0080] The distraction model storage unit 16 stores the distraction model 102, as in the first embodiment (S7).

[0081] 2.3 Specific Example of Property Adjustment Next, a specific example of property adjustment will be described with reference to FIG. 10 . FIG. 10 is an explanatory diagram of a specific example of property adjustment. The example shown in FIG. 10 focuses on the "X" and "Y" of the "object range" in the attention log 101, and illustrates an example of adjusting the parameters of the attention log 101 so that the "attention level" monotonically increases with respect to the position of an object superimposed on the video. (a) of FIG. 10 is a diagram illustrating the relationship between the likelihood of distraction and the position of an object on the horizontal axis in the video. Here, the position of an object on the horizontal axis in the video refers to the position of the object moved horizontally (in the X direction) from the center point of the video. (b) of FIG. 10 is a diagram illustrating the relationship between the attention level and the distance of the object from the center point of the video.

[0082] As shown in (a) of Figure 10, for example, the likelihood of distraction is normally distributed with respect to the position on the horizontal axis of an object superimposed on the image. Similarly, the likelihood of distraction is normally distributed with respect to the position on the vertical axis of an object superimposed on the image. Here, the position of an object on the vertical axis refers to the position of the object moved vertically (in the Y direction) from the center point of the image. In other words, the closer the object is to the center point of the image (the shorter the distance from the object to the center point), the more likely it is that distraction will occur. In other words, the farther the object is from the center point of the image, the higher the level of attention.

[0083] 10(b), the property adjustment unit 17 adjusts "X" and "Y" of the "object range" in the attention level log 101 to the "distance from the center point." That is, the property adjustment unit 17 changes "X" and "Y" of the "object range" to the "distance from the center point." As a result, the attention level log 101 is adjusted so that the "attention level" monotonically increases with respect to the "distance from the center point."

[0084] 2.4 Specific Example of Adjusted Attention Log Next, a specific example of the adjusted attention log 101 will be described with reference to Fig. 11. Fig. 11 is a table showing a specific example of the adjusted attention log 101. The example shown in Fig. 11 shows the result of adjusting the attention log 101 shown in Fig. 4.

[0085] As shown in FIG. 11 , for example, the property adjustment unit 17 adjusts "X" and "Y" of the "object range" to "distance from the center point" for the attention log 101 described with reference to FIG. 4 . Furthermore, the property adjustment unit 17 adjusts "brightness" and "contrast" of the "property" to "Δbrightness" and "Δcontrast," respectively, for the attention log 101 described with reference to FIG. 4 . For example, "Δbrightness" and "Δcontrast" respectively indicate the difference between the average value of the image and the average value of the object. In the example shown in FIG. 11 , "500" is registered in the attention log 101 as the "distance from the center point." Furthermore, "40" and "3.2" are registered as the "Δbrightness" and "Δcontrast," respectively. The other parameters are the same as those in FIG. 4 .

[0086] 2.5 Specific Example of Distraction Model Next, a specific example of the distraction model 102 will be described with reference to Fig. 12. Fig. 12 is a table showing specific examples of the logistic regression coefficients of the parameters of the distraction model 102.

[0087] 12, for example, the logistic regression unit 18 calculates a logistic regression coefficient for each parameter of "object range" and "properties" for each user (user ID) through learning using the attention level log 101 described with reference to FIG. 11 as training data. In the example shown in FIG. 12, "0.88" is registered as the logistic regression coefficient for "distance from center point." "0.13" and "0.73" are registered as the logistic regression coefficients for "Δbrightness" and "Δcontrast," respectively. The other parameters are the same as those in FIG. 5.

[0088] 2.6 Effects of this embodiment With the configuration of this embodiment, the same effects as those of the first embodiment can be obtained.

[0089] Furthermore, with the configuration according to this embodiment, the distraction characteristic identification system 1 can adjust the attention level log 101 so that the attention level monotonically increases with an increase in the parameters of the object property and the object range, thereby enabling more efficient learning of the distraction model 102.

[0090] 3. Other Embodiments The present invention is not limited to the above-described embodiments.

[0091] For example, the distraction characteristic grasping system 1 may be incorporated into a video processing system that performs concealment processing.

[0092] Also, for example, the distraction characteristic grasping system 1 described in the first embodiment may include the property adjusting unit 17 described in the second embodiment.

[0093] Various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0094] 1...Distraction characteristic grasping system 11...Property selection unit 12...Object presentation unit 13...Attention level acquisition unit 14...Attention level log storage unit 15...Model learning unit 16...Distraction model storage unit 17...Property adjustment unit 18...Logistic regression unit 20...Control device 21...Processor 22...ROM 23...RAM 24...Storage medium 25...Input / output interface 26...Bus 31...Measuring device 32...Display device 33...Input device 34...Output device 101...Attention level log 102...Distraction model 1000...Video 2000...Object 3000...Gaze measurement device

Claims

1. A system for understanding distraction characteristics, comprising: a property selection unit that selects the property of an object to be superimposed on an image; an object presentation unit that generates the object based on the selected property and presents the image with the object superimposed to a user; an attention level acquisition unit that acquires the attention level of the user who views the image; and a model learning unit that performs learning of a distraction model based on an attention level log that includes presentation information of the object and the attention level.

2. The distraction characteristic grasping system according to claim 1, further comprising: an attention log storage unit in which the attention log is stored; and a distraction model storage unit in which the distraction model is stored.

3. The distraction characterization system of claim 1, wherein the properties include at least one of saturation, brightness, hue, magnitude, texture, and speed as parameters.

4. The distraction characterization system of claim 3, wherein the texture includes at least one of contrast, correlation, entropy, and angular second moment as parameters.

5. The distraction characteristic grasping system according to claim 1, wherein the attention level acquisition unit calculates the attention level based on the measurement results of the user's eye movement or the user's biological signals.

6. The distraction characterization system of claim 1, wherein the distraction model is a logistic regression model.

7. The distraction characteristic grasping system according to claim 6, further comprising a property adjusting unit that adjusts the attention level log so that the attention level monotonically increases with an increase in the value of each parameter of the attention level log.

Images