Method for evaluating sound or image, device for evaluating sound or image, and program
The method allows users to evaluate sounds or images by editing icon positions on a plane, eliminating ambiguity in word-based evaluations and enabling precise emotional assessment through a two-dimensional space, facilitating sensitivity evaluation of unknown content.
Patent Information
- Application Number
- PCT/JP2024/015084
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-24
- Filing Date
- 2024-04-16
- Publication Date
- 2024-11-28
AI Technical Summary
Conventional methods for emotional evaluation of sounds or images rely on subjective evaluation words, which are ambiguous and fail to accurately express individual perceptions due to multidimensional impressions, leading to incomplete cognition of the evaluator's experience.
A method and device that display three icons on a plane, allowing users to edit their positions using an input device, associating the edited coordinates with sounds or images, enabling emotional evaluation without words by expressing relationships in a two-dimensional space.
This approach allows for precise emotional evaluation of sounds or images by eliminating individual differences in word recognition and using similarity scales, enabling effective expression and evaluation without evaluation words or scales, and allows for sensitivity evaluation of unknown sounds or images through machine learning models.
Smart Images

Figure JP2024015084_28112024_PF_FP_ABST
Abstract
Description
Sound or image evaluation method, sound or image evaluation device and program
[0001] The present invention relates to a technique for evaluating sounds or images.
[0002] Affective evaluations are often performed on information such as sounds. Evaluation words such as warm and comfortable are used in the affective evaluations. For example, Patent Document 1 listed below presents a method for subjective evaluation when a stimulus is presented. When a stimulus is presented, a subject selects the degree to which the stimulus applies to affective words (refreshing, relaxing, etc.).
[0003] JP 2022-154800 A
[0004] However, because the impressions and perceptions of evaluation words vary from person to person, evaluation words contain ambiguity. Furthermore, people who perceive information consciously or unconsciously perceive it not as a single evaluation word, but as a multidimensional space made up of many evaluation axes. Therefore, it cannot be said that conventional methods of sensory evaluation are necessarily able to accurately represent the perceptions of the evaluator.
[0005] An object of the present invention is to express a user's emotional evaluation of a sound or an image without using evaluation words.
[0006] A method for evaluating sounds or images according to one aspect of the present invention displays three icons in a plane displayed on an image display device, outputs a sound or image corresponding to each icon, edits the position of each icon in the plane in response to a first operation in which a user uses an input device to move each icon in the plane, and associates the edited planar coordinates of each icon with a sound or image.
[0007] The present invention is also directed to a sound or image evaluation device and program.
[0008] According to the present invention, a user's emotional evaluation of a sound or an image can be expressed without using evaluation words.
[0009] 1 is a configuration diagram of a physical information evaluation device according to an embodiment. FIG. 2 is a diagram showing the data structure of an affective object. FIG. 3 is a functional block diagram of a physical information evaluation device according to an embodiment. FIG. 4 is a diagram showing the data structure of coordinate data. FIG. 5 is a diagram showing an operation interface for affective evaluation displayed on a display. FIG. 6 is a diagram showing movement of icons displayed on the operation interface. FIG. 7 is a diagram showing icons arranged based on a user's affective evaluation. FIG. 8 is a diagram showing icons arranged based on a user's affective evaluation. FIG. 9 is a diagram showing icons arranged based on a user's affective evaluation. FIG. 10 is a flowchart showing an affective evaluation method for physical information according to an embodiment. FIG. 11 is a flowchart showing affective evaluation processing according to an embodiment. FIG. 12 is a flowchart showing affective space generation processing according to an embodiment. FIG. 13 is a diagram showing a state in which a plurality of affective objects are arranged in a common metric space.
[0010] Hereinafter, a method, an apparatus, and a program for evaluating sound or images according to embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, information including sound and images will be described as physical information.
[0011] {1. Configuration of the Physical Information Evaluation Device} Fig. 1 is a block diagram showing the configuration of a physical information evaluation device 1 according to this embodiment. The physical information evaluation device 1 is configured, for example, by a personal computer. As shown in Fig. 1, the physical information evaluation device 1 includes a CPU (Central Processing Unit) 20, a ROM (Read Only Memory) 21, a RAM (Random Access Memory) 22, a storage device 23, an input device 24, a display 25, a sound source unit 26, a sound system 27, a communication interface (I / F) 28, and a device interface (I / F) 29. These devices are connected via a system bus 30.
[0012] The CPU 20 controls the overall operation of the physical information evaluation device 1. The ROM 21 stores control programs executed by the CPU 20 and various data. The RAM 22 is used as a work area for the CPU 20. The storage device 23 stores the program PR, coordinate data CD, sensory objects BJ, and audio data SD. The storage device 23 can be configured with various storage media such as a hard disk or semiconductor memory. The program PR executes processes such as displaying an icon BJm (an image representing the sensory object BJ) on the display 25, editing the display position of the icon BJm, controlling the playback of audio data SD associated with the icon BJm, and generating an emotional space.
[0013] The input device 24 is an interface that accepts various operations by the user, such as an operation to move the icon BJm and an instruction to play audio data SD. The display 25 is used to display the icon BJm, etc. The sound source unit 26 and the sound system 27 play audio data SD associated with the icon BJm (i.e., associated with the emotional object BJ) in response to an instruction to play audio data SD by the user. The communication interface 28 is an interface that performs wired or wireless communication with other computers. The device interface 29 is an interface that accesses a storage medium 31 such as a CD, DVD, or semiconductor memory.
[0014] The program PR will be described as being stored in the storage device 23. In another embodiment, the program PR may be provided by being stored in a storage medium 31. The CPU 20 may access the storage medium 31 via the device interface 29 and store the program PR stored in the storage medium 31 in the storage device 23 or the ROM 21. Alternatively, the CPU 20 may access the storage medium 31 via the device interface 29 and execute the program PR stored in the storage medium 31. Alternatively, the CPU 20 may download the program PR from a server on the network via the communication interface 28 and store the downloaded program PR in the storage device 23 or the ROM 21. Alternatively, the CPU 20 may directly execute the program PR stored in a server on the network via the communication interface 28.
[0015] {2. Data Structure of Sensory Object BJ} FIG. 2 is a diagram showing the data structure of a sensory object BJ. The sensory object BJ includes an identification number BJn and an icon BJm. The identification number BJn is a number (or symbol) assigned to multiple sensory objects BJ and uniquely identifies each sensory object BJ. The icon BJm is an image for representing the sensory object BJ on the display 25. Multiple sensory objects BJ are stored in the storage device 23. Each sensory object BJ is associated with audio data SD in a one-to-one relationship. For example, the identification number BJn is recorded in header information of the audio data SD, and the sensory object BJ and the audio data SD are associated with each other via the identification number BJn. Multiple audio data SD corresponding to multiple sensory objects BJ are stored in the storage device 23.
[0016] Note that when a plurality of sensory objects BJ use the same icon BJm, the sensory objects BJ do not need to hold the icon BJm. In that case, it is sufficient to use a common icon BJm stored in the storage device 23. Furthermore, the sensory objects BJ may hold data defining the icon BJm, rather than holding the icon BJm as image data.
[0017] {3. Functional Configuration of the Physical Information Evaluation Device} Fig. 3 is a block diagram showing the functional configuration of the physical information evaluation device 1. In Fig. 3, the control unit 10 is a functional unit that is realized by the CPU 20 executing the program PR while using the RAM 22 as a work area. The control unit 10 includes an icon display unit 11, a position editing unit 12, a coordinate registration unit 13, a playback unit 14, and an affective space generation unit 15. The icon display unit 11, the position editing unit 12, the coordinate registration unit 13, the playback unit 14, and the affective space generation unit 15 are functional units that are realized by the execution of the program PR. In other words, each of the functional units 11 to 15 is a functional unit that the CPU 20 includes.
[0018] The icon display unit 11 displays the icons BJm on the display 25 in a movable state. The icon display unit 11 displays the three selected icons BJm in a display area 25A (see FIG. 5, etc.) on the display 25. That is, the icon display unit 11 selects three sensory objects BJ from a plurality of sensory objects BJ stored in the storage device 23 according to a predetermined rule. Then, the icon display unit 11 displays the icons BJm corresponding to the three selected sensory objects BJ on the display 25. In an initial state, the icon display unit 11 can display the icons BJm at any position within the display area 25A. For example, in the initial state, the icon display unit 11 may display the three icons BJm at equal intervals.
[0019] The rule by which the icon display unit 11 selects three sensory objects BJ from the plurality of sensory objects BJ is not particularly limited. For example, the three sensory objects BJ may be selected randomly. Alternatively, the sensory objects BJ may be selected in order based on the identification number BJn.
[0020] The position editing unit 12 edits the position of the icon BJm in the display area 25A of the display 25 in accordance with an operation received by the input device 24. The user uses the input device 24, such as a mouse or keyboard, to perform an operation to move the position of the icon BJm displayed on the display 25. If the display 25 is a touch panel display, the display 25 can also function as the input device 24. In this case, the user can edit the position of the icon BJm by touching the display, which serves as the input device 24, with a finger or a pen.
[0021] The coordinate registration unit 13 registers coordinate data CD based on the positions of the icons BJm edited by the position editing unit 12. The user performs an editing operation on the three icons BJm displayed on the display 25. In response to this operation, the coordinate registration unit 13 registers the coordinates of each icon BJm after it has been moved in the coordinate data CD. The coordinate registration unit 13 registers the planar coordinates of the three icons BJm displayed on the display 25 within the display area 25A in the coordinate data CD. An identification number BJn corresponding to each icon BJm is registered in the coordinate data CD in association with the coordinates.
[0022] 4 is a diagram showing the data structure of the coordinate data CD. FIG. 4 shows one data set included in the coordinate data CD. One data set includes the identification numbers BJn and coordinates of three icons BJm. The coordinate data CD is data in which a plurality of such data sets are recorded. Each of the plurality of data sets is made up of a combination of three different icons BJm. In other words, in a set of three icons BJm included in one data set, at least one icon BJm is different from the other data sets.
[0023] The playback unit 14 plays back the audio data SD associated with the sensory object BJ in response to an operation received by the input device 24. The user issues an instruction to play back audio for any of the icons BJm displayed on the display 25 by operating, for example, a mouse or a keyboard. In response to this operation, the playback unit 14 issues an instruction to play back the audio data SD to the sound source unit 26. An audio signal corresponding to the audio data SD output from the sound source unit 26 is output by the sound system 27.
[0024] The user can check the audio associated with the icon BJm (sensory object BJ) by issuing an instruction to play audio for the icon BJm displayed on the display 25. The user listens to the audio associated with each of the three icons BJm displayed on the display 25 and evaluates the three audio sounds from the perspective of similarity, i.e., whether they are similar or dissimilar. The user moves two icons BJm that the user feels are similar on the display 25 so that they move closer together, and moves two icons BJm that the user feels are dissimilar on the display 25 so that they move farther apart. In other words, the user moves each icon BJm on the display 25 so that the distance between two icons whose audio sounds the user feels are similar becomes shorter than the distance between two icons whose audio sounds the user feels are distant. Since the three icons BJm are displayed on the display 25, the user expresses the mutual similarity between the three sensory objects BJ within the display area 25A on the display 25, which is a two-dimensional plane. The user may enlarge or reduce the display on the display 25 as needed.
[0025] The sensory space generation unit 15 performs matching processing and mapping processing. The matching processing is processing for unifying the scale of distance between all sensory objects BJ (a space with a unified scale of distance will be referred to as a distance space SP). As described above, the position editing unit 12 edits the coordinates of the icons BJm displayed in groups of three. Then, the coordinate registration unit 13 registers the coordinates of the edited icons BJm. The sensory space generation unit 15 uses the coordinate data CD registered by the coordinate registration unit 13 to calculate distances between the sensory objects BJ such that all the sensory objects BJ have a consistent scale. The mapping processing maps the sensory objects BJ into a multidimensional space based on this distance relationship.
[0026] {4. Operation Interface for Affective Evaluation Using Icons} Next, an operation interface for affective evaluation using icons BJm will be described with reference to FIGS. 5 to 10. FIG. 5 shows a display area 25A for icons BJm displayed on the display 25 by the program PR. The display area 25A is an area for displaying the icons BJm and is also used as an operation interface for the user to perform affective evaluation. Three icons BJm(1), BJm(2), and BJm(3) are displayed in the display area 25A by the icon display unit 11. The icons BJm(1) to (3) correspond to three affective objects BJ(1) to (3). In the example shown in FIG. 5, the icon display unit 11 initially displays the three icons BJm(1) to (3) at equal intervals.
[0027] The user can listen to the audio data SD corresponding to each icon BJm(1)-(3) by clicking the icons BJm(1)-(3) displayed in the display area 25A, for example, with a mouse. The user then moves the icons BJm with similar sounds closer together or moves the icons BJm with dissimilar sounds farther apart. Figure 6 shows how the position of the icon BJm(3) has been edited by the user. The user repeats the operations of moving the icons BJm(1)-(3) and listening to the audio associated with the icons BJm(1)-(3) as necessary to reflect the affective evaluations of the affective objects BJ(1)-(3) on the display area 25A.
[0028] 7 to 10 are diagrams showing the arrangement of icons BJm(1) to (3) that have been subjectively evaluated by a user. As shown in FIG. 7, in the display area 25A, icon BJm(1) and icon BJm(2) are arranged close to each other. Icon BJm(3) is arranged at a distance from icons BJm(1) and BJm(2). In this example, the user evaluates the sounds associated with icons BJm(1) and BJm(2) as similar, and the sound associated with icon BJm(3) as dissimilar to the sounds associated with icons BJm(1) and (2). Because the distance between icon BJm(1) and icon BJm(3) and the distance between icon BJm(2) and icon BJm(3) are similar, the user evaluates icon BJm(3) as dissimilar to icons BJm(1) and (2) to the same extent.
[0029] In the example shown in Figure 8, the user evaluates that the audio associated with icon BJm(2) is similar to the audio associated with icons BJm(1) and BJm(3). On the other hand, the user evaluates that the audio associated with icon BJm(3) is dissimilar to the audio associated with icon BJm(1). Because the distance between icon BJm(1) and icon BJm(3) is much greater than the distance between icon BJm(2) and icons BJm(1) and (3), the user evaluates that the similarity between the audio associated with icon BJm(1) and icon BJm(3) is very low.
[0030] In the example shown in Figure 9, the user evaluates the sounds associated with icons BJm(1), BJm(2), and BJm(3) as being equally similar. Because the distances between icons BJm(1), (2), and (3) are relatively small, the user evaluates the three sounds as being relatively similar. In the example shown in Figure 10, the user evaluates the sounds associated with icons BJm(1), BJm(2), and BJm(3) as being less similar. Among these, the user evaluates the sounds associated with icons BJm(2) and BJm(3) as being less similar than the other relationships.
[0031] In the present embodiment, the reason for displaying three icons BJm in the display area 25A is that the relationship between the corresponding audio data SD can be uniquely expressed. For example, the plane on which two icons BJm are arranged is not uniquely specified. Also, for four or more icons BJm, the relationship may not be expressed on a plane.
[0032] {5. Sensory Evaluation Method for Physical Information} Next, a sensory evaluation method for physical information according to this embodiment will be described with reference to FIG. 11. The flowchart shown in FIG. 11 shows the overall processing flow of the sensory evaluation method. The processing shown in FIG. 11 is realized by executing the program PR on the CPU 20. First, in step S1, the control unit 10 performs sensory evaluation processing. Details of the sensory evaluation processing will be described using the flowcharts of FIG. 12 and FIG. 13. Next, in step S2, the control unit 10 performs sensory space generation processing. Details of the sensory space generation processing will be described using the flowchart of FIG. 14.
[0033] (5-1) Affective Evaluation Processing The affective evaluation processing (step S1 in FIG. 11) according to this embodiment will be described with reference to FIGS. 12 and 13. The flowcharts shown in FIGS. 12 and 13 are processes realized by the program PR being executed on the CPU 20. First, in step S11, the icon display unit 11 displays three icons BJm on the display 25. As shown in FIG. 5, the icon display unit 11 displays the three icons BJm in a movable state within the display area 25A.
[0034] Next, in step S12, the playback unit 14 determines whether or not an instruction to output sound corresponding to any of the icons BJm has been received. As described above, the user can operate the input device 24 to select any of the icons BJm displayed on the display 25 and issue an instruction to play sound. If an instruction to output sound for any of the icons BJm has been received, in step S13 the playback unit 14 plays sound corresponding to the selected icon BJm. After step S13, the process returns to step S12.
[0035] If an instruction to output a sound for an icon BJm is not received in step S12, then in step S14 the position editing unit 12 determines whether an operation to move any of the icons BJm within the display area 25A has been received. If a move operation for any of the icons BJm has been received, then in step S15 the position editing unit 12 moves the selected icon BJm within the display area 25A. After step S15, the process returns to step S12.
[0036] If a move operation on the icon BJm is not received in step S14, the coordinate registration unit 13 determines in step S16 whether an instruction to end editing has been received. When the user finishes editing the three icons BJm displayed on the display 25, the user operates the input device 24 to issue an instruction to end editing. If an instruction to end editing has been received, the coordinate registration unit 13 registers the coordinates of the three icons BJm arranged in the display area 25A in the coordinate data CD in step S17. Specifically, the coordinate registration unit 13 associates the coordinates of the three icons BJm arranged in the display area 25A with the identification numbers BJn and registers them in the coordinate data CD. After step S17, the process proceeds to step S18 in FIG. 13 .
[0037] In step S18, the icon display unit 11 determines whether an instruction to end editing for all icons BJm has been received. When the user has finished editing all icons BJm corresponding to all emotional objects BJ to be evaluated, the user operates the input device 24 to issue an instruction to end all editing. If an instruction to end editing for all icons BJm has been received, the program PR terminates the emotional evaluation process. If an instruction to end editing for all icons BJm has not been received, that is, if an instruction to edit three new icons BJm (three new emotional objects BJ) has been received, in step S19, the icon display unit 11 selects another set of three icons BJm (another set of three emotional objects BJ). Here, the icon display unit 11 excludes from the selection any set of icons BJm that is the same as a set of icons BJm selected in the past. The icon display unit 11 selects a new set of three icons BJm (a new set of three emotional objects BJ) in which at least one icon BJm is different from the three icons BJm. After step S19, the process returns to step S11 in Fig. 12, and the icon display unit 11 displays the three newly selected icons BJm in the display area 25A. Then, steps S12 to S19 are repeatedly executed for the new icons BJm. As a result, the user executes the editing operation for all the icons BJm corresponding to all the affective objects BJ to be evaluated.
[0038] The affective evaluation of each affective object BJ needs to be performed at least once. In other words, the editing operation of the icon BJm needs to be performed at least once for each affective object BJ. In order to properly perform the affective evaluation of the affective objects BJ, it is desirable that the editing operation of the icon BJm be performed multiple times for one affective object BJ. In order to properly evaluate the mutual similarity of all the affective objects BJ, it is desirable that the editing operation of each icon BJm be performed within a set of as many icons BJm as possible.
[0039] In this way, the physical information evaluation device 1 of this embodiment can express the relationship between pieces of physical information in coordinates on a two-dimensional plane. By performing a perceptual evaluation of physical information without using evaluation words and evaluation scales, it is possible to eliminate individual differences in recognition of evaluation words and evaluate physical information using a similarity scale. By using the physical information evaluation device 1 of this embodiment, users can enjoy the customer experience of performing a perceptual evaluation of physical information without using evaluation words and evaluation scales.
[0040] Furthermore, by using the physical information evaluation device 1 of this embodiment, it is possible to perform a perceptual evaluation of an unknown sound or image. For example, a perceptual evaluation model can be generated by performing deep learning using the coordinate data CD registered by the processing of the above embodiment and the audio data SD associated with the identification number BJn. In other words, a trained perceptual evaluation model can be generated by machine learning the association between a sound or image and the edited planar coordinates of each icon. By providing an unknown sound or image to this trained perceptual evaluation model, perceptual evaluation of the unknown sound or image can be performed. By providing an unknown sound or image to the physical information evaluation device 1 of this embodiment, users can enjoy the customer experience of obtaining perceptual evaluation results. Alternatively, by creating a generative model, it is possible to generate a sound or image that is estimated to be perceptually similar to a certain sound.
[0041] (5-2) Sensory Space Generation Processing Next, the sensory space generation processing (step S2 in FIG. 11) according to this embodiment will be described with reference to FIG. 14. The flowchart shown in FIG. 14 is processing realized by executing the program PR on the CPU 20. First, in step S21, the sensory space generation unit 15 performs matching processing for all sensory objects BJ. The matching processing is processing for unifying the scale of distances between all sensory objects BJ based on the coordinate data CD stored in the storage device 23. That is, the sensory space generation unit 15 calculates the distances between all sensory objects BJ in a common metric space based on multiple combinations of the planar coordinates of three icons BJm obtained through multiple edits. Note that the matching processing may be considered as processing for arranging all icons BJm in a single metric space SP. However, since processing for displaying the icons BJm on the display 25 based on the coordinates after the matching processing is not essential, the matching processing will be described here assuming that the target of the matching processing is the sensory object BJ.
[0042] FIG. 15 is a diagram illustrating the concept of matching processing into the metric space SP. The diagram illustrates a state in which four sensory objects BJ(1) to (4) corresponding to four icons BJm(1) to (4) are arranged in the metric space SP based on two data sets included in the coordinate data CD. In other words, the diagram illustrates a state in which the scale of distance between the four sensory objects BJ(1) to (4) is unified. The coordinate data CD records a data set including icons BJm(1), (2), and (3) and a data set including icons BJm(1), (2), and (4). The sensory space generation unit 15 calculates the distance in the metric space SP between the sensory objects BJ(1) to (4) based on the two data sets that share the icons BJm(1) and (2), as shown in the diagram.
[0043] The method of matching processing by the sensory space generation unit 15 is not particularly limited. For example, the sensory space generation unit 15 calculates distances between each icon BJm that minimize the error in the distances based on multiple data sets recorded in the coordinate data CD. For example, for N icons BJm(1) to (N) to be evaluated, the difference between the estimated distances (distances in the metric space SP) and the distances between each icon based on the coordinates recorded in the coordinate data CD is calculated. Then, the least squares method is used to calculate the distances between each sensory object in the metric space SP so that the integrated value of the calculated differences is minimized. In the example shown in FIG. 15 , four sensory objects BJ are matched in the metric space SP based on two data sets, but the number of data sets increases depending on the number of sensory objects BJ. For example, if the number of sensory objects BJ is N, the maximum number of data sets is the number of combinations in which three sensory objects BJ are selected from the N sensory objects BJ.
[0044] Next, in step S22, the sensory space generation unit 15 maps all of the sensory objects Bj, for which the distance scale has been unified, into a multidimensional space. In step S21, the N sensory objects BJ(1) to (N) to be evaluated, for which the distances in the common metric space SP have been calculated, are further mapped into a multidimensional space.
[0045] The method of mapping to the multidimensional space by the sensory space generation unit 15 is not particularly limited. Mapping to the multidimensional space can be performed using known techniques. For example, the sensory space generation unit 15 performs mapping to the multidimensional space by using principal coordinate analysis.
[0046] {6. Other Embodiments} In the above embodiment, an example has been described in which the sensory object BJ (icon BJm) is associated with sound data SD. That is, the sensory object BJ (icon BJm) is associated with sound as physical information. The physical information evaluated by the physical information evaluation device 1 of the present embodiment is not limited to sound. The physical information evaluation device 1 of the present embodiment can evaluate an image as physical information in association with the icon BJm.
[0047] For example, the physical information evaluation device 1 can evaluate words (text information) presented to a user via physical information such as audio or images. Each affective object BJ is associated with a word. For example, by selecting an icon BJm using the input device 24, the user can hear or listen to the word associated with the icon BJm through audio. Alternatively, by selecting an icon BJm using the input device 24, the user can confirm the word associated with the icon BJm as an image (visual information). The icon BJm itself may include the word as an image. Other processing is similar to that of the above-described embodiment. By manipulating the icon BJm based on the meaning and nuance of the word, the user can evaluate the similarity of the words and project the affective evaluation onto the layout of the icons BJm on a plane.
[0048] {7. Correspondence between Components of the Claims and Elements of the Embodiments} Examples of correspondence between components of the claims and elements of the embodiments will be described below, but the present invention is not limited to the following examples. In the above embodiment, the display 25 is an example of an image display device, and the display area 25A is an example of a plane displayed on the image display device. Also, in the above embodiment, an operation of moving the icon BJm within the display area 25A using the input device 24 is an example of a first operation, and an operation of selecting the icon BJm using the input device 24 and instructing playback of the audio data SD is an example of a second operation.
[0049] As each component of a claim, various elements having the configuration or function described in the claim may be used.
[0050] 8. Features and Effects of the Embodiments A sound or image evaluation method according to one aspect of the present invention displays three icons in a plane displayed on an image display device, outputs a sound or an image corresponding to each icon, edits the position of each icon in the plane in response to a first operation in which a user uses an input device to move each icon in the plane, and associates the edited plane coordinates of each icon with the sound or image.
[0051] The relationship between sounds or images can be expressed as coordinates on a two-dimensional plane. Sounds or images can be evaluated without the need for evaluation words and evaluation scales. Sounds or images can be evaluated by using the distance on a plane that mathematically produces the least distortion.
[0052] In the first operation, the user may move each icon in a plane so that the distance between two icons whose sounds or images are perceived as being close is shorter than the distance between two icons whose sounds or images are perceived as being farther apart.
[0053] It is possible to perform a sensibility evaluation of sounds or images by determining whether they are similar or not, without using evaluation words.
[0054] In response to a second operation by the user to select an icon displayed on the image display device, a sound associated with the icon may be played.
[0055] The user can make an emotional evaluation while listening to the sound associated with each icon.
[0056] Three objects are selected from a group of objects corresponding to four or more sounds or images, and three icons corresponding to the selected three objects are displayed on the image display device; after editing, three new objects, each of which is different in at least one object, are selected from the group of objects, and multiple edits are repeated on the group of objects; and after each edit, the planar coordinates of the selected three icons are associated with the sound or image.
[0057] By performing an affective evaluation for each of the three objects selected in different combinations, it is possible to perform an affective evaluation based on relative relationships for four or more sounds or images.
[0058] Four or more objects corresponding to four or more sounds or images may be arranged in a common distance space based on multiple combinations of the planar coordinates of the three icons obtained through multiple edits.
[0059] The affective evaluations made on different object combinations can be expressed in a common metric space.
[0060] Four or more objects located in a common metric space may be mapped into a multi-dimensional space.
[0061] Four or more sounds or images can be represented in a multidimensional space based on the affective evaluations arranged in a distance space.
[0062] A distance space or a multidimensional space may be used to form an affective space that reflects affective evaluation of a sound or an image.
[0063] Words expressed by sounds or images may be associated with the icons.
[0064] A user's emotional evaluation of words can be expressed as coordinates on a two-dimensional plane via audio and images.
[0065] The affective evaluation model may be generated by machine learning the association between the sound or image and the planar coordinates of each icon after editing.
[0066] The user's evaluation results can be used to generate a sentiment evaluation model.
[0067] An unknown sound or image may be given to the affective evaluation model to perform affective evaluation on the unknown sound or image.
[0068] It is possible to perform an affective evaluation of unknown sounds or images.
[0069] Another aspect of the present invention provides a sound or image evaluation device that includes an icon display unit that displays three icons within a plane displayed on an image display device, a playback unit that outputs sounds or images corresponding to each icon, a position editing unit that edits the position of each icon within the plane in response to a first operation in which a user uses an input device to move each icon within the plane, and a coordinate registration unit that associates the edited planar coordinates of each icon with sounds or images.
[0070] The relationship between sounds or images can be expressed as coordinates on a two-dimensional plane. Sounds or images can be evaluated without the need for evaluation words and evaluation scales. Sounds or images can be evaluated by using the distance on a plane that mathematically produces the least distortion.
[0071] Another aspect of the present invention is a program that causes a computer to execute a method for evaluating sound or images, which method displays three icons in a plane displayed on an image display device, outputs a sound or image corresponding to each icon, edits the position of each icon in the plane in response to a first operation in which a user uses an input device to move each icon in the plane, and associates the edited planar coordinates of each icon with a sound or image.
[0072] The relationship between sounds or images can be expressed as coordinates on a two-dimensional plane. Sounds or images can be evaluated without the need for evaluation words and evaluation scales. Sounds or images can be evaluated by using the distance on a plane that mathematically produces the least distortion.
[0073] 10...control unit, 11...icon display unit, 12...position editing unit, 13...coordinate registration unit, 14...playback unit, 15...sensory space generation unit, 23...storage device, 24...input device, 25...display, BJ...sensory object, BJm...icon, BJn...identification number, CD...coordinate data, SD...audio data
Claims
1. A method for evaluating a sound or image, comprising: displaying three icons within a plane displayed on an image display device; outputting a sound or image corresponding to each icon; editing the position of each icon within the plane in response to a first operation in which a user uses an input device to move each icon within the plane; and associating the edited planar coordinates of each icon with the sound or image.
2. The method for evaluating sounds or images described in claim 1, wherein in the first operation, the user moves each icon within the plane so that the distance between two icons for which the user feels the sound or image is close is shorter than the distance between two icons for which the user feels the sound or image is far away.
3. A method for evaluating a sound or image as described in claim 1 or claim 2, wherein a sound associated with the icon is reproduced in response to a second operation by the user of selecting the icon displayed on the image display device.
4. A method for evaluating a sound or image as described in claim 1, wherein three objects are selected from a group of objects corresponding to four or more sounds or images, and three icons corresponding to the three selected objects are displayed on the image display device, and after the editing is performed, three new objects each having at least one different object are selected from the group of objects, thereby repeating editing of the group of objects multiple times, and after each editing is performed, the planar coordinates of the three selected icons are associated with the sound or image.
5. A method for evaluating sounds or images as described in claim 4, further comprising arranging four or more objects corresponding to the four or more sounds or images within a common distance space based on multiple combinations of the planar coordinates of the three icons obtained by the multiple edits.
6. The method for evaluating a sound or image according to claim 5, wherein the four or more objects arranged in the distance space are mapped into a multi-dimensional space.
7. The method for evaluating a sound or an image according to claim 5 or 6, wherein the distance space or the multidimensional space constitutes an affective space in which an affective evaluation of the sound or the image is reflected.
8. The method for evaluating sounds or images according to claim 1 or 2, wherein the words expressed by the sounds or images are associated with the icons.
9. A method for evaluating a sound or image as described in claim 1, which generates a trained model for affective evaluation by machine learning the correspondence between the sound or image and the edited planar coordinates of each icon.
10. The method for evaluating sounds or images according to claim 9, further comprising providing an unknown sound or image to the trained model, thereby performing a sensory evaluation of the unknown sound or image.
11. A sound or image evaluation device comprising: an icon display unit that displays three icons within a plane displayed on an image display device; a playback unit that outputs a sound or image corresponding to each icon; a position editing unit that edits the position of each icon within the plane in response to a first operation in which a user moves each icon within the plane using an input device; and a coordinate registration unit that associates the edited planar coordinates of each icon with the sound or image.
12. A program for causing a computer to execute a method for evaluating a sound or an image, the method for evaluating a sound or an image comprising the steps of: displaying three icons within a plane displayed on an image display device; outputting a sound or an image corresponding to each icon; editing the position of each icon within the plane in response to a first operation in which a user uses an input device to move each icon within the plane; and associating the edited planar coordinates of each icon with the sound or image.
Citation Information
Patent Citations
Sensibility evaluation device, sensibility evaluation method, and program
JP2016191979A
Voice reproduction program, voice reproduction method and voice reproduction device
JP2022143296A
