Method for evaluating sound or image, device for evaluating sound or image, and program

The method and device facilitate emotional evaluation of sounds or images by positioning and editing icons on a plane, eliminating ambiguity and individual recognition differences, and enabling similarity-based evaluation.

JP2024168716APending Publication Date: 2024-12-05YAMAHA CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023085616
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional methods of sensibility evaluation, such as emotional evaluation of sounds or images, are inadequate in expressing the evaluator's cognition due to individual differences in impressions and recognition of evaluation words, which are ambiguous and multidimensional.

Method used

A method and device that display three icons on a plane, allowing users to move and edit their positions using an input device, associate edited plane coordinates with sounds or images, and generate emotional spaces without using evaluation words.

Benefits of technology

Enables emotional evaluation of sounds or images without evaluation words, reducing individual recognition differences and allowing for a similarity scale-based evaluation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168716000001_ABST
    Figure 2024168716000001_ABST
Patent Text Reader

Abstract

To express sensory evaluation of a user for sound or image without using words for evaluation.SOLUTION: A method for evaluating sound or image includes the steps of: displaying three icons BJm within a plane (display area 25A) displayed on a display 25; outputting sound data SD corresponding to each icon BJm; editing the position of each icon BJm within the plane (display area 25A) according to the first operation in which a user uses an input device 24 to move each icon BJm within the plane (display area 25A); and associating plane coordinates after editing of each icon BJm with the sound data SD.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for evaluating a sound or an image. [Background technology]

[0002] Affective evaluation is often performed on information such as sound. In the affective evaluation, evaluation words such as warm and comfortable are used. For example, the following Patent Document 1 presents a method for subjective evaluation when a stimulus is given. When a stimulus is given, a subject selects the degree to which the affective word (refreshing, relaxing, etc.) applies to the stimulus. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-154800 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since there are individual differences in the impressions and perceptions of evaluation words, evaluation words contain ambiguity. Also, a person who perceives information consciously or unconsciously perceives information not in terms of a single evaluation word, but in a multidimensional space composed of many evaluation axes. Therefore, it cannot be said that the conventional methods of sensory evaluation are necessarily able to adequately express the perceptions of the evaluator.

[0005] An object of the present invention is to express a user's emotional evaluation of a sound or image without using evaluation words. [Means for solving the problem]

[0006] A method for evaluating a sound or image according to one aspect of the present invention displays three icons in a plane displayed on an image display device, outputs a sound or image corresponding to each icon, edits the position of each icon in the plane in response to a first operation in which a user uses an input device to move each icon in the plane, and associates the edited planar coordinates of each icon with a sound or image.

[0007] The present invention is also directed to a sound or image evaluation device and program. Effect of the Invention

[0008] According to the present invention, a user's emotional evaluation of a sound or image can be expressed without using evaluation words. [Brief description of the drawings]

[0009] [Figure 1] 1 is a configuration diagram of a physical information evaluation device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing a data structure of an affective object. [Diagram 3] 1 is a functional block diagram of a physical information evaluation device according to an embodiment. [Figure 4] FIG. 2 is a diagram showing a data structure of coordinate data. [Diagram 5] FIG. 13 is a diagram showing an operation interface for affective evaluation displayed on a display. [Figure 6] FIG. 11 is a diagram showing the movement of an icon displayed on an operation interface. [Figure 7] FIG. 13 is a diagram showing icons arranged based on a user's affective evaluation. [Figure 8] FIG. 13 is a diagram showing icons arranged based on a user's affective evaluation. [Figure 9] FIG. 13 is a diagram showing icons arranged based on a user's affective evaluation. [Figure 10] FIG. 13 is a diagram showing icons arranged based on a user's affective evaluation. [Figure 11]4 is a flowchart showing a method for evaluating physical information according to an embodiment; [Figure 12] 4 is a flowchart showing a sentiment evaluation process according to the embodiment. [Figure 13] 4 is a flowchart showing a sentiment evaluation process according to the embodiment. [Figure 14] 10 is a flowchart showing a sensory space generation process according to the embodiment. [Figure 15] FIG. 13 is a diagram showing a state in which a plurality of sensory objects are arranged in a common metric space. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A method, an apparatus, and a program for evaluating a sound or image according to an embodiment of the present invention will be described below with reference to the accompanying drawings. In the following description, information including sound and images will be described as physical information.

[0011] {1. Configuration of the physical information evaluation device} Fig. 1 is a block diagram showing the configuration of a physical information evaluation device 1 according to this embodiment. The physical information evaluation device 1 is configured by, for example, a personal computer. As shown in Fig. 1, the physical information evaluation device 1 includes a CPU (Central Processing Unit) 20, a ROM (Read Only Memory) 21, a RAM (Random Access Memory) 22, a storage device 23, an input device 24, a display 25, a sound source unit 26, a sound system 27, a communication interface (I / F) 28, and a device interface (I / F) 29. These devices are connected via a system bus 30.

[0012] The CPU 20 controls the overall operation of the physical information evaluation device 1. The ROM 21 stores the control program executed by the CPU 20 and various data. The RAM 22 is used as a work area for the CPU 20. The storage device 23 stores the program PR, coordinate data CD, the sensory object BJ, and sound data SD. The storage device 23 can be configured with various storage media such as a hard disk or semiconductor memory. The program PR executes a process of displaying an icon BJm (an image representing the sensory object BJ) on the display 25, a process of editing the display position of the icon BJm, a process of controlling the playback of the sound data SD associated with the icon BJm, and a process of generating a sensory space.

[0013] The input device 24 is an interface that accepts various operations by the user, such as an operation to move the icon BJm and an instruction to play the sound data SD. The display 25 is used for displaying the icon BJm, etc. The sound source unit 26 and the sound system 27 play the sound data SD associated with the icon BJm (that is, associated with the emotional object BJ) in response to an instruction to play the sound data SD by the user. The communication interface 28 is an interface that performs wired or wireless communication with other computers. The device interface 29 is an interface that accesses a storage medium 31 such as a CD, a DVD, or a semiconductor memory.

[0014] A case where the program PR is stored in the storage device 23 will be described as an example. As another embodiment, the program PR may be provided by being stored in the storage medium 31. The CPU 20 may access the storage medium 31 via the device interface 29, and store the program PR stored in the storage medium 31 in the storage device 23 or the ROM 21. Alternatively, the CPU 20 may access the storage medium 31 via the device interface 29, and execute the program PR stored in the storage medium 31. Alternatively, the CPU 20 may download the program PR from a server on the network via the communication interface 28, and store the downloaded program PR in the storage device 23 or the ROM 21. Alternatively, the CPU 20 may directly execute the program PR stored in the server on the network via the communication interface 28.

[0015] {2. Data structure of the sensory object BJ} FIG. 2 is a diagram showing the data structure of an emotional object BJ. The emotional object BJ includes an identification number BJn and an icon BJm. The identification number BJn is a number (or symbol) given to a plurality of emotional objects BJ, and uniquely identifies each emotional object BJ. The icon BJm is an image for representing the emotional object BJ on the display 25. A plurality of emotional objects BJ are stored in the storage device 23. Each emotional object BJ is associated with audio data SD in a one-to-one relationship. For example, the identification number BJn is recorded in the header information of the audio data SD, and the emotional object BJ is associated with the audio data SD via the identification number BJn. A plurality of audio data SD corresponding to a plurality of emotional objects BJ are stored in the storage device 23.

[0016] In addition, when a plurality of sensory objects BJ use the same icon BJm, the sensory objects BJ do not need to hold the icon BJm. In that case, it is sufficient to use a common icon BJm stored in the storage device 23. In addition, the sensory objects BJ may hold data defining the icon BJm, instead of holding the icon BJm as image data.

[0017] {3. Functional configuration of the physical information evaluation device} Fig. 3 is a block diagram showing a functional configuration of the physical information evaluation device 1. In Fig. 3, the control unit 10 is a functional unit that is realized by the CPU 20 executing the program PR while using the RAM 22 as a work area. The control unit 10 includes an icon display unit 11, a position editing unit 12, a coordinate registration unit 13, a reproduction unit 14, and an emotional space generation unit 15. The icon display unit 11, the position editing unit 12, the coordinate registration unit 13, the reproduction unit 14, and the emotional space generation unit 15 are functional units that are realized by the execution of the program PR. In other words, each of the functional units 11 to 15 is a functional unit that the CPU 20 includes.

[0018] The icon display unit 11 displays the icons BJm on the display 25 in a movable state. The icon display unit 11 displays the three selected icons BJm in a display area 25A (see FIG. 5, etc.) on the display 25. That is, the icon display unit 11 selects three sensory objects BJ from a plurality of sensory objects BJ stored in the storage device 23 according to a predetermined rule. Then, the icon display unit 11 displays the icons BJm corresponding to the three selected sensory objects BJ on the display 25. In an initial state, the icon display unit 11 can display the icons BJm at any position within the display area 25A. For example, in an initial state, the icon display unit 11 may display the three icons BJm at equal intervals.

[0019] The rule by which the icon display unit 11 selects three sensory objects BJ from among the multiple sensory objects BJ is not particularly limited. For example, the icon display unit 11 may select three sensory objects BJ randomly. Alternatively, the icon display unit 11 may select the sensory objects BJ in order based on the identification number BJn.

[0020] The position editing unit 12 edits the position of the icon BJm in the display area 25A of the display 25 in response to an operation received by the input device 24. The user performs an operation to move the position of the icon BJm displayed on the display 25 using the input device 24, such as a mouse or a keyboard. If the display 25 is a touch panel display, the display 25 can also have a function as the input device 24. In this case, the user can edit the position of the icon BJm by touching the display as the input device 24 with a finger or a pen.

[0021] The coordinate registration unit 13 registers coordinate data CD based on the position of the icon BJm edited by the position editing unit 12. The user performs an editing operation on the three icons BJm displayed on the display 25. In response to this operation, the coordinate registration unit 13 registers the coordinates of each icon BJm after it has been moved in the coordinate data CD. The coordinate registration unit 13 registers the planar coordinates within the display area 25A of the three icons BJm displayed on the display 25 in the coordinate data CD. In the coordinate data CD, an identification number BJn corresponding to each icon BJm is registered in association with the coordinates.

[0022] Fig. 4 is a diagram showing the data structure of the coordinate data CD. Fig. 4 shows one data set included in the coordinate data CD. One data set includes the identification numbers BJn and coordinates of each of the three icons BJm. The coordinate data CD is data in which a plurality of such data sets are recorded. Each of the plurality of data sets is composed of a combination of three different icons BJm. In other words, in the set of three icons BJm included in one data set, at least one icon BJm is different from the other data sets.

[0023] The playback unit 14 plays back the audio data SD associated with the sensory object BJ in response to an operation received by the input device 24. The user issues an instruction to play back audio for any of the icons BJm displayed on the display 25, for example, by operating a mouse, a keyboard, or the like. In response to this operation, the playback unit 14 issues an instruction to the sound source unit 26 to play back the audio data SD. The audio signal corresponding to the audio data SD output from the sound source unit 26 is output by the sound system 27.

[0024] The user can check the sound associated with the icon BJm (sensory object BJ) by instructing the icon BJm displayed on the display 25 to play the sound. The user listens to the sound associated with each of the three icons BJm displayed on the display 25, and evaluates the three sounds from the viewpoint of similarity, i.e., whether they are similar or not. The user moves the two icons BJm that the user feels are similar on the display 25 so that they come closer to each other, and moves the two icons BJm that the user feels are not similar on the display 25 so that they come farther apart. In other words, the user moves each icon BJm on the display 25 so that the distance between the two icons that the user feels have sounds that are close to each other is shorter than the distance between the two icons that the user feels have sounds that are distant from each other. Since the three icons BJm are displayed on the display 25, the user expresses the mutual similarity between the three sensory objects BJ within the display area 25A on the display 25, which is a two-dimensional plane. The user may enlarge or reduce the display on the display 25 as necessary.

[0025] The sensory space generating unit 15 performs matching processing and mapping processing. The matching processing is processing for unifying the scale of distance between all sensory objects BJ (a space with a unified scale of distance is referred to as a distance space SP). As described above, the position editing unit 12 edits the coordinates of the icons BJm displayed in units of three. Then, the coordinate registration unit 13 registers the coordinates of the icons BJm after editing. The sensory space generating unit 15 uses the coordinate data CD registered by the coordinate registration unit 13 to calculate the distance between the sensory objects BJ such that all the sensory objects BJ have a consistent scale. The mapping processing maps the sensory objects BJ into a multidimensional space based on this distance relationship.

[0026] {4. Icon-based operation interface for sensory evaluation} Next, an operation interface for affective evaluation using the icon BJm will be described with reference to Figs. 5 to 10. Fig. 5 shows a display area 25A of the icon BJm displayed on the display 25 by the program PR. The display area 25A is an area for displaying the icon BJm, and is also used as an operation interface for the user to perform affective evaluation. Three icons BJm(1), BJm(2), and BJm(3) are displayed in the display area 25A by the icon display unit 11. The icons BJm(1) to (3) correspond to the three affective objects BJ(1) to (3). In the example shown in Fig. 5, the icon display unit 11 displays the three icons BJm(1) to (3) at equal intervals as an initial state.

[0027] The user can listen to the sound data SD corresponding to each icon BJm(1)-(3) by clicking the icons BJm(1)-(3) displayed in the display area 25A, for example, with a mouse. The user then operates to move the icons BJm with similar sounds closer together, or to move the icons BJm with dissimilar sounds farther apart. FIG. 6 is a diagram showing a state in which the position of the icon BJm(3) has been edited by the user. The user repeats the operation of moving the icons BJm(1)-(3) and the operation of listening to the sounds associated with the icons BJm(1)-(3) as necessary, to reflect the affective evaluation of the affective objects BJ(1)-(3) on the display area 25A.

[0028] 7 to 10 are diagrams showing the arrangement of icons BJm(1) to (3) that have been subjectively evaluated by a user. As shown in FIG. 7, in the display area 25A, icon BJm(1) and icon BJm(2) are arranged close to each other. Icon BJm(3) is arranged at a distance from icons BJm(1) and BJm(2). In this example, the user evaluates that the sounds associated with icons BJm(1) and BJm(2) are similar, and that the sound associated with icon BJm(3) is not similar to the sounds associated with icons BJm(1) and (2). Since the distance between icon BJm(1) and icon BJm(3) and the distance between icon BJm(2) and icon BJm(3) are about the same, the user evaluates that icon BJm(3) is not similar to icons BJm(1) and (2).

[0029] In the example shown in Fig. 8, the user evaluates that the voice associated with icon BJm(2) is similar to the voice associated with icon BJm(1) and icon BJm(3). On the other hand, the user evaluates that the voice associated with icon BJm(3) is not similar to the voice associated with icon BJm(1). Since the distance between icon BJm(1) and icon BJm(3) is much greater than the distance between icon BJm(2) and icons BJm(1) and (3), the user evaluates that the similarity between the voices associated with icon BJm(1) and icon BJm(3) is very low.

[0030] In the example shown in FIG. 9, the user evaluates that the sounds associated with icons BJm(1), BJm(2), and BJm(3) are all equally similar. And, because the distance between icons BJm(1), (2), and (3) is relatively small, the user evaluates that the three sounds are relatively similar. In the example shown in FIG. 10, the user evaluates that the sounds associated with icons BJm(1), BJm(2), and BJm(3) are all not very similar. Among them, the user evaluates that the sounds associated with icons BJm(2) and BJm(3) are less similar than the other relationships.

[0031] In the present embodiment, the reason why three icons BJm are displayed in the display area 25A is that the relationship between the corresponding voice data SD can be uniquely expressed. For example, the plane on which two icons BJm are arranged is not uniquely specified. Also, for four or more icons BJm, the relationship may not be expressed on a plane.

[0032] {5. Sensory evaluation method for physical information} Next, the sensory evaluation method for physical information according to this embodiment will be described with reference to FIG. 11. The flowchart shown in FIG. 11 shows the overall processing flow of the sensory evaluation method. The processing shown in FIG. 11 is realized by executing the program PR on the CPU 20. First, in step S1, the control unit 10 performs sensory evaluation processing. Details of the sensory evaluation processing will be described with reference to the flowcharts of FIG. 12 and FIG. 13. Next, in step S2, the control unit 10 performs sensory space generation processing. Details of the sensory space generation processing will be described with reference to the flowchart of FIG. 14.

[0033] (5-1) Sensitivity evaluation processing The affective evaluation process (step S1 in FIG. 11) according to this embodiment will be described with reference to FIG. 12 and FIG. 13. The flowcharts shown in FIG. 12 and FIG. 13 are processes realized by executing the program PR on the CPU 20. First, in step S11, the icon display unit 11 displays three icons BJm on the display 25. The icon display unit 11 displays the three icons BJm in a movable state within the display area 25A as shown in FIG. 5.

[0034] Next, in step S12, the playback unit 14 determines whether or not an instruction to output a sound corresponding to any of the icons BJm has been received. As described above, the user can operate the input device 24 to select any of the icons BJm displayed on the display 25 and issue an instruction to play the sound. If an instruction to output a sound for any of the icons BJm has been received, in step S13 the playback unit 14 plays the sound corresponding to the selected icon BJm. After step S13, the process returns to step S12.

[0035] If a voice output instruction for the icon BJm is not received in step S12, the position editing unit 12 determines in step S14 whether or not an operation to move any of the icons BJm within the display area 25A has been received. If a move operation for any of the icons BJm has been received, in step S15 the position editing unit 12 moves the selected icon BJm on the display area 25A. After step S15, the process returns to step S12.

[0036] If the movement operation for the icon BJm is not accepted in step S14, the coordinate registration unit 13 judges in step S16 whether or not an instruction to end editing has been accepted. When the user finishes the editing operation for the three icons BJm displayed on the display 25, the user operates the input device 24 to issue an instruction to end editing. If an instruction to end editing has been accepted, in step S17, the coordinate registration unit 13 registers the coordinates of the three icons BJm arranged in the display area 25A in the coordinate data CD. Specifically, the coordinate registration unit 13 registers the coordinates of the three icons BJm arranged in the display area 25A in the coordinate data CD in association with the identification number BJn. After step S17, the process proceeds to step S18 in FIG. 13.

[0037] In step S18, the icon display unit 11 judges whether or not an instruction to end editing for all the icons BJm has been received. When the editing operation for all the icons BJm corresponding to all the emotional objects BJ to be evaluated is completed, the user operates the input device 24 to instruct the end of all editing. If an instruction to end editing for all the icons BJm has been received, the program PR ends the emotional evaluation process. If an instruction to end editing for all the icons BJm has not been received, that is, if an instruction to end editing for new three icons BJm (new three emotional objects BJ) has been received, in step S19, the icon display unit 11 selects a set of three other icons BJm (a set of three other emotional objects BJ). Here, the icon display unit 11 excludes from the selection a set that is the same as the set of icons BJm selected in the past. The icon display unit 11 selects a set of three new icons BJm (a set of three new emotional objects BJ) in which at least one icon BJm is different from the three icons BJm. After step S19, the process returns to step S11 in Fig. 12, and the icon display unit 11 displays the newly selected three icons BJm in the display area 25A. Then, steps S12 to S19 are repeatedly executed for the new icons BJm. In this way, the user executes the editing operation for all the icons BJm corresponding to all the affective objects BJ to be evaluated.

[0038] The affective evaluation of each affective object BJ needs to be performed at least once. In other words, for each affective object BJ, an editing operation of the icon BJm needs to be performed at least once. In order to properly perform the affective evaluation of the affective objects BJ, it is desirable that an editing operation of the icon BJm is performed multiple times for one affective object BJ. In order to properly evaluate the mutual relationship of the similarities of all the affective objects BJ, it is desirable that an editing operation is performed for each icon BJm in a set of as many icons BJm as possible.

[0039] In this way, according to the physical information evaluation device 1 of this embodiment, the relationship of the physical information can be expressed by coordinates on a two-dimensional plane. By performing a perceptual evaluation of the physical information without using evaluation words and evaluation scales, it is possible to eliminate individual differences in recognition of the evaluation words and evaluate the physical information using a similarity scale. By using the physical information evaluation device 1 of this embodiment, the user can enjoy the customer experience of performing a perceptual evaluation of the physical information without using evaluation words and evaluation scales.

[0040] In addition, by using the physical information evaluation device 1 of this embodiment, it is also possible to perform a sensibility evaluation of an unknown sound or image. For example, a sensibility evaluation model can be generated by performing deep learning using the coordinate data CD registered by the processing of the above embodiment and the voice data SD associated with the identification number BJn. That is, a trained model for sensibility evaluation can be generated by machine learning the association between the sound or image and the planar coordinates after editing of each icon. By providing an unknown sound or image to this trained model for sensibility evaluation, it is possible to perform a sensibility evaluation of the unknown sound or image. By providing an unknown sound or image to the physical information evaluation device 1 of this embodiment, the user can enjoy the customer experience of obtaining a sensibility evaluation result. Alternatively, it is also possible to generate a sound or image that is estimated to be sensibly close to a certain sound by creating a model of a generative system.

[0041] (5-2) Sensitivity space generation processing Next, the sensory space generation process (step S2 in FIG. 11) according to the present embodiment will be described with reference to FIG. 14. The flowchart shown in FIG. 14 is a process realized by executing the program PR on the CPU 20. First, in step S21, the sensory space generation unit 15 performs a matching process for all sensory objects BJ. The matching process is a process for unifying the scale of the distance between all sensory objects BJ based on the coordinate data CD stored in the storage device 23. That is, the sensory space generation unit 15 calculates the distance in a common distance space between all sensory objects BJ based on multiple combinations of the planar coordinates of the three icons BJm obtained by multiple edits. The matching process may be considered as a process for arranging all icons BJm in one distance space SP. However, since the process for displaying the icons BJm on the display 25 based on the coordinates after the matching process is not essential, the matching process will be described here assuming that the target of the matching process is the sensory object BJ.

[0042] FIG. 15 is a diagram showing the concept of matching processing to the metric space SP. The diagram shows a state in which four sensory objects BJ(1)-(4) corresponding to four icons BJm(1)-(4) are arranged in the metric space SP based on two data sets included in the coordinate data CD. In other words, the diagram shows a state in which the scale of the distance between the four sensory objects BJ(1)-(4) is unified. The coordinate data CD records a data set including icons BJm(1),(2),(3) and a data set including icons BJm(1),(2),(4). The sensory space generation unit 15 obtains the distance in the metric space SP between the sensory objects BJ(1)-(4) based on the two data sets that share the icons BJm(1) and (2), as shown in the diagram.

[0043] The method of the matching process by the sensory space generating unit 15 is not particularly limited. For example, the sensory space generating unit 15 calculates a distance that minimizes the error of the distance between each icon BJm based on a plurality of data sets recorded in the coordinate data CD. For example, for N icons BJm(1) to (N) to be evaluated, the difference between the estimated distance (distance in the distance space SP) and the distance between each icon based on the coordinates recorded in the coordinate data CD is calculated. Then, the least square method is used to calculate the distance between each sensory object in the distance space SP so that the integrated value of the calculated differences is minimized. In the example shown in FIG. 15, four sensory objects BJ are matched in the distance space SP based on two data sets, but the number of data sets increases according to the number of sensory objects BJ. For example, when the number of sensory objects BJ is N, the maximum number of data sets is the number of combinations in which three sensory objects BJ are selected from the N sensory objects BJ.

[0044] Next, in step S22, the sensory space generation unit 15 maps all the sensory objects Bj, for which the distance scale has been unified, into a multidimensional space. In step S21, the N sensory objects BJ(1)-(N) to be evaluated, for which the distances in the common metric space SP have been calculated, are further mapped into a multidimensional space.

[0045] The method of mapping to the multidimensional space by the sensory space generation unit 15 is not particularly limited. Mapping to the multidimensional space can be performed using a known technique. For example, the sensory space generation unit 15 performs mapping to the multidimensional space by using principal coordinate analysis.

[0046] {6. Other embodiments} In the above embodiment, the sensory object BJ (icon BJm) is associated with sound data SD. That is, the sensory object BJ (icon BJm) is associated with sound as physical information. The physical information evaluated by the physical information evaluation device 1 of the present embodiment is not limited to sound. The physical information evaluation device 1 of the present embodiment can evaluate an image as physical information in association with the icon BJm.

[0047] For example, the physical information evaluation device 1 can evaluate words (text information) presented to the user through physical information such as sound or image. Words are associated with each affective object BJ. For example, the user can select an icon BJm using the input device 24 to hear and hear the words associated with the icon BJm by sound. Alternatively, the user can select an icon BJm using the input device 24 to confirm the words associated with the icon BJm as an image (visual information). The icon BJm itself may include the words as images. Other processing is the same as in the above embodiment. The user can evaluate the similarity of the words by operating the icon BJm based on the meaning and nuance expressed by the words, and project the affective evaluation onto the arrangement of the icons BJm on a plane.

[0048] {7. Correspondence between each element of the claims and each element of the embodiment} Below, an example of the correspondence between each component of the claims and each element of the embodiment will be described, but the present invention is not limited to the following example. In the above embodiment, the display 25 is an example of an image display device, and the display area 25A is an example of a plane displayed on the image display device. Also, in the above embodiment, an operation of moving the icon BJm within the display area 25A using the input device 24 is an example of a first operation, and an operation of selecting the icon BJm using the input device 24 and instructing playback of the audio data SD is an example of a second operation.

[0049] As each component of a claim, various elements having the configuration or function described in the claim may be used.

[0050] {8. Features and Effects of the Embodiment} A sound or image evaluation method according to one embodiment of the present invention displays three icons in a plane displayed on an image display device, outputs a sound or image corresponding to each icon, edits the position of each icon in the plane in response to a first operation in which a user uses an input device to move each icon in the plane, and associates the edited planar coordinates of each icon with a sound or image.

[0051] The relationship between sounds or images can be expressed by coordinates on a two-dimensional plane. Sounds or images can be evaluated without the need for evaluation words and evaluation scales. Sounds or images can be evaluated by using the distance on a plane that mathematically produces the least distortion.

[0052] In the first operation, the user may move each icon in a plane so that the distance between two icons whose sounds or images are perceived to be close is shorter than the distance between two icons whose sounds or images are perceived to be far apart.

[0053] It is possible to perform a sensibility evaluation of sounds or images by judging whether they are similar or not, without using evaluation words.

[0054] In response to a second operation by the user to select an icon displayed on the image display device, a sound associated with the icon may be played.

[0055] The user can make an emotional evaluation while listening to the sound associated with each icon.

[0056] Three objects are selected from a group of objects corresponding to four or more sounds or images, and three icons corresponding to the three selected objects are displayed on an image display device; after editing, three new objects having at least one different object are selected from the group of objects, thereby repeating editing on the group of objects multiple times; and after each edit, the planar coordinates of the three selected icons may be associated with a sound or image.

[0057] By performing an affective evaluation for each of the three objects selected in different combinations, it is possible to perform an affective evaluation based on the relative relationships of four or more sounds or images.

[0058] Four or more objects corresponding to four or more sounds or images may be arranged in a common distance space based on multiple combinations of the planar coordinates of the three icons obtained by multiple edits.

[0059] The affective evaluations made on different object combinations can be expressed in a common metric space.

[0060] Four or more objects positioned within a common metric space may be mapped into a multi-dimensional space.

[0061] Four or more sounds or images can be represented in a multi-dimensional space based on the sensory evaluations arranged in a distance space.

[0062] A distance space or a multi-dimensional space may be used to construct an affective space that reflects an affective evaluation of a sound or an image.

[0063] Words expressed by sounds or images may be associated with the icons.

[0064] A user's emotional evaluation of words can be expressed in coordinates on a two-dimensional plane through audio and images.

[0065] An affective evaluation model may be generated by machine learning the association between sounds or images and the edited planar coordinates of each icon.

[0066] The user's evaluation results can be used to generate a sentiment evaluation model.

[0067] An unknown sound or image may be given to the affective evaluation model to perform affective evaluation of the unknown sound or image.

[0068] It is possible to carry out a sensory evaluation of unknown sounds or images.

[0069] A sound or image evaluation device according to another aspect of the present invention includes an icon display unit that displays three icons within a plane displayed on an image display device, a playback unit that outputs a sound or image corresponding to each icon, a position editing unit that edits the position of each icon within the plane in response to a first operation in which a user moves each icon within the plane using an input device, and a coordinate registration unit that associates the edited planar coordinates of each icon with a sound or image.

[0070] The relationship between sounds or images can be expressed by coordinates on a two-dimensional plane. Sounds or images can be evaluated without the need for evaluation words and evaluation scales. Sounds or images can be evaluated by using the distance on a plane that mathematically produces the least distortion.

[0071] A program according to another aspect of the present invention is a program that causes a computer to execute a method for evaluating sound or images, the method for evaluating sound or images including displaying three icons in a plane displayed on an image display device, outputting a sound or image corresponding to each icon, editing the position of each icon in the plane in response to a first operation in which a user uses an input device to move each icon in the plane, and associating the edited planar coordinates of each icon with a sound or image.

[0072] The relationship between sounds or images can be expressed by coordinates on a two-dimensional plane. Sounds or images can be evaluated without the need for evaluation words and evaluation scales. Sounds or images can be evaluated by using the distance on a plane that mathematically produces the least distortion. [Explanation of symbols]

[0073] 10...control unit, 11...icon display unit, 12...position editing unit, 13...coordinate registration unit, 14...playback unit, 15...sensory space generation unit, 23...storage device, 24...input device, 25...display, BJ...sensory object, BJm...icon, BJn...identification number, CD...coordinate data, SD...audio data

Claims

1. Displaying the three icons on a plane displayed on an image display device; Outputs a sound or image corresponding to each icon, editing positions of each icon within the plane in response to a first operation by a user using an input device to move each icon within the plane; A method for evaluating a sound or an image, comprising: associating the edited planar coordinates of each icon with the sound or the image.

2. 2. The method for evaluating sounds or images according to claim 1, wherein in the first operation, the user moves each icon within the plane so that the distance between two icons for which the user feels that the sound or image is close is shorter than the distance between two icons for which the user feels that the sound or image is far away.

3. 3. The method for evaluating a sound or an image according to claim 1, further comprising the step of: reproducing a sound associated with the icon in response to a second operation by a user of selecting the icon displayed on the image display device.

4. 2. The method for evaluating a sound or image as described in claim 1, wherein three objects are selected from a group of objects corresponding to four or more sounds or images, and three icons corresponding to the three selected objects are displayed on the image display device, and after the editing is performed, three new objects having at least one different object are selected from the group of objects, thereby repeating editing of the group of objects multiple times, and after each editing is performed, the planar coordinates of the three selected icons are associated with the sound or image.

5. The method for evaluating sounds or images described in claim 4, further comprising arranging four or more objects corresponding to the four or more sounds or images within a common distance space based on multiple combinations of the planar coordinates of the three icons obtained by the multiple edits.

6. The method for evaluating a sound or an image according to claim 5 , wherein the four or more objects arranged in the distance space are mapped into a multi-dimensional space.

7. 7. The sound or image evaluation method according to claim 5, wherein the distance space or the multidimensional space constitutes an affective space in which an affective evaluation of the sound or image is reflected.

8. 3. The method for evaluating a sound or an image according to claim 1, wherein the words expressed by the sounds or the images are associated with the icons.

9. The method for evaluating a sound or an image according to claim 1 , further comprising the step of: generating a trained model for affective evaluation by machine learning the correspondence between the sound or the image and the edited planar coordinates of each icon.

10. The method for evaluating a sound or image according to claim 9 , further comprising providing an unknown sound or image to the trained model, thereby performing a sensory evaluation of the unknown sound or image.

11. an icon display unit that displays three icons on a plane displayed on an image display device; a playback unit that outputs a sound or an image corresponding to each icon; a position editing unit that edits positions of each icon within the plane in response to a first operation by a user to move each icon within the plane using an input device; a coordinate registration unit for associating the edited planar coordinates of each icon with the sound or image; A sound or image evaluation device comprising:

12. A program for causing a computer to execute a sound or image evaluation method, The method for evaluating a sound or an image comprises the steps of: Displaying the three icons on a plane displayed on an image display device; Outputs a sound or image corresponding to each icon, editing positions of each icon within the plane in response to a first operation by a user using an input device to move each icon within the plane; A program for associating the edited planar coordinates of each icon with the sound or image.

Citation Information

Patent Citations

  • Evaluation method, and evaluation system

    JP2022154800A