Image generating device and image generating method

Abstract

To allow a viewer to properly recognize an association between an object and text data.SOLUTION: An image generation device (10) includes: an acquisition unit (11) that acquires image data indicating an image and audio data indicating audio accompanying the image; an identification processing execution unit (14) that executes identification processing for identifying an object that is generating the audio, from among one or more objects included in the image by inputting the image data and the audio data acquired by the acquisition unit into a learned model; and a superimposing unit (15) that superimposes text data corresponding to the audio data on a position in the image according to the result of the identification processing.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image generating device and an image generating method. [Background technology]

[0002] Conventionally, a device for creating subtitles from video data is known. In this device, a face detection means detects face features and face positions from video data, and a voice identification means detects voice features from video data. Each detected feature is sent to a speaker identification means and compared with features of the speaker registered in a voice-face correspondence data storage means, and as a result, the position of the speaker is identified. The voice of the identified speaker is converted into text by a voice recognition means. A speech bubble is created by a speech bubble creation means based on the speaker's position and text data, and a video creation means combines the video data, voice data, and speech bubble data to create new video data (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-027990 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques have the following problems. If the face feature amount or voice feature amount detected from the video data is not registered in the voice-face correspondence data storage means, there is a possibility that the speaker and the voice will be linked incorrectly. As a result, the voice of the wrong speaker will be displayed in the speech bubble, and the viewer may mistakenly recognize the correspondence between the speaker and the voice.

[0005] An aspect of the present invention is to enable a viewer to appropriately recognize the association between an object and text data. [Means for solving the problem]

[0006] In order to solve the above problems, an image generating device according to one embodiment of the present invention includes an acquisition unit that acquires image data representing an image and audio data representing audio accompanying the image, a specific processing execution unit that executes a specific processing to identify an object that is generating audio among one or more objects contained in the image by inputting the image data and audio data acquired by the acquisition unit into a learned model, and a superimposition unit that superimposes text data corresponding to the audio data at a position within the image that corresponds to a result of the identification processing.

[0007] In order to solve the above problems, an image generating device according to another aspect of the present invention includes an acquisition unit that acquires image data representing an image and audio data representing audio accompanying the image, an identification process execution unit that executes an identification process to identify an object that is generating audio among one or more objects contained in the image by inputting the image data and audio data acquired by the acquisition unit into a learned model, and a superposition unit that generates an image on which information identifying the object is superimposed, the information having a display form corresponding to accuracy information indicating the accuracy of the identification process.

[0008] In order to solve the above problems, an image generation method according to one embodiment of the present invention includes an acquisition step of acquiring image data representing an image and audio data representing audio accompanying the image, a specific processing execution step of executing a specific processing to identify an object that is generating audio among one or more objects contained in the image by inputting the image data and audio data acquired in the acquisition step into a trained model, and a superimposition step of superimposing text data corresponding to the audio data at a position within the image that corresponds to a result of the identification processing.

[0009] In order to solve the above problems, an image generating method according to another aspect of the present invention includes an acquisition step of acquiring image data representing an image and audio data representing audio accompanying the image, a specific processing execution step of executing a specific processing to identify an object that is generating the audio among one or more objects contained in the image by inputting the image data and audio data acquired in the acquisition step into a learned model, and a superimposition step of generating an image on which information identifying the object is superimposed, the information having a display form corresponding to accuracy information indicating the accuracy of the identification process. Effect of the Invention

[0010] According to one aspect of the present invention, a viewer can appropriately recognize the association between an object and text data. [Brief description of the drawings]

[0011] [Figure 1] 1 is a perspective view of a display device according to a first embodiment of the present invention. [Diagram 2] 2 is a block diagram showing a configuration of a main part of an image generating device included in the display device shown in FIG. [Diagram 3] 2 is a diagram showing an example of a display displayed on a display unit of the display device shown in FIG. 1. FIG. [Figure 4] 1. FIG. 4 is a diagram showing another example of display displayed on the display unit of the display device shown in FIG. [Diagram 5] 1. FIG. 4 is a diagram showing another example of display displayed on the display unit of the display device shown in FIG. [Figure 6] 1. FIG. 4 is a diagram showing another example of display displayed on the display unit of the display device shown in FIG. [Figure 7] 4 is a flowchart showing an image generating method executed by an image generating device included in the display device shown in FIG. [Figure 8] FIG. 11 is a block diagram showing a configuration of a main part of an image generating device according to a second embodiment of the present invention. [Figure 9] 9 is a diagram showing an example of one display displayed on the display unit shown in FIG. 8. FIG. [Figure 10] 9A and 9B are diagrams showing other examples of display displayed on the display unit shown in FIG. 8. [Figure 11] 9 is a flowchart showing an image generating method executed by the image generating device shown in FIG. 8. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] [Embodiment 1] Hereinafter, one embodiment of the present invention will be described with reference to FIGS.

[0013] 1 is a front perspective view of a display device 1 according to this embodiment. In this embodiment, the display device 1 is realized as a television receiver. As shown in FIG. 1, the display device 1 includes at least an image generating device 10 and a display unit 40.

[0014] The configuration of the image generating device 10 included in the display device 1 shown in Fig. 1 will be described in detail below with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the main parts of the image generating device 10.

[0015] As shown in FIG. 2, the image generating device 10 includes an acquisition unit 11, a voice / image recognition unit 12, a text data generating unit 13, a subtitle display position determination unit 14, and a superimposition unit 15. The acquisition unit 11 is an example of the "acquisition unit" in the claims. The voice / image recognition unit 12 and the subtitle display position determination unit 14 are an example of the "specific process execution unit" in the claims. The superimposition unit 15 is an example of the "superimposition unit" in the claims. In FIG. 2, the reference numeral 20 denotes a voice recognition DB, the reference numeral 30 denotes an image recognition DB, and the reference numeral 40 denotes a display unit. The voice recognition DB 20 and the image recognition DB 30 may be (1) provided inside the display device 1, or (2) provided outside the display device 1 and configured to be able to communicate with the display device 1.

[0016] The acquisition unit 11 is configured to acquire image data indicating an image and audio data indicating audio accompanying the image. As an example, the acquisition unit 11 separates and extracts image data and audio data from data obtained by demodulating a broadcast wave. As another example, the acquisition unit 11 separates and extracts image data and audio data from data obtained by decoding content data. Note that the image represented by the image data may be a still image or a moving image (video). Then, as shown in FIG. 2, the image data acquired by the acquisition unit 11 is output to each of the image recognition unit included in the audio / image recognition unit 12 and the superimposition unit 15. Also, the audio data acquired by the acquisition unit 11 is output to each of the text data generation unit 13 and the audio recognition unit included in the audio / image recognition unit 12.

[0017] The audio / image recognition unit 12 is configured to (1) identify the object that emitted the audio represented by the audio data acquired by the acquisition unit 11, (2a) identify each object included as a subject in the image represented by the image data acquired by the acquisition unit 11, and (2b) specify the position of each object on the screen. The audio / image recognition unit 12 outputs (1a) identification information of the object that emitted the audio represented by the audio data, (2a) identification information of each object included as a subject in the image represented by the image data, and (2b) position information of each object included as a subject in the image represented by the image data to the subtitle display position determination unit 14.

[0018] The voice / image recognition unit 12 can be composed of a voice recognition unit and an image recognition unit, for example, as shown in FIG.

[0019] The voice recognition unit identifies an object that has emitted a voice represented by the voice data acquired by the acquisition unit 11. To perform this identification (an example of the "second identification process" in the claims), the voice recognition unit uses, for example, a trained model (an example of the "second trained model" in the claims) stored in a voice recognition DB. This trained model is a trained model (for example, a neural network) that receives voice data as input and outputs identification information of an object that has emitted a voice represented by the voice data. Here, the identification information of an object is information for distinguishing the object from other objects, for example, an identifier given to the object. This trained model can be constructed by machine learning using, for example, a combination of voice data and identification information of an object that has emitted a voice represented by the voice data as training data.

[0020] For example, when audio data representing the voice of Prime Minister A is input to this trained model, identification information of Prime Minister A is output from this trained model. Also, when audio data representing the voice of President T is input to this trained model, identification information of President T is output from this trained model. Also, when audio data representing the barking of a dog is input to this trained model, identification information of the dog is output from this trained model. Also, when audio data representing the sound of an ambulance siren is input to this trained model, identification information of the ambulance is output from this trained model.

[0021] The image recognition unit identifies each object included as a subject in the image represented by the image data acquired by the acquisition unit 11, and specifies the position of each object on the screen. To perform this identification and specification (an example of the "first specification process" in the claims), the image recognition unit uses, for example, a trained model (an example of the "first trained model" in the claims) stored in an image recognition DB. This trained model is a trained model (for example, a neural network) that receives image data as input and outputs identification information and position information of each object included as a subject in the image represented by the image data. Here, the position information of an object is information that represents the position of the object on the screen, for example, the coordinates of a representative point of the object. This trained model can be constructed by machine learning using, for example, a combination of image data and identification information and position information of an object included as a subject in the image represented by the image data as training data.

[0022] For example, when image data representing an image including Prime Minister A and President T as subjects is input to this trained model, the trained model outputs identification information and location information of Prime Minister A, and identification information and location information of President T. Also, for example, when image data representing an image including a dog and an ambulance as subjects is input to this trained model, the trained model outputs identification information and location information of the dog, and identification information and location information of the ambulance.

[0023] The text data generating unit 13 is configured to convert the voice represented by the voice data acquired by the acquiring unit 11 into text data. A method of converting the voice represented by the voice data into text data is well known, and therefore a description thereof will be omitted here. The text data generating unit 13 outputs the generated text data to the superimposing unit 15.

[0024] The subtitle display position determination unit 14, based on the identification information and position information generated by the audio / image recognition unit 12, identifies an object that emits the sound represented by the audio data acquired by the acquisition unit 11 among objects included as subjects in the image represented by the image data acquired by the acquisition unit 11. Here, the object identified by the subtitle display position determination unit 14 is an object whose identification information obtained by the image recognition unit matches the identification information obtained by the audio recognition unit. Then, based on the position information of the identified object, the subtitle display position determination unit 14 determines the position at which the text generated by the text data generation unit 13 is displayed as a subtitle. For example, the subtitle display position determination unit 14 sets the position at which the text generated by the text data generation unit 13 is displayed as a subtitle near the identified object. Alternatively, the subtitle display position determination unit 14 displays the position at which the text generated by the text data generation unit 13 is displayed as a subtitle inside a speech bubble image displayed near the identified object.

[0025] The superimposition unit 15 is configured to superimpose the text data generated by the text data generation unit 13, which corresponds to the audio data acquired by the acquisition unit 11, on an image represented by the image data acquired by the acquisition unit 11. The superimposition unit 15 sets the position at which to superimpose the text data generated by the text data generation unit 13, at the position determined by the subtitle display position determination unit 14. The image with subtitles generated by the superimposition unit 15 is displayed on the display unit 40.

[0026] According to the display device 1, it is possible to display the subtitles obtained by converting the voice into text at a location corresponding to the position of the object that emitted the voice. Therefore, even a viewer watching the display device 1, for example, a viewer with a hearing impairment, can properly recognize the association between the object and the subtitles.

[0027] The superimposing unit 15 may determine the position at which to superimpose the text data by referring to accuracy information indicating the accuracy of the identification process. Here, the identification process refers to a process including object identification in the voice recognition unit, object identification in the image recognition unit, and object identification in the subtitle display position determination unit 14.

[0028] For example, when the likelihood indicated by the likelihood information is higher than a predetermined level, the superimposition unit 15 superimposes the text data as subtitles near the object generating the sound. In other cases, the superimposition unit 15 superimposes the text data as subtitles in a predetermined area, such as the bottom edge, top edge, right edge, or left edge of the screen, regardless of the object generating the sound.

[0029] Alternatively, when the likelihood indicated by the likelihood information is higher than a predetermined level, the superimposing unit 15 superimposes a speech bubble image indicating that the sound is generated from the object near the object generating the sound, and superimposes the text data as subtitles inside the speech bubble image. In other cases, the superimposing unit 15 superimposes the text data as subtitles in a predetermined area, such as the bottom edge, top edge, right edge, or left edge of the screen, regardless of the object generating the sound.

[0030] According to the above configuration, the superimposing unit 15 can appropriately determine the position at which to superimpose the text data.

[0031] The "predetermined height" is not particularly limited and may be set as needed.

[0032] (Example of what is displayed on the display) FIG. 3 is a diagram showing one display example displayed on the display unit 40 of the display device 1 shown in FIG. 1, and FIGS. 4 to 6 are diagrams showing other display examples.

[0033] As shown in FIG. 3, when the above-mentioned identification process identifies the object generating the voice as a human being OB1, the superimposition unit 15 can superimpose text data TD1 corresponding to the voice (in this example, "Hello") on a speech bubble image near the OB1 generating the voice.

[0034] This allows a viewer watching the display unit 40, even a viewer with a hearing impairment, to properly recognize the association between the object (OB1 in this example) and the subtitle ("Hello" in this example).

[0035] On the other hand, as shown in Fig. 4, when the object generating the voice is identified as a human OB2 by the above-mentioned identification process, the superimposing unit 15 can superimpose the text data TD2 corresponding to the voice (in this example, "Welcome") on the balloon image near the OB2 generating the voice. This also produces the same effect as the example shown in Fig. 3.

[0036] In addition, if the identification process fails to identify the object generating the sound, the superimposition unit 15 may superimpose, as subtitles, text data corresponding to the audio data indicating the sound generated by the object in a predetermined area unrelated to the object generating the sound, such as the bottom edge of the screen, the top edge of the screen, the right edge of the screen, or the left edge of the screen.

[0037] (Modification of the first embodiment) In the present embodiment, the object is described as a human being, but the object is not limited to a human being and may be any object that emits a sound. For example, the object may be a dog that emits a barking sound or an ambulance that emits a siren sound.

[0038] As shown in Fig. 5, when the object generating the sound is identified as a dog OB3 by the above-mentioned identification process, the superimposing unit 15 can superimpose text data TD3 corresponding to the sound (in this example, "woof woof") on the balloon image near the OB3 generating the sound. This also produces the same effect as the example shown in Fig. 3.

[0039] On the other hand, as shown in FIG. 6, when the object generating the sound is identified as OB4, an ambulance, by the above-mentioned identification process, the superimposition unit 15 can superimpose text data TD4 corresponding to the sound (in this example, "Peep Peep Peep") on the speech bubble image near OB4 generating the sound. This also produces the same effect as the example shown in FIG. 3.

[0040] In the present embodiment, a configuration has been described in which only text data representing a voice uttered by an object is superimposed near the object, but the present invention is not limited to this. For example, the present invention may be configured as follows.

[0041] That is, the superimposing unit 15 may display, near an object, specific information of the object in addition to text data representing a voice uttered by the object. Here, the specific information of the object is information for identifying the object, such as the name of the object. Also, the superimposition of the specific information may be performed only when the likelihood indicated by the above-mentioned likelihood information is higher than a predetermined level, as in the case of the superimposition of the text data.

[0042] In this case, although not particularly shown, for example, in the above-mentioned FIGS. 3 to 6, the specific information may be displayed at positions on the display unit 40 corresponding to the text data TD1 to TD4.

[0043] According to the above configuration, even a viewer watching the display unit 40, for example, a viewer who is hearing impaired, can more appropriately recognize the association between the object and the name.

[0044] The display method of this embodiment will be described below with reference to Fig. 7. Fig. 7 is a flow chart showing an image generating method executed by the image generating device 10 included in the display device 1 shown in Fig. 1.

[0045] As shown in FIG. 7, the image generating method includes an acquiring step S10, a specific processing executing step S11, and a superimposing step S12.

[0046] In the acquisition step S10, the image generating device 10 acquires image data representing an image input from the outside and audio data representing audio accompanying the image.

[0047] Next, in the identification process execution step S11, the image generating device 10 inputs the image data and audio data acquired in the acquisition step S10 into a trained model to identify the object that is generating the audio among one or more objects contained in the image.

[0048] Finally, in a superimposing step S12, the image generating device 10 superimposes text data corresponding to the voice data at a position within the image that corresponds to the result of the identification process.

[0049] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0050] 8 is a block diagram showing the main configuration of an image generating device 10a according to this embodiment. Compared to the first embodiment, the only difference is that the image generating device 10a according to this embodiment does not include a text data generating unit. The following description will focus on this difference.

[0051] In this embodiment, the image generating device 10a does not have a text data generating unit, and therefore does not generate text data. In this embodiment, the superimposing unit 15 generates an image on which information specifying the object is superimposed, the information having a display form corresponding to the accuracy information indicating the accuracy of the above-mentioned identification process.

[0052] (Example of what is displayed on the display) FIG. 9 is a diagram showing one display example displayed on the display unit 40 shown in FIG. 8, and FIG. 10 is a diagram showing another display example.

[0053] 9, the image generating device 10a identifies two objects that are generating sounds by the above-mentioned identification process. Then, the image generating device 10a generates an image in which the names OB1 and OB2 of these two objects are superimposed in a display manner indicating that the identification process is highly likely. Here, as a display manner indicating that the identification process is highly likely, a display manner is adopted in which a rectangular speech bubble having a protrusion on the side close to the object is displayed together with the object names OB1 and OB2.

[0054] This allows even a viewer watching the display unit 40, for example a viewer with a hearing impairment, to properly recognize the objects, OB1 and OB2 in this example.

[0055] 10, the image generating device 10a identifies two objects that are generating sounds by the above-mentioned identification process. The image generating device 10a then generates an image in which the names OB1 and OB2 of these two objects are superimposed in a display manner indicating that the identification process is unlikely to be reliable. Here, as a display manner indicating that the identification process is unlikely to be reliable, a display manner is adopted in which a rectangular speech bubble with no protruding part on the side closest to the object is displayed together with the object names OB1 and OB2.

[0056] As a result, even a viewer watching the display unit 40, for example a viewer with a hearing impairment, can recognize that the likelihood of the objects, OB1 and OB2 in this example, is low.

[0057] The display method of this embodiment will be described below with reference to Fig. 11. Fig. 11 is a flow chart showing the image generating method executed by the image generating device 10a shown in Fig. 8.

[0058] As shown in FIG. 11, the image generating method includes an obtaining step S20, a specific processing execution step S21, and a superimposing step S22.

[0059] As shown in FIG. 11, in this embodiment, the process contents of steps S20 and S21 are similar to the process contents of steps S10 and S11 in the first embodiment, respectively, and therefore the description thereof will be omitted.

[0060] Then, in the superimposing step S22, the image generating device 10a generates an image on which information identifying the object, which information has a display form corresponding to the accuracy information indicating the accuracy of the identifying process, is superimposed.

[0061] 〔summary〕 An image generating device (10) according to aspect 1 of the present invention includes an acquisition unit (11) that acquires image data representing an image and audio data representing audio accompanying the image, a specific processing execution unit (14) that executes a specific processing to identify an object that is generating audio among one or more objects contained in the image by inputting the image data and audio data acquired by the acquisition unit into a learned model, and a superimposition unit (15) that superimposes text data corresponding to the audio data at a position within the image that corresponds to the result of the identification processing.

[0062] According to the above configuration, a viewer viewing the display device (1), even a viewer with a hearing impairment, can properly recognize the association between the object and the text data (subtitles).

[0063] In an image generating device (10) according to aspect 2 of the present invention, in aspect 1 above, the superimposition unit may determine the position at which to superimpose the text data by referring to accuracy information indicating the accuracy of the identification process.

[0064] According to the above configuration, the superimposing unit (15) can appropriately determine the position at which to superimpose the text data.

[0065] In an image generating device (10) according to aspect 3 of the present invention, in aspect 2 above, the identification process execution unit may execute a first identification process to obtain positions and identification information of one or more objects contained in the image by inputting the image data into a first learned model, and execute a second identification process to obtain identification information of the audio data by inputting the audio data into a second learned model, and identify the object generating the audio by referring to a result of the first identification process and a result of the second identification process.

[0066] According to the above configuration, the identification process execution unit (14) can reliably identify the object that is generating the sound.

[0067] In an image generating device (10) according to aspect 4 of the present invention, in aspect 2 or 3 above, the superimposition unit may superimpose the text data near the object generating the voice when the likelihood indicated by the likelihood information is higher than a predetermined level.

[0068] In an image generating device (10) according to aspect 5 of the present invention, in aspect 4 above, the superimposition unit may, when the likelihood indicated by the likelihood information is higher than a predetermined level, superimpose a speech bubble image indicating that the sound is emitted from the object near the object generating the sound, and superimpose the text data on the speech bubble image.

[0069] According to the above configuration, the superimposing unit (15) can superimpose the text data at a more appropriate position.

[0070] An image generating device (10) according to aspect 6 of the present invention is any one of aspects 2 to 5 above, wherein the superimposition unit may generate an image in which information identifying the object is superimposed near the object when the likelihood indicated by the likelihood information is higher than a predetermined height.

[0071] According to the above configuration, a viewer watching the display unit (40), even a viewer with a hearing impairment, can more appropriately recognize the association between the object and the text data (subtitles).

[0072] An image generating device (10a) according to aspect 7 of the present invention includes an acquisition unit (11) that acquires image data representing an image and audio data representing audio accompanying the image, a specific processing execution unit (14) that executes a specific processing to identify an object that is generating a sound among one or more objects contained in the image by inputting the image data and audio data acquired by the acquisition unit into a learned model, and a superposition unit (15) that generates an image on which information identifying the object is superimposed, the information having a display mode corresponding to accuracy information indicating the accuracy of the identification processing.

[0073] According to the above configuration, the same effects as those of the above first aspect are achieved.

[0074] The image generating method according to aspect 8 of the present invention includes an acquisition step of acquiring image data representing an image and audio data representing audio accompanying the image, an identification process execution step of executing an identification process to identify an object that is generating audio among one or more objects contained in the image by inputting the image data and audio data acquired in the acquisition step into a learned model, and a superimposition step of superimposing text data corresponding to the audio data at a position within the image that corresponds to the result of the identification process.

[0075] According to the above configuration, the same effects as those of the above first aspect are achieved.

[0076] The image generating method according to aspect 9 of the present invention includes an acquisition step of acquiring image data representing an image and audio data representing audio accompanying the image, a specific processing execution step of executing a specific processing to identify an object that is generating the audio among one or more objects contained in the image by inputting the image data and audio data acquired in the acquisition step into a learned model, and a superimposition step of generating an image on which information identifying the object is superimposed, the information having a display mode corresponding to accuracy information indicating the accuracy of the identification process.

[0077] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present invention also includes embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Furthermore, new technical features can be formed by combining the technical means disclosed in the respective embodiments. [Explanation of symbols]

[0078] 1 Display device 10, 10a Image generating device 11 Acquisition Department 12 Voice and image recognition section 13 Text data generation unit 14 Specific Processing Execution Unit 15 Overlapping section 20 Speech recognition database 30 Image Recognition Database 40 Display section OB1~4 Object 1~4 TD1~4 Text data 1~4

Claims

1. an acquisition unit that acquires image data representing an image and audio data representing audio accompanying the image; A specific processing execution unit that executes a specific processing to identify an object that is generating a sound among one or more objects included in the image by inputting the image data and the voice data acquired by the acquisition unit into a trained model; a superimposition unit that superimposes text data corresponding to the voice data at a position within the image according to a result of the identification process, The overlay unit determines a position at which to overlay the text data by referring to probability information indicating a probability in the specified process.

2. The specific process execution unit is A first identification process is executed to obtain positions and identification information of one or more objects included in the image by inputting the image data into a first trained model; A second identification process is executed to obtain identification information of the voice data by inputting the voice data into a second trained model; The object generating the sound is identified by referring to the result of the first identification process and the result of the second identification process.

2. The image generating device of claim 1.

3. The superimposing unit superimposes the text data near the object generating the voice when the likelihood indicated by the likelihood information is higher than a predetermined level.

3. An image generating device according to claim 1 or 2.

4. The overlapping portion is If the likelihood indicated by the likelihood information is higher than a predetermined level, a balloon image indicating that the sound is emitted from the object is superimposed near the object emitting the sound; The text data is superimposed on the speech bubble image.

4. The image generating device of claim 3.

5. The superimposing unit generates an image in which information identifying the object is superimposed near the object when the likelihood indicated by the likelihood information is higher than a predetermined height.

5. An image generating device according to claim 1.

6. an acquisition unit that acquires image data representing an image and audio data representing audio accompanying the image; A specific processing execution unit that executes a specific processing to identify an object that is generating a sound among one or more objects included in the image by inputting the image data and the voice data acquired by the acquisition unit into a trained model; a superimposing unit that generates an image on which information that identifies the object is superimposed, the information having a display form corresponding to certainty information that indicates the certainty of the identification process; and An image generating device comprising:

7. an acquisition step of acquiring image data representing an image and audio data representing audio accompanying the image; A process for executing a process for identifying an object that is generating a sound among one or more objects included in the image by inputting the image data and the sound data acquired in the acquisition process into a trained model; a superimposing step of superimposing text data corresponding to the voice data at a position within the image according to a result of the identification process, In the superimposing step, a position at which to superimpose the text data is determined by referring to accuracy information indicating accuracy in the specific process.

8. an acquisition step of acquiring image data representing an image and audio data representing audio accompanying the image; A process for executing a process for identifying an object that is generating a sound among one or more objects included in the image by inputting the image data and the sound data acquired in the acquisition process into a trained model; a superimposing step of generating an image on which information for identifying the object is superimposed, the information having a display form corresponding to certainty information indicating the certainty of the identification process; An image generating method comprising:

Images