Image analysis device, image analysis method, and program

Abstract

This invention provides an image analysis device, an image analysis method, and a program for easily evaluating scenes captured in images. [Solution] The image analysis device comprises: an image information acquisition unit that acquires image information of a target image; an evaluation index acquisition unit that acquires evaluation indices for evaluating a scene captured in the image; a prompt acquisition unit that acquires a prompt requesting a language model to evaluate the scene captured in the target image based on the evaluation indices acquired by the evaluation index acquisition unit; an evaluation acquisition unit that acquires an evaluation value generated by a language model in response to the prompt acquired by the prompt acquisition unit being transmitted to the language model; and an output unit that outputs the evaluation value acquired by the evaluation acquisition unit in association with the target image.

Description

Technical Field

[0001] The present invention relates to an image analysis apparatus, an image analysis method, and a program.

Background Art

[0002] As a conventional technique, there is a technique for analyzing an image. For example, image processing for performing object recognition on an image captured by a surveillance camera or the like, or image analysis using pre-trained object detection is performed. For example, Patent Document 1 discloses a technique for automatically and continuously monitoring a site using a surveillance camera and an infrared camera installed at the site.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, object recognition technology can only recognize the objects captured in the image, and it is impossible to evaluate what kind of situation the scene captured in the image shows, for example, whether it is a dangerous situation or not. Therefore, in order to evaluate the situation of the scene, there is a problem that it is necessary for a person to look at the image and judge, or to use pattern recognition, a learned model, etc. to judge the situation, which is time-consuming.

[0005] <于 In view of the above problems, an object of the present invention is to provide an image analysis apparatus, an image analysis method, and a program that can easily evaluate the scene captured in an image. [[ID=于0]]

Means for Solving the Problems

[0006] An image analysis apparatus according to an aspect of the present invention includes an image information acquisition unit that acquires image information of a target image, and an image... Place the table It should be noted that there seems to be an error in the original text where "于0000028" is present. It is translated as "<于 " for the purpose of maintaining the original text as accurately as possible. Please check the original text for any such inaccuracies.An evaluation index acquisition unit acquires evaluation indices to evaluate the scene, and the target image Place the table The system includes: a prompt acquisition unit that acquires a prompt requesting a language model to evaluate the scene based on evaluation metrics acquired by the evaluation metric acquisition unit; an evaluation acquisition unit that acquires an evaluation value generated by the language model in response to the prompt acquired by the prompt acquisition unit being sent to the language model; and an output unit that outputs the evaluation value acquired by the evaluation acquisition unit in association with the target image. [Effects of the Invention]

[0007] According to the present invention, scenes captured in images can be easily evaluated. Therefore, it becomes easy to extract images containing a desired scene from a large number of images included in a video or the like. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of the image analysis system according to the embodiment. [Figure 2] This is a block diagram showing the configuration of an image analysis device according to an embodiment. [Figure 3] This is a diagram illustrating the processing performed by the image analysis device according to the embodiment. [Figure 4] This is a diagram illustrating the processing performed by the image analysis device according to the embodiment. [Figure 5] This is a diagram illustrating the processing performed by the image analysis device according to the embodiment. [Figure 6] This is a diagram illustrating the processing performed by the image analysis device according to the embodiment. [Figure 7] This is a flowchart showing the processing flow performed by the image analysis device according to the embodiment. [Figure 8] This is a diagram illustrating the processing performed by the image analysis device according to a modified example 2 of the embodiment. [Figure 9] This is a diagram illustrating the processing performed by the image analysis device according to the third modified embodiment. [Figure 10] This is a diagram illustrating the processing performed by the image analysis device according to the third modified embodiment. [Figure 11] This is a diagram illustrating the processing performed by the image analysis device according to the third modified embodiment. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0010] Figure 1 is a block diagram showing the configuration of an image analysis system according to an embodiment. The image analysis system 1 comprises an image analysis device 10, a language model server 30, a message distribution server 40, a user terminal 50, a message distribution destination terminal 60, and a message distribution destination server 70. These are connected to each other via a network NW such as the Internet, enabling them to communicate with one another.

[0011] The language model server 30 is an AI that learns patterns and relationships based on collected training data and performs natural language processing based on what it has learned. For natural language processing, it can use multimodal LLMs (Large Language Models) that have the ability to process multiple data formats in an integrated manner, such as images, audio, and video, in addition to text. For example, ChatGPT(registered trademark), XLNet, T5, RoBERTa, ALBERT, etc. can be used as the language model server 30. The language model server 30 is connected to the image analysis device 10 in a communicative manner, receives requests from the image analysis device 10 via an API (Application Programming Interface), and performs processing according to the received requests.

[0012] The message delivery server 40 is a computer that provides a message delivery service. The message delivery service is a service that provides a platform for sharing messages (which may be only text or a combination of text and image or sound information), so that the message sent by the sender can be received by the recipient who is the destination of the message. On the message delivery server 40, a server-side message delivery application program (server-side delivery app) is installed, and by executing this program (server-side delivery app) on a CPU or the like provided in the message delivery server 40, processing for delivering messages is realized. As processing for delivering messages, the message delivery server 40, for example, receives a request for message transmission from the sender, and receives the destination of the message, the message content, etc. from the sender terminal. A client-side message delivery application program (client-side delivery app) is installed on the sender terminal, and by executing this program (client-side delivery app) on a CPU or the like provided in the sender terminal, processing for transmitting messages is realized. Also, as processing for delivering messages, the message delivery server 40 transmits a message to the recipient terminal which is the destination of the message, and controls so that the message is received by the recipient terminal. A client-side delivery app is installed on the recipient terminal, and by executing this program (client-side delivery app) on a CPU or the like provided in the recipient terminal, processing for receiving messages is realized. In the present embodiment, the message delivery server 40 transmits a message to the destination specified from the image analysis device 10. Here, the destination specified from the image analysis device 10 assumed in the present embodiment is any one of the user terminal 50, the message delivery destination terminal 60, and the message delivery destination server 70.

[0013] The image analysis device 10 is a computer that provides an image analysis service. As the image analysis device 10, a PC, a server, a cloud, etc. can be applied. The image analysis service is a service that evaluates what situation the scene captured in the image indicates based on evaluation indicators and provides the evaluation result. On the image analysis device 10, an image analysis application program on the server side (image analysis app on the server side) is installed, and by executing this program (image analysis app on the server side) with a CPU or the like provided in the image analysis device 10, processing for analyzing an image is realized.

[0014] Note that the function of analyzing an image in the image analysis device 10 may be provided in the user terminal 50. In this case, a program corresponding to the image analysis app on the server side is installed in the user terminal 50. By executing this program corresponding to the image analysis app on the server side with a CPU or the like provided in the user terminal 50, processing for analyzing an image is realized. Thereby, an image can be analyzed at the user terminal 50, and the image analysis device 10 can be omitted. The function of language analysis in the language model server 30 may be provided in the user terminal 50. In this case, the user terminal 50 has a function of performing natural language processing, for example, an LLM, a small-scale language model, or a multimodal language model that can support images and other formats. Thereby, natural language processing can be performed at the user terminal 50, and the language model server 30 can be omitted. Thereby, for example, even when it is difficult for the user terminal 50 to communicate with the image analysis device 10, such as when the radio wave condition is poor, the image analysis using natural language processing of the present embodiment can be executed.

[0015] The user terminal 50 is a communication terminal operated by the user, such as a smartphone or a PC (personal computer). Here, the user is one who utilizes the image analysis provided by the image analysis system 1. The user terminal 50 has a client-side image analysis application program (client-side image analysis app) installed, and the CPU or other components of the user terminal 50 execute this program (client-side image analysis app) to perform image analysis. The message delivery terminal 60 is a terminal device that can be set as the destination for messages (warning messages to the monitored object) delivered by the message delivery server 40. The message delivery terminal 60 can be, for example, a terminal device operated by a person related to the user, such as the user's family or friends, or a person authorized to operate the monitored object. The message delivery server 70 is a server device that can be set as the destination for messages (warning messages to the monitored target) delivered by the message delivery server 40. For example, a server device managed by the business operator managing the monitored target can be used as the message delivery server 70.

[0016] The above explanation uses the example of providing an image analysis service via an image analysis application, but is not limited to this. The image analysis service may also be provided as a web service via a web browser or the like. Furthermore, as mentioned above, by equipping the user terminal 50 with image analysis and language analysis functions, the image analysis service may be provided without the user terminal 50 communicating with the outside world.

[0017] As part of the image analysis process, the image analysis device 10, for example, receives an image analysis request from a user, evaluates what kind of situation the captured scene in the image represents based on evaluation indicators, and transmits the evaluation result to the user terminal 50. The image analysis process will be described in detail below.

[0018] Figure 2 is a block diagram showing the configuration of the image analysis device 10 according to the embodiment. The image analysis device 10 includes an image information acquisition unit 101, an evaluation index acquisition unit 102, a prompt acquisition unit 103, an evaluation acquisition unit 104, a warning determination unit 105, an evaluation result generation unit 106, and an output unit 107.

[0019] The image information acquisition unit 101 acquires image information of the image to be analyzed. For example, when a user operates the user terminal 50 and launches the image analysis application, an input field (image input field) for entering the image to be analyzed is displayed. The user enters information specifying the image to be analyzed into the image input field. The information used to specify an image can include the image file itself, or a URL (Uniform Resource Locator) indicating the location where the image is stored. When using an image file as the information to specify an image, for example, the user can specify the image to be analyzed by dragging and dropping the image file into the image input field. When using a URL as the information to specify an image, the user can specify the image to be analyzed by entering the URL of the image to be analyzed into the image input field. The user terminal 50 acquires information to specify an image in response to the user's input and displays the image based on the acquired information in the display area of ​​the image to be analyzed (see area R1 in Figure 3, described later). Based on the interval for capturing the image to be analyzed (analysis interval) set by the user (see area R4 in Figure 3, described later), the user terminal 50 sequentially transmits the captured images of the image to be analyzed to the image analysis device 10. The image analysis device 10 receives the image information of the captured image notified from the user terminal 50 and outputs the received image information to the image information acquisition unit 101. As a result, the image information acquisition unit 101 acquires the image information of the image to be analyzed.

[0020] The evaluation indicator acquisition unit 102 acquires indicator information. For example, when a user operates the user terminal 50 and launches the image analysis application, an input field for entering the image to be analyzed (image input field) and an input field for entering information to specify the evaluation metrics (evaluation metric input field) are displayed. The user enters the information to specify the evaluation metrics into the evaluation metric input field.

[0021] Any evaluation metric can be used, as long as it is expressed in a way that the language model can recognize.

[0022] For example, a user enters a term expressing the degree of danger, such as "danger level," as an evaluation metric in the evaluation metric input field. By setting such a term as an evaluation metric and entering a prompt in the prompt input field that requests the language model to perform an evaluation according to the set evaluation metric "danger level," scenes with a high "danger level," such as fires, collapses, or accidents, will be extracted from the images being analyzed. On the other hand, even if flames are captured in an image, if the situation is such as a cooking video where the flames are controllable, that scene will be recognized as having a low "danger level" and will not be extracted.

[0023] Furthermore, users enter a term called "Suspiciousness Level" in the evaluation metric input field, which expresses the degree to which a person is behaving suspiciously. By setting such a term as an evaluation metric and entering a prompt in the prompt input field that requests the language model to perform an evaluation according to the set evaluation metric "Suspiciousness Level," scenes with a high "Suspiciousness Level," such as a scene where person A is attempting to enter from a balcony, will be extracted from the images being analyzed. On the other hand, even if it is the same person A, if the scene is simply one of them walking on the sidewalk, the "Suspiciousness Level" will be recognized as low and the scene will not be extracted.

[0024] Furthermore, users enter a phrase expressing the degree of need for rescue, called "Rescue Needs," into the evaluation metric input field. By setting such a phrase as an evaluation metric and prompting the language model to perform an evaluation according to the set evaluation metric "Rescue Needs," scenes with a high "Rescue Needs" are extracted from the images being analyzed, such as those involving fires, collapses, or accidents where people or structures likely to contain people (such as houses or shops) are involved. On the other hand, scenes of similar fires, collapses, or accidents occurring in deserted areas such as open grasslands or coastlines are recognized as having a low "Rescue Needs" and are not extracted.

[0025] Alternatively, the user enters a term in the evaluation metric input field, "Baseball Player B Degree," which expresses the degree to which a popular and famous baseball player B is performing actions that are considered "worthy of footage" (likely to be used in broadcast). By setting such a term as an evaluation metric and entering a prompt in the prompt input field that requests the language model to evaluate according to the set evaluation metric "Baseball Player B Degree," scenes such as baseball player B standing in the batter's box will be extracted from the images being analyzed as having a high "Baseball Player B Degree." On the other hand, even if it is the same baseball player B, scenes where he is looking down or are unclear will be recognized as having a low "Baseball Player B Degree" and will not be extracted.

[0026] The user terminal 50 acquires information specifying evaluation indicators in response to the user's input and transmits the acquired information to the image analysis device 10. The image analysis device 10 receives the information notified from the user terminal 50 (information specifying evaluation indicators) and outputs the received information (evaluation indicators) to the evaluation indicator acquisition unit 102. As a result, the evaluation indicator acquisition unit 102 acquires the evaluation indicators to be analyzed.

[0027] For example, an image analysis app displays an input field for entering information that specifies evaluation metrics (evaluation metric input field), as well as an input field for entering prompts (prompt input field). The user enters a prompt in the prompt input field, such as "Please rate this image's (evaluation metric) on a numerical scale from 0 to 100," requesting the language model to evaluate the evaluation metric. When a user enters "risk level" as an evaluation metric in the evaluation metric input field, they will enter a prompt in the prompt input field such as "Please express the (risk level) of this image on a numerical value from 0 to 100." The user terminal 50 acquires information to specify a prompt in response to the user's input and transmits the acquired information to the image analysis device 10. The image analysis device 10 receives the information notified from the user terminal 50 (information specifying a prompt) and outputs the received information (prompt) to the evaluation index acquisition unit 102. As a result, the evaluation index acquisition unit 102 acquires the prompt. This prompt is a natural language (text statement) sent to the language model server 30 to cause the language model server 30 to perform processing in accordance with the prompt.

[0028] Furthermore, the prompt acquisition unit 103 may generate a prompt according to the evaluation index when the user does not enter a prompt in the prompt input field. In this case, for example, the prompt acquisition unit 103 generates a prompt such as, "Please express the (evaluation index) of this image as a number from 0 to 100." The (evaluation index) field here displays the evaluation index acquired by the evaluation index acquisition unit 102, such as danger level, suspiciousness level, rescue need level, baseball player B level, etc.

[0029] Furthermore, the prompt acquisition unit 103 may generate a prompt that requests the user to indicate the reason for the evaluation along with the evaluation value. For example, the prompt acquisition unit 103 generates a prompt such as, "Please express the (evaluation index) of this image as a number from 0 to 100 and explain the reason." The (evaluation index) field here displays the evaluation index acquired by the evaluation index acquisition unit 102, such as danger level, suspiciousness level, rescue need level, baseball player B level, etc.

[0030] The evaluation unit 104 obtains an evaluation. If the image to be analyzed is a moving image, the evaluation acquisition unit 104 generates a group of still images captured from the moving image at time intervals set in advance by the user. The evaluation acquisition unit 104 then sends each of the generated still images and a prompt to the language model server 30, requesting the language model to analyze the image according to the prompt. The image to be analyzed here is an image based on image information acquired by the image information acquisition unit 101. The prompt here is a prompt generated by the prompt acquisition unit 103.

[0031] The language model server 30 evaluates the scene captured in the image according to the evaluation index, in accordance with the prompt notified by the image analysis device 10, and transmits the evaluation value to the image analysis device 10 as a response to the prompt.

[0032] If the prompt notified to the language model server 30 requests an evaluation according to an evaluation index, the language model server 30 sends a response to the prompt to the image analysis device 10, such as, "The danger level of this image is 85," "The suspiciousness level of this image is 10," "The rescue need level of this image is 90," or "The baseball player B level of this image is 0 (zero)."

[0033] If the prompt notified to the language model server 30 requests that the evaluation value be accompanied by the reason for the evaluation, the language model server 30 will send a response to the prompt to the image analysis device 10, such as: "Risk level: 85, This image shows a collapse, so it is highly dangerous.", "Suspicious person level: 10, No suspicious person was observed, and it appears safe.", "Rescue necessity level: 90, Due to the collapse, rescue is extremely necessary.", "Baseball player B level: 0 (zero), Baseball player B is not present."

[0034] The image analysis device 10 receives a response notified from the language model server 30 and outputs the received response to the evaluation acquisition unit 104. As a result, the evaluation acquisition unit 104 acquires evaluation values ​​for evaluation indicators and the reasons for them as a response to the prompt.

[0035] The warning determination unit 105 determines whether or not to issue a warning to the user.

[0036] This warning system alerts the user when an indicator value exceeds a threshold. Users can configure whether or not to issue a warning, and what the threshold value is. For example, when a user operates the user terminal 50 and launches the image analysis application, an input field for entering the image to be analyzed (image input field) is displayed, along with a warning settings field. The warning settings field includes, for example, whether or not to set a threshold, the threshold (if a threshold is set), and the selection of an action. The action here refers to selecting the notification method to which the warning will be sent, for example, selecting one or more notification methods such as SNS (Social Networking Service), SMS (Short Message Service), and email. In the settings section, users can choose whether or not to set a threshold, and if so, enter its value. For example, the threshold can be set to any value within a range of evaluation values ​​corresponding to the evaluation metric (e.g., 0 to 100). Users can also select a notification method from the action options and enter the recipient of notifications using the selected method (e.g., SNS account, mobile phone number, email address, etc.). The user terminal 50 acquires information set in the warning settings field (such as whether a threshold is set, the threshold, and the result of the action selection) in response to the user's input operation, and transmits the acquired information to the image analysis device 10. The image analysis device 10 receives the information notified from the user terminal 50 (information set in the warning settings field) and outputs the received information to the warning determination unit 105.

[0037] The warning determination unit 105 compares the evaluation value acquired by the evaluation acquisition unit 104 with the threshold value for each evaluation index for which a threshold value has been set. If the evaluation value is greater than or equal to the threshold value, the warning determination unit 105 determines to issue a warning. On the other hand, if the evaluation value is less than the threshold value, the warning determination unit 105 determines not to issue a warning. If a warning is to be issued, the warning determination unit 105 sends the warning via a notification receiving stage corresponding to the action selection result. More specifically, the warning determination unit 105 sends a user-addressed message as a warning to the message distribution server 40 via the communication unit (not shown) of the image analysis device 10. The user-addressed message is notified to the destination set as the notification destination in area R3, for example, to the user terminal 50, the message distribution destination terminal 60, the message distribution destination server 70, or a combination thereof.

[0038] The evaluation result generation unit 106 generates evaluation results. If the image to be analyzed is a moving image, the evaluation result generation unit 106 generates, for example, a graph showing the evaluation values ​​of each still image captured from the moving image in chronological order. Furthermore, the evaluation result generation unit 106 may include detailed information in the evaluation results for each still image, associating the evaluation value with the reason for that evaluation. In this case, for example, when a graph or time axis showing the evaluation values ​​corresponding to each still image captured from the video is selected, detailed information for the still image corresponding to the selected position is displayed in a pop-up window. This allows the user to visually check the time-series graph showing the evaluation results to understand the changes in the evaluation values ​​of the video, or to check the images corresponding to high evaluation values ​​to understand the reasons for those evaluations. The output unit 107 outputs various information. For example, the output unit 107 transmits an image showing the evaluation results generated by the evaluation result generation unit 106 to the user terminal 50 via the communication unit (not shown) of the image analysis device 10. As a result, the evaluation results are displayed on the user terminal 50 (on the image analysis application screen).

[0039] Here, we will explain the processes performed by the image analysis device according to the embodiment using Figures 3 to 6. Figures 3 to 6 are diagrams illustrating the processes performed by the image analysis device according to the embodiment.

[0040] Figure 3 shows an example of an image displayed on the user terminal 50. In this figure, an image is displayed in each of the regions R1 to R4 on the user terminal 50. Region R1 displays the image to be analyzed. In region R2, a time-series graph is displayed showing the evaluation results for each still image captured over time from the video image being analyzed, as part of the analysis results. Area R3 displays input fields for setting evaluation metrics, prompts, and warning thresholds. Area R4 contains operation buttons for controlling image analysis, including a "Share Image" button to start analysis and a "Stop Analysis" button to stop analysis. Area R4 also has a setting field for setting the interval (analysis interval) at which the target image (video) is captured for analysis.

[0041] Figure 4 shows a magnified view of the image displayed in region R3 of Figure 3. In this figure, an operation button B for adding prompts and input fields E (input fields E1, E2) for entering evaluation metrics are provided. The number of input fields E is provided according to the number of evaluation metrics set in the image analysis. Input fields E are provided in region R3 depending on when the operation button is pressed. In this example, two input fields E (input fields E1, E2) are provided because operation button B has been pressed twice.

[0042] Input field E1 includes a title field T1, where the evaluation metric used for analysis is set. In this figure, "Risk Level" is set as the evaluation metric. Input field E1 also includes a prompt field P1, which displays a prompt generated according to the evaluation metric set in title field T1. In this figure, the prompt "Please express the risk level of this image on a scale from 0 to 100 and explain the reason" has been generated. Input field E1 also includes a threshold setting field M1. The threshold setting field M1 includes a checkbox to specify whether or not to set a threshold, a numerical box to specify the threshold, and a selection box to specify the action (notification method for issuing a warning). In this figure, the threshold is set to "69," and if the risk level of the target image exceeds the threshold after analysis, a warning will be sent via LINE® message.

[0043] Input field E2 has the same configuration as input field E1, and includes a title field T2, a prompt field P2, and a threshold setting field M2. In this figure, "Suspiciousness Level" is set as the evaluation index in the title field T2. The prompt field P2 shows that a prompt has been generated: "Please express the suspiciousness level of this image on a number from 0 to 100 and explain the reason." The threshold setting field M2 shows that the threshold is set to "51," and that a notification method for when the suspiciousness level exceeds the threshold has not yet been selected.

[0044] Figure 5 shows a magnified view of the image displayed in region R2 of Figure 3. In this figure, the horizontal axis represents time (playback time of the video), and the vertical axis represents the evaluation value. In this figure, three evaluation indicators are set: "Danger Level," "Suspiciousness Level," and "Rescue Need Level." A threshold th1 is set for the evaluation value of "69" for the first evaluation indicator, "Danger Level." A threshold th2 is set for the evaluation value of "51" for the second evaluation indicator, "Suspiciousness Level." In this figure, it is shown that in a still image captured with a video playback time of "17:22:26," the evaluation value for "Danger Level" is 90, the evaluation value for "Suspiciousness Level" is 10, and the evaluation value for "Rescue Need Level" is 80.

[0045] Figure 6, like Figure 5, shows a magnified view of the image displayed in region R2 of Figure 3. This figure also shows an example of a pop-up image PG that appears when the user selects an evaluation value corresponding to the playback time T in the graph display. The pop-up image PG includes a captured image DG and a descriptive text DT. Captured image DG displays the captured image corresponding to the playback time T selected by the user. The descriptive text DT shows the evaluation value and the reason for it as the analysis result for the captured image corresponding to playback time T. In this figure, captured image DG shows a house being swept away by a landslide. The descriptive text DT states that the danger level is "90" and the reason is "This image shows a collapse, so it is of high danger." The descriptive text DT also states that the suspicious person level is "10" and the reason is "No suspicious person was observed, and it appears safe." Furthermore, the descriptive text DT states that the rescue necessity level is "80" and the reason is "Due to the collapse, there is a high need for rescue."

[0046] Here, the processing flow performed by the image analysis device 10 will be explained using Figure 7. Figure 7 is a flowchart showing the processing flow performed by the image analysis device 10 according to this embodiment.

[0047] The image analysis device 10 acquires image information of the image to be analyzed (step S101). The image analysis device 10 acquires image information of the image to be analyzed by receiving a captured image from the user terminal 50. The image analysis device 10 acquires evaluation indicators (step S102). The image analysis device 10 acquires evaluation indicators by receiving the evaluation indicators set by the user from the user terminal 50 according to the analysis screen displayed on the user terminal 50 (for example, the screen shown in Figure 3).

[0048] In this case, if the image to be analyzed is a moving image, the image analysis device 10 sequentially notifies the user terminal 50 of still images (captured images) captured from the moving image at analysis intervals predetermined by the user, and for each of these sequentially notified captured images, the process described in steps S103 to S108 is repeatedly executed.

[0049] The image analysis device 10 generates a prompt according to the evaluation index acquired in step S102, and sends the generated prompt to the language model server 30, thereby requesting the language model to evaluate the image to be analyzed (step S103). The image analysis device 10 acquires evaluation values ​​(step S104). The language model server 30 receives a prompt notified by the image analysis device 10, analyzes the image according to the received prompt, and transmits the analysis results to the image analysis device 10. The analysis results include evaluation values ​​for the evaluation indicators shown in the prompt, and the reasons for those evaluation values. The image analysis device 10 receives the analysis results notified by the language model server 30 and acquires evaluation values ​​from the received analysis results.

[0050] The image analysis device 10 determines whether or not to issue a warning to the user (step S105). The image analysis device 10 determines to issue a warning to the user if the user has pre-set thresholds and actions, and the evaluation value obtained in step S104 is greater than or equal to the threshold. On the other hand, if the user has not pre-set thresholds or actions, or if the evaluation value obtained in step S104 is less than the threshold, the image analysis device 10 determines not to issue a warning to the user. If a warning is to be issued, the image analysis device 10 sends a warning that the evaluation value is above a threshold value via a notification receiving step (action) pre-configured by the user (step S106).

[0051] If no warning is issued, or if a warning is issued, after the warning is executed, the image analysis device 10 generates the evaluation results (step S107). The image analysis device 10 generates a graph showing the changes in the evaluation value, as shown in region R2 of Figure 3 and in Figure 5, as the evaluation results. The image analysis device 10 also generates an evaluation result, as shown in Figure 6, which associates the evaluation value of the playback time T with the captured image, the evaluation value, and the reason for that evaluation value. The image analysis device 10 transmits the evaluation results generated in step S107 to the user terminal 50 (step S108). The user terminal 50 receives the evaluation results notified by the image analysis device 10 and displays the received evaluation results on the user terminal 50's display. This allows the user to visually observe the changes in evaluation values, as shown in region R2, in an image like the one shown in Figure 3. Furthermore, by selecting points of interest where the evaluation values ​​change, the user can check what kind of image it is and the reasons for the evaluation.

[0052] As described above, according to this embodiment, the image analysis device 10 comprises an image information acquisition unit 101, an evaluation index acquisition unit 102, a prompt acquisition unit 103, an evaluation acquisition unit 104, and an output unit 107. The image information acquisition unit 101 acquires image information of a target image. The evaluation index acquisition unit 102 acquires evaluation indexes for evaluating a scene captured in an image. The prompt acquisition unit 103 acquires a prompt that requests the generating AI (language model) to evaluate the scene captured in the target image based on the evaluation index acquired by the evaluation index acquisition unit 102. The evaluation acquisition unit 104 acquires an evaluation value generated by the generating AI (language model) in response to the prompt acquired by the prompt acquisition unit 103 being sent to the generating AI (language model). The output unit 107 outputs the evaluation value acquired by the evaluation acquisition unit 104 in association with the target image. As a result, the image analysis device 10 of this embodiment can easily evaluate a scene because it can make the generating AI (language model) evaluate the scene captured in an image.

[0053] Furthermore, according to this embodiment, the image analysis device 10 has an evaluation index acquisition unit 102 that acquires multiple evaluation indices. The prompt acquisition unit 103 acquires a prompt that requests the generating AI to evaluate the scene captured in the target image based on each of the multiple evaluation indices acquired by the evaluation index acquisition unit 102. The evaluation acquisition unit 104 acquires the evaluation value for each of the multiple evaluation indices generated by the generating AI. The output unit 107 outputs the multiple evaluation values ​​acquired by the evaluation index acquisition unit 102, associating them with the target image. As a result, the image analysis device 10 of this embodiment can evaluate the scene captured in the image for each of the multiple evaluation indices, making it easy to analyze the image from various perspectives.

[0054] Furthermore, according to this embodiment, in the image analysis device 10, the prompt acquisition unit 103 generates a prompt requesting the generation AI to generate an explanatory text explaining the basis for the evaluation. The evaluation acquisition unit 104 acquires the evaluation value and explanatory text generated by the generation AI. The output unit 107 outputs the evaluation value associated with the explanatory text. As a result, the image analysis device 10 of this embodiment can present to the user the reasons for evaluating the scene captured in the image.

[0055] Furthermore, according to this embodiment, the image analysis device 10 includes an image information acquisition unit 101 that acquires image information from a moving image. The evaluation acquisition unit 104 sequentially sends prompts to the language model requesting the generating AI to evaluate the scenes captured in the target images captured from the moving image at predetermined analysis intervals. The output unit 107 outputs a graph showing the evaluation values ​​acquired by the evaluation index acquisition unit 102 along the time series of the moving image. As a result, the image analysis device 10 of this embodiment makes it easy to grasp the changes in the evaluation values ​​of each scene captured in the moving image. It is possible to understand what kind of scenes are captured and where in the video without having to play the moving image back and forth.

[0056] Furthermore, according to this embodiment, the image analysis device 10 includes an image information acquisition unit 101 which acquires image information from a moving image. The evaluation acquisition unit 104 evaluates the scenes captured in target images captured from the moving image at predetermined analysis intervals and sequentially sends prompts to the language model requesting the generation AI to generate explanatory text explaining the basis for the evaluation. The output unit 107 outputs a graph showing the evaluation values ​​acquired by the evaluation index acquisition unit 102 along the time series of the moving image, and provides information that associates the target image, evaluation value, and basis for the evaluation with a point in the graph corresponding to a specific time. As a result, in the image analysis device 10 of this embodiment, it becomes easy to view the changes in the evaluation value of each scene captured in the moving image, to visually examine the captured images, and to check the reasons for the evaluation for points of interest.

[0057] Furthermore, according to this embodiment, the image analysis device 10's output unit 107 displays the target image acquired by the image information acquisition unit 101 and an input field for inputting evaluation indicators on the same screen of the user terminal 50. The evaluation indicator acquisition unit 102 acquires the evaluation indicators entered in the input field. As a result, the image analysis device 10 of this embodiment can display the target image on the user terminal 50 and also provide an input field on the same screen where evaluation indicators can be set. Therefore, users can monitor the target image and, if they want to extract a scene of particular interest, set an evaluation indicator corresponding to that scene, thus improving convenience.

[0058] Furthermore, according to this embodiment, in the image analysis device 10, the image information acquisition unit 101 acquires image information of the moving image. The output unit 107 displays the moving image acquired by the image information acquisition unit 101 and an input field for inputting evaluation indicators on the same screen of the user terminal. The evaluation indicator acquisition unit 102 acquires the evaluation indicators entered in the input field. The evaluation acquisition unit 104 sequentially sends prompts to the language model requesting the generating AI to evaluate the scenes captured in the target images, which are captured from the moving image at predetermined analysis intervals. The output unit 107 displays a graph on the same screen of the user terminal 50 showing the moving image acquired by the image information acquisition unit 101, the input field, and the evaluation values ​​acquired by the evaluation acquisition unit 104, along with the time series of the moving image. As a result, in this embodiment, the image analysis device 10 can display the moving image (region R1), a graph showing the trend of evaluation values ​​(region R2), and an input field for setting evaluation values ​​(region R3) on the same screen of the user terminal 50, as shown in Figure 3. Therefore, users can monitor the target image, set evaluation metrics, and check the trend of evaluation values ​​all on the same screen, improving convenience.

[0059] (Modification of Embodiment 1) Here, we will describe a modification of the embodiment 1. This modification differs from the embodiment described above in that it improves evaluation accuracy based on user feedback.

[0060] In the embodiment described above, an objective evaluation is achieved by having a language model perform evaluations based on evaluation indicators. However, some users may feel that the evaluation is inconsistent. For example, they may feel that "since the image shows a fire in progress, the evaluation score for danger should be higher," or "the person in the center of the image is wearing a full-face helmet, so the evaluation score for suspiciousness should be higher." As a countermeasure, this modified version allows for adjustment of the evaluation level by the language model to match the user's perception. Below, we will explain the specific processing flow for improving evaluation accuracy based on user feedback.

[0061] The image analysis device 10 displays an input field (feedback input field) on the user terminal 50 for the user to set feedback on the evaluation value. For example, in the pop-up image PG shown in Figure 6, a button labeled "Give Feedback" (feedback button) is provided. When this feedback button is operated, a screen is displayed for uploading natural language (text) and an image or video. For example, suppose the "suspiciousness level" evaluated by the language model for a certain target image is lower than the user's perception. In this case, the user inputs the target image along with natural language (text) such as "In this image, the suspicious person is wearing a full-face helmet, so the suspiciousness level should be high" on the upload screen. Through this user operation, the target image and the natural language (text) "In this image, the suspicious person is wearing a full-face helmet, so the suspiciousness level should be high" are uploaded. Alternatively, feedback could be provided to help the language model recognize individuals or other elements that the user cannot identify when viewing the target image, thereby enabling the language model to recognize those elements. For example, if a language model is asked to evaluate "Actor C Degree," a term that expresses the degree to which a popular and famous actor C behaves in line with actor C's image, and the language model fails to recognize actor C even though actor C is captured in the image, the user can input "This is actor C. Remember that" along with a photo of actor C on the feedback screen. By providing this kind of feedback, the language model can recognize actor C with greater accuracy and output more appropriate evaluation values ​​for "Actor C Degree" in the future. Alternatively, when the feedback button is pressed, a screen (feedback selection screen) may be displayed where the user can select options such as "evaluation value is too high," "evaluation value is appropriate," or "evaluation value is too low" for each evaluation metric. In this case, the user selects an option according to the feedback selection screen and presses the submit button provided on the feedback selection screen. Upon pressing the submit button, the user terminal 50 acquires the selection result from the feedback selection screen and transmits the acquired selection result to the image analysis device 10. The image analysis device 10 acquires feedback from the user by receiving the selection result notified from the user terminal 50.

[0062] The image analysis device 10 stores user feedback in external data. This external data is a database that the language model references. By requesting the language model to generate a response by referencing the external data, the language model can derive evaluation values ​​using information about the user's unique evaluation tendencies in addition to the existing knowledge gained during training. This method of having a language model generate a response by referencing external information is sometimes called RAG (Retrieval-Augmented Generation).

[0063] The image analysis device 10 stores information that associates external data with, for example, images, evaluation indices, evaluation values ​​from a language model, and user feedback. User feedback is provided by pointing out discrepancies between the evaluation by the language model and the user's perception using natural language (text). Alternatively, user feedback is provided by pointing out matters not recognized by the language model using natural language (text). Alternatively, the evaluation value from the language model may be judged as either "the evaluation value is too high," "the evaluation value is appropriate," or "the evaluation value is too low."

[0064] The prompt acquisition unit 103 of the image analysis device 10 generates prompts that instruct the user to perform an evaluation by referring to external data. For example, the prompt acquisition unit 103 generates prompts such as, "Please generate an evaluation value representing the (evaluation index) of this image as a number from 0 to 100 by referring to external data, and explain the reason for that evaluation value." As a result, the evaluation acquisition unit 104 of the image analysis device 10 can acquire evaluation values ​​from the language model that are generated by referencing external data, or more specifically, evaluation values ​​that are evaluated in accordance with the user's perception as shown in the external data.

[0065] As described above, according to the modified embodiment 1, the image analysis device 10 receives feedback from the user terminal 50 and stores the received feedback in external data. The prompt acquisition unit 103 refers to the external data and generates a prompt instructing the user to perform an evaluation. As a result, the image analysis device 10 of the modified embodiment can acquire evaluation values ​​that are evaluated in accordance with the user's perception.

[0066] (Modified embodiment 2) Here, a second modification of the embodiment will be described. This modification differs from the embodiment described above in that, when evaluating each still image captured from the moving image, multiple images taken in the vicinity of the still image to be evaluated, in chronological order, are used along with the still image to be evaluated.

[0067] In the embodiment described above, still images captured from video footage were used as the evaluation target for analysis by the language model, and numerical values ​​(evaluation values) and logs (reasons for evaluation) were output as analysis results. As a result, these still images were not used in subsequent analyses. However, the performance of language models has improved remarkably, and it is expected that in the future, videos will be able to be used as a basis for evaluation. From this perspective, this modified version incorporates a configuration that allows evaluation based on multiple still images that are sequentially arranged in a time series within a moving image. Furthermore, it includes a configuration that allows audio information, such as voices collected within the moving image, to be included in the evaluation. This is expected to improve the accuracy of image analysis.

[0068] Figure 8 is a diagram illustrating the processing performed by the image analysis device according to the modified embodiment 2. In this figure, the horizontal axis represents time. The upper section schematically shows the audio and images in chronological order in a moving image. The lower section schematically shows a time-series graph of evaluation results corresponding to region R2 in Figure 3. In this diagram, the video is analyzed every 5 seconds. The video starts at time T1, and time T5 is the current time.

[0069] To derive the evaluation value P1 at time T1, a still image CG1 captured at any point between time T0 (not shown) and time T1 (20 seconds before the current time), and audio CV1 collected continuously from time T0 to time T1, or at a point or interval between time T0 and time T1, are used. The derivation of the evaluation value P2 at time T2 involves not only the still image CG2 captured at time T2 (15 seconds prior to the present) and the audio CV2 collected at time T2, but also multiple still images (which may or may not include still image CG1) captured at arbitrary time intervals between time T0 and time T2, and multiple audio recordings taken at a specific point in time or interval (which may or may not include audio CV1) between time T0 and time T2, or continuously from time T0 to time T2. The derivation of the evaluation value P3 at time T3 involves not only the still image CG3 captured at time T3 (10 seconds prior to the present) and the audio CV3 collected at time T3, but also multiple still images captured at arbitrary time intervals between time T0 and time T3 (which may or may not include all or part of the still images CG1 and CG2), and a point in time or interval between time T0 and time T3 (which may or may not include all or part of the audio CV1 and CV2), or multiple audio recordings continuously collected from time T0 to time T3. The derivation of the evaluation value P4 at time T4 involves not only the still image CG4 captured at time T4 (5 seconds prior to the present) and the audio CV4 collected at time T4, but also multiple still images captured at arbitrary time intervals between time T1 and time T4 (which may or may not include all or part of the still images CG1, CG2, and CG3), as well as a point in time or interval between time T1 and time T4 (which may or may not include all or part of the audio CV1, CV2, and CV3), or multiple audio recordings continuously collected from time T1 to time T4. The derivation of the evaluation value P5 at time T5 involves not only the still image CG5 captured at time T5 (the current time) and the audio CV5 collected at time T5, but also multiple still images captured at arbitrary time intervals between time T2 and time T5 (which may or may not include all or part of the still images CG2, CG3, and CV4), as well as a specific point in time or interval between time T2 and time T5 (which may or may not include all or part of the audio CV2, CV3, and CV4), or multiple audio recordings continuously collected from time T2 to time T5.

[0070] The above example illustrates the use of both current and past still images and audio for evaluation, but it is not limited to this. In addition to current still images and audio, only one of past still images and audio may be used for evaluation. Furthermore, in addition to only current still images and audio, either one or both of past still images and audio may be used for evaluation. In addition to still images from a given time, audio recordings from the current time may also be used for evaluation. When using current and / or past audio for evaluation, speech recognition may be used to generate a string of text representing the audio, which can then be evaluated by a language model. Furthermore, the audio used here does not need to be recorded at the exact same time as the capture time used to generate the still image; it may include all or part of a series of utterances recorded around the time of the capture. Furthermore, evaluation may be performed using captured still images and / or collected audio, along with related data. As related data, for example, temperature and atmospheric pressure for the time period corresponding to the time of capture and / or audio collection, location information of the monitored object, and information from various sensors installed in and around the monitored object can be used. The image analysis device 10 may be configured to evaluate the audio and images together with this related data.

[0071] Thus, according to the modified embodiment 2, in the image analysis device 10, the evaluation acquisition unit 104 sequentially sends prompts to the language model requesting the generating AI to evaluate the scene captured in the target image using the target image, which is a moving image captured at a predetermined analysis interval, and a plurality of past images that are sequentially continuous with the target image. In addition, the image analysis device 10 may have an evaluation acquisition unit 104 that sequentially sends prompts to a language model requesting the generating AI to evaluate the scene captured in the target image, using the target image captured from the moving image at predetermined analysis intervals and the target sound collected in correspondence with the target image. Furthermore, in the image analysis device 10, the evaluation acquisition unit 104 may sequentially send prompts to the language model requesting the generating AI to evaluate the scene captured in the target image, using a set of target image and target sound collected in correspondence with the target image, and a set of multiple past images that are sequentially continuous with the target image and past sound collected in correspondence with each past image. Furthermore, in the image analysis device 10, when the evaluation acquisition unit 104 requests the generating AI to evaluate the scene captured in the target image using the target audio collected in correspondence with the target image captured from the moving image at predetermined analysis intervals, it may also send a prompt to the language model that includes a string of characters representing the target audio as text through speech recognition. As a result, according to the modified embodiment 2, the language model can be made to perform evaluations using not only the target image but also the current audio, past images, past audio, etc., which is expected to improve the accuracy of image analysis.

[0072] (Modification of Embodiment 3) Here, a third modification of the embodiment will be described. This modification differs from the embodiment described above in that the regions R2 and R3 shown in Figure 3 can be displayed using various evaluation axis units and graph types. Figures 9 to 11 are diagrams illustrating the processing performed by the image analysis device according to the third modified embodiment.

[0073] Figure 9 shows an example of a pie chart region R2 displaying evaluation results and a region R3 displaying prompts. The pie chart in this figure shows the evaluation values ​​for each of the multiple evaluation axes as evaluation results for the target image at a given time. By displaying such a pie chart, users can easily understand how the evaluation values ​​for each of the multiple evaluation axes compare to the evaluation values ​​for the other evaluation axes in the target image.

[0074] Figure 10 shows an example of region R2, which displays the evaluation results in a heatmap, and region R3, which displays a prompt. The heatmap shown in this figure shows the evaluation values ​​for each region into which the target image has been divided, as an evaluation result for the target image at a certain point in time. In this case, the evaluation acquisition unit 104 of the image analysis device 10 sends a prompt to the language model, such as "Divide this image into multiple (for example, 9) regions, express the (evaluation index) for each divided region as a number from 0 to 100, and explain the reason," and uses the evaluation results obtained from the language model to create the heatmap shown in this figure and display it on the user terminal 50. By displaying such a heatmap, users can easily understand which areas of the target image have high evaluation values ​​and which areas have low evaluation values.

[0075] Figure 11 shows an example of an image displayed on the user terminal 50 in the modified embodiment 3. In this figure, regions R2-1 and R3-1, and regions R2-2 and R3-2 are displayed on the right side of the image. Region R2-1 displays a time-series line graph as shown in Figure 5. Region R3-1 displays an input field for a prompt corresponding to region R2-1. Region R2-2 displays a pie chart as shown in Figure 9. Region R3-2 displays an input field for a prompt corresponding to region R2-2.

[0076] The evaluation results may be displayed in various ways, not limited to those described above. For example, in a time-series line graph as shown in Figure 5, the resolution (scale) of the vertical axis (evaluation axis) may be changed, a log scale may be used, or only the selected evaluation indicators may be enlarged and displayed.

[0077] Thus, according to the modified embodiment 3, the image analysis device 10, in which the evaluation result generation unit 106 generates a pie chart as an evaluation result showing the evaluation values ​​of each of the multiple evaluation axes for the target image at a certain point in time. Furthermore, the evaluation result generation unit 106 may generate a heat map as an evaluation result, which shows the evaluation value of each of the divided regions obtained by dividing the target image into multiple regions at a given point in time. In the modified image analysis device 10 of the embodiment 3, images showing the evaluation results in various ways can be displayed on the user terminal 50, allowing the user to analyze the evaluation results from various angles.

[0078] In at least one of the embodiments described above, the image analysis device 10 may be configured to include a data storage function. In this case, the image analysis device 10 associates and stores in real time the target image, the target audio collected in accordance with the target image, the prompt, and the evaluation value (the evaluation value evaluated by the language model in response to the prompt). Furthermore, in addition to accumulating data in real time, it may also be possible to perform data storage-related processing on the accumulated videos retrospectively. For example, a user can store surveillance video footage over a certain time period using the data storage function of the image analysis device 10, or using a recording device located outside the image analysis device 10. Later, the user uses the image analysis application to evaluate the stored surveillance video footage. At this time, the user specifies the stored video footage as the analysis target via the image analysis application on the user terminal 50, and inputs evaluation criteria such as the analysis interval and prompts. As a result, the image analysis device 10 performs image analysis of the stored video footage, and as a result of the analysis, evaluation values, which evaluate the video footage at each analysis interval according to the prompts, are notified from the image analysis device 10 to the user terminal 50. The user terminal 50 obtains the evaluation values ​​from the image analysis device 10 and displays a graph showing the time-series changes in the evaluation values. The image analysis application may, when a user selects a location corresponding to a certain time in a graph (a graph showing the time-series change of evaluation values) displayed on the image analysis device 10, display a pop-up image PG or the like, which will play the scene in which the image corresponding to the time selected by the user was captured. This allows the user to easily identify the locations in the stored video where important scenes are captured, and to immediately see what kind of images are captured in those important scenes.

[0079] The image analysis device 10 in the above-described embodiment may be implemented in whole or in part using a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in such cases. The program may be for implementing a part of the functions described above, or it may be a program that can implement the functions described above in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA.

[0080] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of symbols]

[0081] 1…Image analysis system 10…Image analysis device 30…Language model server 40…Message delivery server 101...Image information acquisition unit 102... Evaluation Metric Acquisition Department 103... Prompt acquisition unit 104...Evaluation Acquisition Department 105...Warning judgment section 106...Evaluation result generation unit 107...Output section

Claims

1. An image information acquisition unit that acquires image information of the target image, An evaluation index acquisition unit that acquires evaluation indices to evaluate the scene represented in the image, A prompt acquisition unit acquires a prompt that requests the language model to evaluate the scene represented in the target image based on the evaluation index acquired by the evaluation index acquisition unit, An evaluation acquisition unit that acquires an evaluation value generated by a language model in response to a prompt acquired by the prompt acquisition unit being sent to the language model, An output unit that outputs the evaluation value acquired by the evaluation acquisition unit in association with the target image, An image analysis device equipped with the following features.

2. The aforementioned evaluation indicator acquisition unit acquires multiple evaluation indicators, The evaluation acquisition unit sends a prompt to the language model requesting it to evaluate the scene represented in the target image based on each of the multiple evaluation indices acquired by the evaluation index acquisition unit, and acquires the evaluation values ​​for each of the multiple evaluation indices generated by the language model. The output unit outputs a plurality of evaluation values ​​acquired by the evaluation acquisition unit, associating them with the target image. The image analysis apparatus according to claim 1.

3. The prompt acquisition unit generates a prompt that requests the language model to generate an explanatory text explaining the basis for the evaluation. The evaluation acquisition unit acquires the evaluation value generated by the language model and the explanatory text, The output unit outputs an evaluation value associated with the explanatory text. The image analysis apparatus according to claim 1.

4. The aforementioned image information acquisition unit acquires image information of the moving image, The evaluation acquisition unit sequentially sends prompts to the language model requesting it to evaluate the scenes represented in the target images captured at predetermined analysis intervals, and acquires evaluation values ​​generated by the language model for each of the target images captured at predetermined analysis intervals. The output unit outputs a graph showing the evaluation values ​​acquired by the evaluation acquisition unit in accordance with the time series of the video. The image analysis apparatus according to claim 1.

5. The aforementioned image information acquisition unit acquires image information of the moving image, The prompt acquisition unit generates a prompt that requests the language model to evaluate the scene represented in the target image captured at a predetermined analysis interval and to generate an explanatory text explaining the basis for the evaluation. The output unit outputs a graph showing the evaluation values ​​acquired by the evaluation acquisition unit in a time series of the video, and provides information that associates the target image corresponding to a point in the graph corresponding to a specific time, the evaluation value, and the basis for the evaluation. The image analysis apparatus according to claim 1.

6. The output unit displays the target image and evaluation index acquired by the image information acquisition unit on the same screen of the user terminal, and input fields for inputting these fields. The evaluation index acquisition unit acquires the evaluation index entered in the input field. The image analysis apparatus according to claim 1.

7. The aforementioned image information acquisition unit acquires image information of the moving image, The output unit displays on the same screen of the user terminal an input field for inputting the video and evaluation indicators acquired by the image information acquisition unit. The evaluation index acquisition unit acquires the evaluation index entered in the input field, The evaluation acquisition unit sequentially sends prompts to the language model requesting it to evaluate the scenes represented in the target images captured at predetermined analysis intervals. The output unit displays a graph on the same screen of the user terminal showing the video footage acquired by the image information acquisition unit, the input field, and the evaluation values ​​acquired by the evaluation acquisition unit, in chronological order of the video footage. The image analysis apparatus according to claim 1.

8. An image analysis method performed by a computer-based image analysis device, The image information acquisition unit acquires the image information of the target image, The evaluation index acquisition unit acquires evaluation indexes to evaluate the scene represented in the image, The prompt acquisition unit acquires a prompt that requests the language model to evaluate the scene represented in the target image based on the evaluation index acquired by the evaluation index acquisition unit, The evaluation acquisition unit acquires an evaluation value generated by the language model in response to the prompt acquired by the prompt acquisition unit being sent to the language model. The output unit outputs the evaluation value acquired by the evaluation acquisition unit, associating it with the target image. Image analysis methods.

9. In the computer-based image analysis device, Obtain the image information of the target image, Obtain an evaluation metric to assess the scene represented in the image. The language model receives a prompt requesting it to evaluate the scene represented in the target image based on the acquired evaluation index. The evaluation value generated by the language model in response to sending the acquired prompt is obtained. The acquired evaluation value is output in association with the target image. program.

Images