Information analysis system, information analysis method, and program

The information analysis system addresses the limitations of conventional behavioral recognition models by using a behavior recognition model and a language model to analyze complex behaviors in moving image data, enabling flexible and detailed analysis of video content.

WO2024231989A1PCT designated stage expired Publication Date: 2024-11-14KONICA MINOLTA INC

Patent Information

Application Number
PCT/JP2023/017332
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-08
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Conventional behavioral recognition models are limited in analyzing long-term video data and can only recognize specific tasks trained during model training, making it difficult to perform flexible behavioral analysis.

Method used

An information analysis system comprising an acquisition unit for non-verbal time-series data, a recognition unit using a behavior recognition model to generate log information, and a language model to process user-defined analysis content, allowing for the recognition of complex behaviors and flexible analysis of moving image data.

Benefits of technology

Enables the analysis of complex behaviors and flexible specification of analysis content for moving image data, overcoming the limitations of conventional models by dividing video data into segments and using a combination of behavior recognition and language models to generate detailed analysis results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023017332_14112024_PF_FP_ABST
    Figure JP2023017332_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed is a technique for realizing information analysis based on analysis contents flexibly designated for moving image data. One aspect of the present disclosure relates to an information analysis system comprising: an acquisition unit that acquires non-language time-series data; a recognition unit that recognizes a target situation on the basis of the non-language time-series data; a storage unit that stores the target situation as language data log information; a reception unit that receives an analysis request; an analysis unit that executes analysis on the basis of the log information and the analysis request; and an output unit that outputs the results of the analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Information analysis system, information analysis method and program

[0001] The present disclosure relates to an information analysis system, an information analysis method, and a program.

[0002] Recent advances in deep learning technology have expanded the opportunities for using machine learning models. For example, action recognition models trained to recognize human actions from videos containing people, objects, and other objects are now being used, making it possible to recognize what people are doing in videos captured by video cameras.

[0003] “Language Models are Few-Shot Learners”, arXiv:2005.14165v4, July 22, 2020. “Towards General Purpose Vision Systems: An End-to-End Task-Agnostic Vision-Language Architecture”, Proceedings - 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022.

[0004] However, it is difficult to perform behavioral analysis by inputting long video data into a machine learning model, and conventional behavioral recognition models can only analyze behavior for specific tasks specified during model training.

[0005] In view of the above problems, one object of the present disclosure is to provide a technique for realizing information analysis based on analysis content that is flexibly specified for video image data.

[0006] One aspect of the present disclosure relates to an information analysis system having an acquisition unit that acquires non-verbal time series data, a recognition unit that recognizes a target situation based on the non-verbal time series data, a memory unit that stores the target situation as log information of language data, a reception unit that receives an analysis request, an analysis unit that performs analysis based on the log information and the analysis request, and an output unit that outputs a result of the analysis.

[0007] According to the present disclosure, it is possible to provide a technique for realizing information analysis based on analysis content flexibly specified for video image data.

[0008] FIG. 1 is a schematic diagram illustrating information analysis processing according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram illustrating an information analysis device according to an embodiment of the present disclosure. FIG. 3 is a block diagram illustrating a hardware configuration of an information analysis device according to an embodiment of the present disclosure. FIG. 4 is a block diagram illustrating a functional configuration of an information analysis device according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating behavior recognition processing according to an embodiment of the present disclosure. FIG. 6 is a diagram illustrating training processing of a behavior recognition model according to an embodiment of the present disclosure. FIG. 7 is a diagram illustrating inference processing of a behavior recognition model according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating an example screen for specifying analysis content according to an embodiment of the present disclosure. FIG. 9 is a flowchart illustrating information analysis processing according to an embodiment of the present disclosure.

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0010] In the following embodiment, an information analysis system is disclosed that generates analysis results relating to a target in non-linguistic time-series data such as video data.

[0011] [Outline of the Present Disclosure] As shown in FIG. 1 , in an information analysis process according to an embodiment of the present disclosure, a behavior recognition model 40 is used to acquire log information indicating recognition results of the captured behavior of a subject and / or the movement of an object from video data captured by an imaging device 20 such as a camera. For example, the log information may be a time-series recognition result of the subject's behavior and / or the object's movement. Analysis content and the log information for the video data provided from a user terminal 30 are then input into a language model 50 such as ChatGPT, and the results of an analysis performed on the log information in accordance with the analysis content are returned to the user terminal 30. For example, the analysis content may be, "What is the total time for the subject to go from walking to jogging on bb / cc / aa?" In this case, the language model 50 may add up the time determined by the behavior recognition model 40 to change from "walking" to "jogging" in the log information for bb / cc / aa, and return an analysis result such as, "The total time for going from walking to jogging is xx minutes." to the user terminal 30.

[0012] A typical behavior recognition model can recognize individual actions such as "walking" and "jogging," but it is difficult to set a complex action such as "starting from walking and then jogging" as the analysis content. The information analysis process according to the illustrated embodiment makes it possible to set such complex actions as the analysis content.

[0013] 2 is a schematic diagram illustrating an information analysis system according to an embodiment of the present disclosure. As shown in FIG. 2, the information analysis system 10 includes an imaging device 20, a user terminal 30, an information analysis device 100, and a server device 200.

[0014] The image capturing device 20 is, for example, a video camera, and captures an image of a target area to generate moving image data. The moving image data is transmitted to the information analyzing device 100.

[0015] The user terminal 30 may be realized by any information processing device, such as a personal computer (PC), tablet, or smartphone, and sends an analysis request including analysis content for video data to the information analysis device 100 and obtains analysis results for the analysis content from the information analysis device 100.

[0016] The information analyzing device 100 performs an analysis on the video data received from the imaging device 20 based on the analysis content received from the user terminal 30, and returns the analysis result to the user terminal 30. The information analyzing device 100 according to the following embodiment performs the analysis using a behavior recognition model 40 and a language model 50, as will be described in detail below.

[0017] When the server device 200 acquires the log information and the analysis content from the information analyzing device 100 , the server device 200 executes the language model 50 and returns the analysis result based on the analysis content for the log information to the information analyzing device 100 .

[0018] In the illustrated embodiment, the behavior recognition model 40 is stored in the information analysis device 100, and the language model 50 is stored in the server device 200, but the behavior recognition model 40 and the language model 50 according to the present disclosure are not limited to this, and other configurations may be applied.

[0019] The information analyzing device 100 may be realized by a computing device such as a server, a personal computer (PC), a smartphone, or a tablet, and may have, for example, the hardware configuration shown in Fig. 3. That is, the information analyzing device 100 has a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106, which are interconnected via a bus B.

[0020] The programs or instructions that realize the various functions and processes in the information analyzing device 100 may be stored in a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 101, the programs or instructions are installed from the storage medium to the storage device 102 or the memory device 103 via the drive device 101. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0021] The storage device 102 is realized by a hard disk drive or the like, and stores installed programs or instructions as well as files, data, etc. used to execute the programs or instructions.

[0022] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or instruction is activated, it reads and stores the program, instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory storage medium.

[0023] The processor 104 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and executes various functions and processes of the information analysis device 100 in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc., stored in the memory device 103.

[0024] The user interface (UI) device 105 may be composed of input devices such as a keyboard, mouse, camera, and microphone, output devices such as a display, speaker, headset, and printer, and input / output devices such as a touch panel, and realizes an interface between a user and the information analyzing device 100. For example, a user operates a GUI (Graphical User Interface) displayed on a display or touch panel using a keyboard, mouse, and the like to operate the information analyzing device 100.

[0025] The communication device 106 is realized by various communication circuits that execute wired and / or wireless communication processing with communication networks such as external devices, the Internet, a LAN (Local Area Network), and a cellular network.

[0026] However, the above-described hardware configuration is merely an example, and the information analyzing device 100 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0027] [Information Analysis Apparatus] Next, a description will be given of the information analysis apparatus 100 according to an embodiment of the present disclosure. Fig. 4 is a block diagram showing the functional configuration of the information analysis apparatus 100 according to an embodiment of the present disclosure.

[0028] 4 , the information analyzing device 100 includes an acquisition unit 110, a recognition unit 120, a storage unit 130, a reception unit 140, an analysis unit 150, and an output unit 160. The information analyzing device 100 may be realized by a program that causes one or more processors 104 to operate as the acquisition unit 110, the recognition unit 120, the storage unit 130, the reception unit 140, the analysis unit 150, and / or the output unit 160. Furthermore, it is not necessary for all of the acquisition unit 110, the recognition unit 120, the storage unit 130, the reception unit 140, the analysis unit 150, and the output unit 160 to be included in the information analyzing device 100. One or more of the acquisition unit 110, the recognition unit 120, the storage unit 130, the reception unit 140, the analysis unit 150, and / or the output unit 160 may be included in the information analyzing system 10. For example, the acquisition unit 110 and the recognition unit 120 may be included in the imaging device 20 , and / or the storage unit 130 may not be included in the information analyzing device 100 .

[0029] The acquisition unit 110 acquires non-verbal time-series data. For example, the non-verbal time-series data may be time-series data such as video, images, and audio acquired at multiple points in time or within a certain period of time. As shown in Figures 1 and 2, in this embodiment, the acquisition unit 110 acquires video image data from the imaging device 20.

[0030] The recognition unit 120 recognizes the situation of the subject based on non-verbal time-series data. As shown in FIGS. 1 and 2 , in this embodiment, the recognition unit 120 acquires log information from video data using the behavior recognition model 40. The log information may, for example, indicate the recognition results of the subject's behavior recognized by the behavior recognition model 40 in the video data in association with time. For example, as shown in FIG. 1 , the log information may indicate each behavior of the subject and / or each movement of the subject in chronological order, such as "dd: ee to ff: walking" or "gg: hh to ii: jogging" on "aa year, bb month, cc day."

[0031] In one embodiment, the recognition unit 120 may divide the video data into multiple video data segments. For example, if the behavior recognition model 40 is implemented as a machine learning model for video analysis, long video data may impose a large processing load on the behavior recognition model 40. For this reason, the recognition unit 120 may divide the video data into video data segments at a predetermined time interval appropriate for the computational load of the behavior recognition model 40. For example, as shown in FIG. 5 , if the video data VD is divided into video data segments VD_1, VD_2, ..., VD_N, the recognition unit 120 may input each video data segment VD_i to the behavior recognition model 40 and acquire video information VI_i for each video data segment VD_i from the behavior recognition model 40. In this case, the video information VI_i may indicate the situation of a target in the video data segment VD_i (e.g., the target's behavior, the movement of the target, etc.) in association with time information.

[0032] Alternatively, in another example, the recognition unit 120 may input VD_1 to VD_i collectively into the behavior recognition model 40 and acquire video information VI_1 to VI_i corresponding to VD_1 to VD_i from the behavior recognition model 40. That is, the recognition unit 120 may treat multiple video data segments as a single video data segment and acquire video information corresponding to the multiple video data segments. In this case, the video information VI_1 to VI_i may indicate the situation of the target in the video data segments VD_1 to VD_i (for example, the behavior of the target person, the movement of the target object, etc.) in association with time information.

[0033] In addition, each video information VI_i may indicate the attributes of the person in the video data segment VD_i (e.g., male / female, adult / minor, age group, clothing, etc.), the time, the type and / or duration of the detected behavior, etc.

[0034] The behavior recognition model 40 used by the recognition unit 120 can be generated, for example, by a training process such as that shown in FIG. 6 (for further details, see, for example, “Zero-Shot Abnormal Behavior Recognition Using Pre-Trained DNN” (25th Image Recognition and Understanding Symposium)). Specifically, a video feature extractor and a language feature extractor are prepared, training video data is input to the video feature extractor, and behavior labels (e.g., walking, swimming, etc.) corresponding to the video data input to the video feature extractor are input to the language feature extractor. Then, parameters of the video feature extractor are adjusted so that the feature vectors output from the video feature extractor and the feature vectors output from the language feature extractor are similar. When a predetermined termination condition is met, such as when all training data has been processed, the training process is terminated, and the video feature extractor with updated parameters is made available to the information analysis device 100 as the behavior recognition model 40.

[0035] In the inference process, as shown in Fig. 7, the behavior recognition model 40 can recognize any behavior based on the similarity between a video feature vector obtained from video data and a language feature vector obtained from any behavior label (e.g., walking, swimming, etc.). The recognition unit 120 can recognize the target situation from video data using the behavior recognition model 40 trained in this way. In this way, the behavior recognition model 40 can be a machine learning model trained to recognize any behavior from non-language time-series data using a video feature model and a language feature model.

[0036] However, the behavior recognition model 40 according to the present disclosure is not necessarily limited to this and may be another type of machine learning model. For example, the behavior recognition model 40 may be any 2D model that aggregates and recognizes features for each image using a 2D CNN, a Transformer, or the like. Furthermore, the behavior recognition model 40 may be any 3D model that acquires and recognizes features of video including time-series data using a 3D CNN, a Cube-based Transformer, or the like. Furthermore, the behavior recognition model 40 may be a two-stream model that aggregates and recognizes features of RGB images and optical flow images (including motion information). Furthermore, the behavior recognition model 40 may be a skeleton-based model that estimates key points for each image (such as human joint points and object endpoints) and aggregates and recognizes them. Alternatively, the behavior recognition model 40 may be a combination of any of the above-mentioned models.

[0037] The storage unit 130 stores the target situation as log information of language data. For example, the storage unit 130 may store video information VI_1 to VI_N acquired from the behavior recognition model 40 as log information. For example, the log information may be a chronological list of video information VD_1 to VD_N, that is, data in any format, such as a specific file format (e.g., a json file), a table format, or a sentence format, that chronologically lists the attributes of people captured in the video data VD_1 to VD_N (e.g., male / female, adult / minor, age group, clothing, etc.), the time, the type and / or duration of detected behavior, etc.

[0038] The receiving unit 140 receives an analysis request. Specifically, the receiving unit 140 may receive an analysis request including analysis content for video data from the user terminal 30. For example, if the user is able to set the target person's age group, the target person's jacket color, the target behavior, and the content as the analysis content, the receiving unit 140 may transmit a setting screen 60 as shown in FIG. 8 to the user terminal 30. The user may select input items for the target person's age group 61, the target person's jacket color 62, the target behavior 63, and the content 64 displayed using pull-down menus on the setting screen 60. For example, in the illustrated example, "20 to 29 years old" is selected as the target person's age group 61, "red" as the target person's jacket color 62, "violence" as the target behavior 63, and "time period when activity is most frequent" as the content 64. Note that the analysis content is not limited to the above and may be any other content. Furthermore, it is not limited to the pull-down menu format and may be text input by the user. For example, as described above, the user may input the analysis content as text data into the user terminal 30, such as "What is the total time for which the subject started walking and then jogged on bb / cc / aa year?"

[0039] The analysis unit 150 performs analysis based on the log information and the analysis request. Specifically, the analysis unit 150 inputs the log information and the analysis content included in the analysis request into the language model 50 and obtains the analysis result from the language model 50. Here, the language model 50 may be, for example, a general-purpose question-and-answer type natural language analysis model such as ChatGPT, or may be any type of natural language analysis model trained using training data.

[0040] For example, suppose that the imaging device 20 captures an image of a street in a busy shopping district, and the behavior recognition model 40 recognizes the behavior of passersby from the video image data and generates log information. For example, the log information indicates attributes such as the age group and clothing of passersby, and indicates the behavior of passersby such as "walking," "stopping," "joggering," "conversing," and "violence" in association with time. Furthermore, if the reception unit 140 receives analysis content indicating the age group 61 "20 to 29 years old," the jacket color 62 "red," the behavior 63 "violence," and the content 64 "time period during which the behavior is most prevalent," as shown in FIG. 8 , the analysis unit 150 inputs this log information and analysis content into the language model 50 and obtains an analysis result for the log information from the language model 50. For example, the analysis unit 150 may obtain from the language model 50 a response such as, "The time period during which people aged 20 to 29, wearing red jackets, and committing violent acts are most prevalent is between 11:00 PM and 2:00 AM."

[0041] Alternatively, if the user inputs an analysis content such as "What is the total time that the subject started walking and then jogged on bb / cc / aa year?" into the user terminal 30, the analysis unit 150 inputs the analysis content and log information into the language model 50 and may obtain an analysis result such as "The total time that the subject started walking and then jogged is xx minutes" from the language model 50. In this way, by using the language model 50, it becomes possible to detect, from the log information, a composite action such as "starting from walking to jogging" that combines actions such as "walking" and "jogging" that can be recognized by the behavior recognition model 40. Here, the "composite action" may represent a combination of multiple actions that can be recognized by the behavior recognition model 40.

[0042] The output unit 160 outputs the analysis result. Specifically, the output unit 160 transmits the answer obtained from the language model 50 by the analysis unit 150 to the user terminal 30. This allows the user terminal 30 to obtain the analysis result for the requested analysis content.

[0043] [Information Analysis Processing] Next, an information analysis processing according to an embodiment of the present disclosure will be described. The information analysis processing is executed by the information analysis device 100, and more specifically, may be realized by one or more processors 104 of the information analysis device 100 executing one or more programs or instructions stored in one or more memory devices 103. Fig. 9 is a flowchart showing the information analysis processing according to an embodiment of the present disclosure.

[0044] In step S101, the information analyzing device 100 acquires non-verbal time-series data. Here, the non-verbal time-series data may be audio-video data such as moving images (video), images, and sounds. For example, the information analyzing device 100 may acquire video data captured by a video camera in real time, or may acquire video data files captured over a certain period of time from a video database or the like.

[0045] In step S102, the information analyzing device 100 recognizes the situation of the subject. Specifically, the information analyzing device 100 recognizes the behavior of the subject and / or the movement of the subject from non-verbal time-series data using the behavior recognition model 40 or the like. For example, the behavior recognition model 40 may generate video information that indicates the attributes, behavior, etc. of the subject captured in the video data in association with time information. Furthermore, the information analyzing device 100 may segment the video data into video data segments of an appropriate size and perform behavior recognition processing on each video data segment using the behavior recognition model. This makes it possible to process long video data using a machine learning model.

[0046] In step S103, the information analyzing device 100 stores log information. The log information may be created based on, for example, video information. That is, attributes, behavior, and the like of a subject shown in the video information may be stored as log information in association with time information.

[0047] In step S104, the information analyzing device 100 receives an analysis request. Specifically, the information analyzing device 100 receives an analysis request including analysis content for the log information from the user terminal 30.

[0048] In step S105, the information analyzing device 100 executes analysis in accordance with the log information and the analysis content. Specifically, the information analyzing device 100 inputs the log information and the analysis content into the language model 50 and obtains an analysis result based on the analysis content for the log information from the language model 50.

[0049] In step S106, the information analyzing device 100 outputs the analysis result. Specifically, the information analyzing device 100 transmits the analysis result acquired from the language model 50 to the user terminal 30. The analysis result provided to the user terminal 30 may be the one acquired from the language model 50, or may be the one edited to improve visibility.

[0050] According to the above-described embodiments, it is possible to acquire, as log information, the recognition results of the subject's behavior and / or the object's motion acquired from the behavior recognition model, set complex actions that are generally difficult to recognize using the behavior recognition model as analysis content, and use a language model to acquire analysis results for the acquired log information and the set analysis content. Furthermore, by dividing video data into multiple video data segments and inputting them into the behavior recognition model, it is possible to realize information analysis based on flexibly specified analysis content for long video data. Furthermore, in the embodiments of the present disclosure, the behavior recognition model and / or language model can utilize existing models, allowing the analysis target to be expanded without developing a new model.

[0051] The following supplementary notes are provided in addition to the above description. (Supplementary Note 1) An information analysis system comprising: an acquisition unit that acquires non-linguistic time-series data; a recognition unit that recognizes a situation of a target based on the non-linguistic time-series data; a storage unit that stores the situation of the target as log information of linguistic data; a reception unit that receives an analysis request; an analysis unit that performs analysis based on the log information and the analysis request; and an output unit that outputs a result of the analysis. (Supplementary Note 2) The information analysis system according to Supplementary Note 1, wherein the time-series data is video data; the situation of the target is behavior of the target; the analysis unit utilizes a natural language analysis model that performs analysis based on the log information and the analysis request. (Supplementary Note 3) The information analysis system according to Supplementary Note 2, wherein the recognition unit divides the video data into a plurality of video data segments. (Supplementary Note 4) The information analysis system according to any one of Supplements 1 to 3, wherein the recognition unit utilizes a behavior recognition model that recognizes one or both of an attribute and a behavior of the target from the non-linguistic time-series data. (Supplementary Note 5) The information analysis system according to any one of Supplements 1 to 4, wherein the behavior recognition model is a machine learning model trained to recognize any behavior from the non-language time-series data using a video feature model and a language feature model. (Supplementary Note 6) The information analysis system according to any one of Supplements 1 to 5, wherein the log information indicates attributes and behavior of the target in association with time. (Supplementary Note 7) The information analysis system according to any one of Supplements 1 to 6, wherein the analysis request includes analysis content indicating one or more of attributes, behavior, and content of the target. (Supplementary Note 8) An information analysis method, wherein a computer executes the following steps: acquiring non-language time-series data; recognizing a situation of the target based on the non-language time-series data; storing the situation of the target as log information of language data; accepting an analysis request; performing an analysis based on the log information and the analysis request; and outputting a result of the analysis.(Supplementary Note 9) A program that causes a computer to perform the following operations: acquiring non-verbal time series data; recognizing a target situation based on the non-verbal time series data; storing the target situation as log information of language data; accepting analysis content; performing analysis based on the log information and the analysis request; and outputting results of the analysis.

[0052] Although examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure as set forth in the claims.

[0053] REFERENCE SIGNS LIST 10 Information analysis system 20 Imaging device 30 User terminal 40 Behavior recognition model 50 Language model 100 Information analysis device 110 Acquisition unit 120 Recognition unit 130 Storage unit 140 Reception unit 150 Analysis unit 160 Output unit

Claims

1. An information analysis system having an acquisition unit that acquires non-verbal time series data, a recognition unit that recognizes a target situation based on the non-verbal time series data, a memory unit that stores the target situation as log information of language data, a reception unit that receives an analysis request, an analysis unit that performs analysis based on the log information and the analysis request, and an output unit that outputs results of the analysis.

2. The information analysis system of claim 1, wherein the time series data is video data, the subject's situation is the subject's behavior, the analysis unit utilizes a natural language analysis model, and the natural language analysis model performs analysis based on the log information and the analysis content of the analysis request.

3. The information analysis system according to claim 2, wherein the recognition unit divides the video data into a plurality of video data segments.

4. An information analysis system as described in claim 1, wherein the recognition unit utilizes a behavior recognition model that recognizes one or both of the attributes and behavior of the target from the non-verbal time-series data.

5. The information analysis system of claim 1, wherein the behavior recognition model is a machine learning model trained to recognize any behavior from the non-verbal time-series data using a video feature model and a language feature model.

6. The information analysis system according to claim 1, wherein the log information indicates attributes and actions of the subject in association with time.

7. The information analysis system according to claim 1, wherein the analysis request includes analysis content indicating one or more of attributes, behavior, and content of the target.

8. An information analysis method executed by a computer, comprising: acquiring non-verbal time series data; recognizing a situation of a target based on the non-verbal time series data; storing the situation of the target as log information of language data; accepting an analysis request; performing an analysis based on the log information and the analysis request; and outputting a result of the analysis.

9. A program that causes a computer to execute the following operations: acquiring non-verbal time series data; recognizing a target situation based on the non-verbal time series data; storing the target situation as log information of language data; accepting analysis contents; performing analysis based on the log information and the analysis request; and outputting results of the analysis.

Citation Information

Patent Citations

  • Processing program, processing method and processor

    JP2016218692A

  • Learning model generation device, learning model, behavior recognition device, learning data generation device, and learning data generation method

    JP2022064460A

  • Tree structure estimation device, parameter learning device, method for estimating tree structure, method for learning parameter, and program

    JP2022135518A

  • Data conversion device, data conversion method, and data conversion program

    WO2022244106A1

Cited By

  • Interactive time series analysis system and method of the same

    JP2025162971A

  • Interactive time series analysis system and method

    JP7844695B2

  • Abnormal behavior detection system

    JP7895584B1

  • Moving image generation device

    WO2026167865A1

  • Information processing device and information processing method

    WO2026176656A1