System

Abstract

To provide a system capable of detecting abnormal behavior and reporting the same immediately.SOLUTION: A system in an embodiment includes a camera unit, an analysis unit, a generation unit, and a notification unit. The camera unit records images. The analysis unit analyzes the images recorded by the camera unit and detects abnormal behavior. The generation unit generates a text based on the abnormal behavior detected by the analysis unit. The notification unit notifies of the text generated by the generation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology had the problem that surveillance cameras could only record footage and could not immediately report abnormal behavior when it occurred.

[0005] The system according to the embodiment aims to detect abnormal behavior and report it immediately. [Means for solving the problem]

[0006] The system according to the embodiment includes a camera unit, an analysis unit, a generation unit, and a reporting unit. The camera unit records video. The analysis unit analyzes the video recorded by the camera unit and detects abnormal behavior. The generation unit generates text based on the abnormal behavior detected by the analysis unit. The reporting unit reports the text generated by the generation unit. Effect of the Invention

[0007] The system according to the embodiment can detect abnormal behavior and report it immediately. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The security system according to the embodiment of the present invention is a system that uses AI to analyze surveillance camera footage and immediately report when an incident occurs. In this security system, surveillance cameras record footage, and AI analyzes the footage to detect abnormal behavior or signs of an incident. For example, when a robbery occurs, AI generates text from the footage such as "A man dressed in black, estimated to be 170 cm tall, entered the robbery." This text is immediately reported to the police and related organizations. This allows victims to quickly call for help even in scenes where they cannot report. For example, a surveillance camera records footage. At this time, the camera records continuously and stores the video data. For example, a surveillance camera in a store records footage 24 hours a day. Next, AI analyzes the recorded footage. AI monitors movements and behavior in the footage in real time and detects abnormal behavior or signs of an incident. For example, AI analyzes people's movements, facial expressions, object movements, etc. to detect abnormal behavior such as robbery or assault. When abnormal behavior is detected, AI generates text based on the footage. For example, when a robbery occurs, AI generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the robbery." This text is a concise summary of the details of the incident. The generated text is immediately reported to the police and relevant agencies. For example, the text generated by AI is automatically sent to the police system to notify them of the occurrence of an incident. This allows the victim to quickly call for help even in scenes where they are unable to report. This mechanism significantly improves the crime prevention function of surveillance cameras. Conventional surveillance cameras only record, but in this invention, AI analyzes the footage and immediately reports when an incident occurs, ensuring the safety of the victim. For example, if a robbery occurs in a store, AI can immediately report it, allowing the police to respond quickly and minimizing damage. This allows the crime prevention system to immediately report when an incident occurs, improving the crime prevention function.

[0029] The crime prevention system according to the embodiment includes a camera unit, an analysis unit, a generation unit, and a reporting unit. The camera unit records video. For example, the camera unit records video 24 hours a day using a surveillance camera in a store. The camera unit records video continuously and saves the video data. The analysis unit analyzes the video recorded by the camera unit to detect abnormal behavior. For example, the analysis unit monitors movements and actions in the video in real time to detect abnormal behavior and signs of an incident. For example, the analysis unit analyzes people's movements, facial expressions, and object movements to detect abnormal behavior such as robbery and assault. The analysis unit can also analyze the speed and direction of movement, and the posture and facial expressions of people. The generation unit generates text based on the abnormal behavior detected by the analysis unit when a robbery occurs. For example, when a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the building as a robber." The generation unit generates text related to abnormal behavior based on a database that has been trained in advance. The generation unit generates text that briefly summarizes the details of the abnormal behavior. The generation unit generates text regarding abnormal behavior using a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that briefly summarizes the details of the abnormal behavior," and the generation AI generates the text. The reporting unit reports the text generated by the generation unit. The reporting unit, for example, automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. This enables the crime prevention system according to the embodiment to immediately report an incident when it occurs, improving the crime prevention function. A part or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI, or may be performed without using AI. For example, the reporting unit can input the generated text to an AI model, determine the urgency of the report, and report it.

[0030] The camera unit records the video. For example, the camera unit is a surveillance camera in a store that records video 24 hours a day. The camera unit records continuously and stores the video data. Specifically, the camera unit uses a high-resolution camera and covers a wide range, allowing it to monitor every corner of the store. The camera is equipped with an infrared function so that it can record clear video even in low-light environments. In addition, the camera unit has a motion detection function, and when it detects abnormal movement, it can automatically zoom in and record detailed video. The recorded video data is uploaded to a cloud server in real time and stored for a long period of time. This makes it possible to quickly search past videos and obtain necessary information. Furthermore, the camera unit can link multiple cameras to achieve wide-area monitoring and minimize blind spots. The camera unit is always operated in an optimal state by performing regular maintenance and software updates. This allows the camera unit to ensure safety in the store and contribute to early detection of abnormal situations.

[0031] The analysis unit analyzes the video recorded by the camera unit and detects abnormal behavior. For example, the analysis unit monitors movements and actions in the video in real time and detects abnormal behavior or signs of an incident. For example, the analysis unit analyzes people's movements, facial expressions, and object movements to detect abnormal behavior such as robbery and assault. The analysis unit can also analyze the speed and direction of movement, and the posture and facial expressions of people. Specifically, the analysis unit uses image recognition technology using AI to automatically identify people and objects in the video. AI can distinguish between normal and abnormal behavior patterns based on a data set that has been learned in advance. For example, AI detects a person's sudden running or throwing of an object as abnormal behavior. In addition, the analysis unit can also detect abnormal sounds such as screams and sounds of destruction by using voice recognition technology to analyze the audio data in the video. The analysis unit comprehensively analyzes this information and detects the occurrence of abnormal behavior with high accuracy. Furthermore, when abnormal behavior is detected, the analysis unit can immediately issue an alert and notify relevant parties. This enables the analysis unit to detect abnormal behavior in real time and respond quickly.

[0032] The generation unit generates text based on the abnormal behavior detected by the analysis unit. For example, when a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the store." The generation unit generates text related to the abnormal behavior based on a database that has been trained in advance. The generation unit generates text that concisely summarizes the details of the abnormal behavior. The generation unit generates text related to the abnormal behavior using a generation AI. The generation AI is a text generation AI (e.g., LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that concisely summarizes the details of the abnormal behavior," and the generation AI generates the text. Specifically, the generation unit inputs a prompt to the generation AI based on the data of the abnormal behavior provided by the analysis unit. The generation AI generates detailed text based on information such as the type of abnormal behavior, the location where it occurred, the time, and the characteristics of the person involved. For example, a specific text such as "At 3:15 p.m., a man dressed in black, estimated to be 170 cm tall, entered the store near the entrance. The man had a knife and threatened the store clerk." is generated. The generator automatically formats the generated text and prepares it for transmission to the reporting unit, thereby enabling the generator to quickly and accurately generate detailed information about abnormal behavior and prepare it for reporting.

[0033] The reporting unit reports the text generated by the generating unit. For example, the reporting unit automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. As a result, the crime prevention system according to the embodiment is capable of immediate reporting when an incident occurs, improving crime prevention functions. A part or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI, or may be performed without using AI. For example, the reporting unit can input the generated text into an AI model, determine the urgency of the report, and report it. Specifically, the reporting unit uses an AI algorithm to analyze the generated text and evaluate the urgency and seriousness of the incident. The AI ​​analyzes keywords and phrases in the text and scores the urgency of the incident. For example, if keywords such as "robbery" or "knife" are included, it is evaluated as a high urgency. The reporting unit makes an appropriate report to the police and related organizations based on this evaluation result. The report is automatically performed via the Internet, and the police system is notified immediately. In addition, the reporting unit records the report content so that it can be checked at a later date. This allows the reporting department to report incidents quickly and accurately, and support the rapid response of related agencies. Furthermore, the reporting department can improve and optimize the system by managing the report history and analyzing the content of past reports.

[0034] The analysis unit can monitor movements or actions in the video in real time and detect abnormal behavior. The analysis unit, for example, monitors movements and actions in the video in real time and detects abnormal behavior or signs of an incident. The analysis unit, for example, analyzes human movements, facial expressions, object movements, etc., and detects abnormal behavior such as robbery or assault. The analysis unit can also analyze the speed and direction of movement, the posture and facial expression of a person, etc. The analysis unit, for example, measures the speed of movement by the moving speed in pixels or the amount of change between frames. The analysis unit measures the direction of movement by the change in angle or the direction of a vector. The analysis unit measures the posture by the angle of the joints or the tilt of the body. The analysis unit measures the facial expression by the change in facial feature points or the classification of emotions. This makes it possible to detect abnormal behavior in real time and respond quickly. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input video data into an AI model and cause the AI ​​to detect abnormal behavior.

[0035] The generation unit can generate text related to abnormal behavior based on a database that has been trained in advance. The generation unit generates text related to abnormal behavior based on a database that has been trained in advance, for example. When a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the house." The generation unit generates text related to abnormal behavior based on a database that has been trained in advance. The generation unit generates text that briefly summarizes the details of the abnormal behavior. The generation unit generates text related to abnormal behavior using a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that briefly summarizes the details of the abnormal behavior," and the generation AI generates text. This enables highly accurate reporting by generating text based on a training database. Some or all of the above-mentioned processing in the generation unit may be performed using an AI, for example, or may be performed without using an AI. For example, the generation unit may input data on abnormal behavior to an AI model and cause the AI ​​to generate text.

[0036] The reporting unit can automatically transmit the generated text to a police system via the Internet. The reporting unit, for example, automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. The reporting unit automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. This enables a quick report by automatic transmission. A part or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI, or may be performed without using AI. For example, the reporting unit can input the generated text into an AI model, determine the urgency of the report, and make the report.

[0037] The analysis unit can analyze the speed or direction of movement, the posture or facial expression of a person, and the like. For example, the analysis unit measures the speed of movement by the moving speed in pixel units or the amount of change between frames. The analysis unit measures the direction of movement by the change in angle or the direction of a vector. The analysis unit measures the posture by the angle of a joint or the tilt of the body. The analysis unit measures the facial expression by the change in facial feature points or the classification of emotions. For example, the analysis unit measures the speed of movement by the moving speed in pixel units or the amount of change between frames. The analysis unit measures the direction of movement by the change in angle or the direction of a vector. The analysis unit measures the posture by the angle of a joint or the tilt of the body. The analysis unit measures the facial expression by the change in facial feature points or the classification of emotions. This improves the accuracy of detecting abnormal behavior through detailed analysis. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input video data into an AI model and cause the AI ​​to detect abnormal behavior.

[0038] The generation unit can generate text that concisely summarizes the details of the abnormal behavior. For example, when a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the house." The generation unit generates text regarding the abnormal behavior based on a database that has been trained in advance. The generation unit generates text that concisely summarizes the details of the abnormal behavior. The generation unit generates text regarding the abnormal behavior using a generation AI. The generation AI is a text generation AI (e.g., LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that concisely summarizes the details of the abnormal behavior," and the generation AI generates the text. This makes the report content clear through the concise text. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the generation unit may input data of the abnormal behavior to an AI model and cause the AI ​​to generate the text.

[0039] When the camera unit detects abnormal behavior, it can zoom in on a specific area and record detailed video. When the camera detects abnormal behavior, for example, the camera unit zooms in on a person's face and records detailed video. When the camera detects abnormal behavior, the camera unit zooms in on the movement of the person's hands and records detailed video. When the camera detects abnormal behavior, the camera unit zooms in on a specific area (for example, around the cash register) and records detailed video. This makes it possible to record detailed video by zooming in. Some or all of the above-mentioned processing in the camera unit may be performed using, for example, AI, or may be performed without using AI. For example, the camera unit can input abnormal behavior detection data to the generation AI and cause the generation AI to control the zoom-in.

[0040] The camera unit can be added with a function of simultaneously recording audio when abnormal behavior is detected. For example, when the camera detects abnormal behavior, the camera unit simultaneously records surrounding audio. When the camera detects abnormal behavior, the camera unit simultaneously records a person's conversation. When the camera detects abnormal behavior, the camera unit simultaneously records environmental sounds (e.g., alarm sounds). This allows more detailed information to be obtained from audio recording. Some or all of the above-mentioned processing in the camera unit may be performed, for example, using AI, or may be performed without using AI. For example, the camera unit can input audio data to a generation AI and have the generation AI perform an analysis of the audio.

[0041] When abnormal behavior is detected, the camera unit can automatically adjust the surrounding lighting to improve the quality of the image. When the camera detects abnormal behavior, the camera unit, for example, brightens the lighting to improve the quality of the image. When the camera detects abnormal behavior, the camera unit, dims the lighting to improve the quality of the image. When the camera detects abnormal behavior, the camera unit, adjusts the color temperature of the lighting to improve the quality of the image. This improves the quality of the image by adjusting the lighting. Some or all of the above-mentioned processing in the camera unit may be performed, for example, using AI, or may be performed without using AI. For example, the camera unit can input lighting data to the generation AI and cause the generation AI to adjust the lighting.

[0042] The camera unit can be added with a function of emitting a warning sound when abnormal behavior is detected. For example, when the camera detects abnormal behavior, the camera unit emits a warning sound to notify the surroundings. When the camera detects abnormal behavior, the camera unit emits a warning sound to intimidate the criminal. When the camera detects abnormal behavior, the camera unit emits a warning sound to notify a security guard. This makes it possible to notify the surroundings of abnormal behavior by the warning sound. Some or all of the above-mentioned processing in the camera unit may be performed using, for example, AI, or may be performed without using AI. For example, the camera unit can input abnormal behavior detection data to a generation AI and cause the generation AI to generate a warning sound.

[0043] When detecting abnormal behavior, the analysis unit can identify a pattern of abnormal behavior by comparing with past data. For example, the analysis unit identifies a pattern of abnormal behavior by comparing with past data. The analysis unit identifies a frequency of abnormal behavior by comparing with past data. The analysis unit identifies a time period of abnormal behavior by comparing with past data. This makes it possible to identify a pattern of abnormal behavior by comparison with past data. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input past data to the generation AI and cause the generation AI to identify a pattern of abnormal behavior.

[0044] When detecting abnormal behavior, the analysis unit can integrate and analyze images from multiple cameras. For example, the analysis unit integrates images from multiple cameras to analyze abnormal behavior. The analysis unit integrates images from multiple cameras to identify the location of abnormal behavior. The analysis unit integrates images from multiple cameras to identify the time of abnormal behavior. In this way, by integrating images from multiple cameras, more accurate analysis is possible. Some or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI or may be performed without using AI. For example, the analysis unit may input video data from multiple cameras to a generation AI and cause the generation AI to perform an integrated analysis of the video.

[0045] When detecting abnormal behavior, the analysis unit can also analyze the voice data to identify the abnormal behavior. The analysis unit, for example, analyzes the voice data to identify the abnormal behavior. The analysis unit analyzes the voice data to identify the location where the abnormal behavior occurred. The analysis unit analyzes the voice data to identify the time when the abnormal behavior occurred. This improves the accuracy of identifying the abnormal behavior by analyzing the voice data. Some or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the voice data to a generation AI and cause the generation AI to perform analysis of the voice.

[0046] When detecting abnormal behavior, the analysis unit can perform analysis based on surrounding environmental data. For example, the analysis unit analyzes abnormal behavior taking into account temperature data. The analysis unit analyzes abnormal behavior taking into account humidity data. The analysis unit identifies the cause of abnormal behavior taking into account environmental data. As a result, the analysis accuracy of abnormal behavior is improved by taking environmental data into account. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input environmental data to the generation AI and cause the generation AI to perform an analysis of abnormal behavior.

[0047] The generation unit can include a detailed timeline of the abnormal behavior when generating the text. The generation unit generates text including, for example, the occurrence time of the abnormal behavior. The generation unit generates text including the duration of the abnormal behavior. The generation unit generates text including the end time of the abnormal behavior. By including a detailed timeline, the report content becomes more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input data of the abnormal behavior to the generation AI and cause the generation AI to generate text including a timeline.

[0048] When generating the text, the generation unit can describe in detail the location where the abnormal behavior occurred. For example, the generation unit generates text that describes in detail the location where the abnormal behavior occurred. The generation unit generates text including the coordinates of the location where the abnormal behavior occurred. The generation unit generates text including a map of the location where the abnormal behavior occurred. As a result, the detailed description of the location makes the report more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input data of the abnormal behavior to the generation AI and cause the generation AI to generate text including the location of the occurrence.

[0049] The generation unit can include background information of the abnormal behavior when generating the text. The generation unit generates text including information on similar incidents in the past, for example. The generation unit generates text including background information of the abnormal behavior. The generation unit generates text including the cause of the abnormal behavior. By including the background information, the report content becomes more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input data on the abnormal behavior to the generation AI and cause the generation AI to generate text including the background information.

[0050] The generation unit can describe the range of influence of the abnormal behavior when generating the text. For example, the generation unit generates text describing the range of influence of the abnormal behavior. The generation unit generates text including a map of the range of influence of the abnormal behavior. The generation unit generates text including details of the range of influence of the abnormal behavior. Thereby, the description of the range of influence makes the report more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input data of the abnormal behavior to the generation AI and cause the generation AI to generate text including the range of influence.

[0051] The reporting unit can set the priority of the destination when transmitting a text. For example, when transmitting a text, the reporting unit sets the police as the top priority destination. When transmitting a text, the reporting unit sets the security company as the next priority. When transmitting a text, the reporting unit sets the administrator as the last priority. This allows for a quick response by setting the priority of the destination. A part or all of the above-mentioned processing in the reporting unit may be performed using, for example, AI, or may be performed without using AI. For example, the reporting unit can input the priority of the destination into an AI model and have the AI ​​determine the optimal destination.

[0052] The reporting unit can save the transmission history when transmitting text so that it can be referred to later. For example, the reporting unit automatically saves the transmission history when transmitting text. The reporting unit saves the transmission history in the cloud when transmitting text. The reporting unit allows the administrator to refer to the transmission history later when transmitting text. In this way, by saving the transmission history, it becomes possible to refer to it later. Some or all of the above-mentioned processing in the reporting unit may be performed using, for example, AI, or may be performed without using AI. For example, the reporting unit can input the transmission history into an AI model and have the AI ​​manage the history.

[0053] The reporting unit can add a function of simultaneously making a voice report when sending a text. The reporting unit, for example, makes a voice report when sending a text. The reporting unit records and sends a voice report when sending a text. The reporting unit makes a voice report in real time when sending a text. This allows for a rapid response by making a voice report at the same time. Some or all of the above-mentioned processing in the reporting unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reporting unit can input voice data to a generation AI and have the generation AI execute a voice report.

[0054] The reporting unit can add a function of automatically updating the contact information of the destination when sending a text. The reporting unit, for example, automatically updates the contact information of the destination when sending a text. The reporting unit periodically updates the contact information of the destination when sending a text. The reporting unit allows an administrator to manually update the contact information of the destination when sending a text. This makes it possible to always report with the latest information by automatically updating the contact information. Some or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI or may be performed without using AI. For example, the reporting unit can input contact information to the generation AI and cause the generation AI to update the information.

[0055] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0056] The security system may further include a voice recognition unit. The voice recognition unit may analyze surrounding sounds in synchronization with the video from the surveillance camera and detect signs of abnormal behavior. For example, the voice recognition unit may detect abnormal sounds such as screams or the sound of glass breaking, and transmit the voice data to the analysis unit. The analysis unit may integrate the voice data and video data to more accurately detect abnormal behavior. Furthermore, the voice recognition unit may detect specific keywords (such as "help") and make an immediate report based on the keyword. In this way, by analyzing both voice and video, the accuracy of detecting abnormal behavior may be improved, enabling a quick response.

[0057] The analysis unit can further include a temperature sensor. The temperature sensor can monitor the temperature in the monitored area in real time and detect abnormal temperature changes. For example, if a fire occurs, the temperature sensor detects a sudden rise in temperature and transmits the data to the analysis unit. The analysis unit can integrate the temperature data and the video data to detect the occurrence of a fire and send an immediate report. Furthermore, the temperature sensor can also emit an alarm when an abnormal temperature change is detected. In this way, by monitoring temperature changes, it becomes possible to quickly detect and respond to abnormal situations such as fires.

[0058] The generation unit can further include a function of recording the frequency of occurrence of abnormal behavior. The generation unit records the date, time, and location of the occurrence of abnormal behavior, and accumulates the data. For example, the generation unit can analyze the frequency of occurrence of abnormal behavior in a specific area, and provide the results to an administrator. Furthermore, the generation unit can adjust the surveillance intensity of a specific area based on the frequency of occurrence of abnormal behavior. This makes it possible to grasp the occurrence pattern of abnormal behavior and take effective crime prevention measures.

[0059] The reporting unit may further include a function for generating report content in multiple languages. The reporting unit may translate the generated text into multiple languages ​​and report in each language. For example, the reporting unit may generate report content in major languages ​​such as English, Japanese, and Chinese, and send the report to relevant agencies. Furthermore, the reporting unit may report in an appropriate language based on the language settings of the reporting destination. This allows for multilingual support, enabling international crime prevention measures and the expectation of rapid response.

[0060] The analysis unit can further include a function for predicting abnormal behavior. The analysis unit can learn patterns of abnormal behavior based on past data and predict future abnormal behavior. For example, the analysis unit can detect patterns in which abnormal behavior is likely to occur during certain times of the day or on certain days of the week, and take preventive measures based on that information. Furthermore, the analysis unit can adjust the placement of surveillance cameras and the intensity of surveillance based on the predicted abnormal behavior. This makes it possible to take measures in advance by predicting abnormal behavior, improving crime prevention effectiveness.

[0061] The process flow of the first embodiment will be briefly described below.

[0062] Step 1: The camera unit records video. For example, a surveillance camera in a store records video 24 hours a day, continuously records video, and stores the video data. Step 2: The analysis unit analyzes the video recorded by the camera unit and detects abnormal behavior. For example, it monitors movements and actions in the video in real time, and analyzes people's movements, facial expressions, and object movements to detect abnormal behavior such as robbery or assault. The analysis unit can also analyze the speed and direction of movement, people's posture, facial expressions, etc. Step 3: The generation unit generates text based on the abnormal behavior detected by the analysis unit. For example, if a robbery occurs, it generates text such as "A man dressed in all black, estimated to be 170 cm tall, entered the building." The generation unit generates text about abnormal behavior based on a pre-trained database, and uses a generation AI to generate text that concisely summarizes the details of the abnormal behavior. Step 4: The reporting unit reports the text generated by the generation unit. For example, the generated text is automatically sent to a police system via the Internet to notify the occurrence of an incident. The reporting unit automatically sends the generated text to the police system, enabling immediate reporting when an incident occurs.

[0063] (Example 2) The security system according to the embodiment of the present invention is a system that uses AI to analyze surveillance camera footage and immediately report when an incident occurs. In this security system, surveillance cameras record footage, and AI analyzes the footage to detect abnormal behavior or signs of an incident. For example, when a robbery occurs, AI generates text from the footage such as "A man dressed in black, estimated to be 170 cm tall, entered the robbery." This text is immediately reported to the police and related organizations. This allows victims to quickly call for help even in scenes where they cannot report. For example, a surveillance camera records footage. At this time, the camera records continuously and stores the video data. For example, a surveillance camera in a store records footage 24 hours a day. Next, AI analyzes the recorded footage. AI monitors movements and behavior in the footage in real time and detects abnormal behavior or signs of an incident. For example, AI analyzes people's movements, facial expressions, object movements, etc. to detect abnormal behavior such as robbery or assault. When abnormal behavior is detected, AI generates text based on the footage. For example, when a robbery occurs, AI generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the robbery." This text is a concise summary of the details of the incident. The generated text is immediately reported to the police and relevant agencies. For example, the text generated by AI is automatically sent to the police system to notify them of the occurrence of an incident. This allows the victim to quickly call for help even in scenes where they are unable to report. This mechanism significantly improves the crime prevention function of surveillance cameras. Conventional surveillance cameras only record, but in this invention, AI analyzes the footage and immediately reports when an incident occurs, ensuring the safety of the victim. For example, if a robbery occurs in a store, AI can immediately report it, allowing the police to respond quickly and minimizing damage. This allows the crime prevention system to immediately report when an incident occurs, improving the crime prevention function.

[0064] The crime prevention system according to the embodiment includes a camera unit, an analysis unit, a generation unit, and a reporting unit. The camera unit records video. For example, the camera unit records video 24 hours a day using a surveillance camera in a store. The camera unit records video continuously and saves the video data. The analysis unit analyzes the video recorded by the camera unit to detect abnormal behavior. For example, the analysis unit monitors movements and actions in the video in real time to detect abnormal behavior and signs of an incident. For example, the analysis unit analyzes people's movements, facial expressions, and object movements to detect abnormal behavior such as robbery and assault. The analysis unit can also analyze the speed and direction of movement, and the posture and facial expressions of people. The generation unit generates text based on the abnormal behavior detected by the analysis unit when a robbery occurs. For example, when a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the building as a robber." The generation unit generates text related to abnormal behavior based on a database that has been trained in advance. The generation unit generates text that briefly summarizes the details of the abnormal behavior. The generation unit generates text regarding abnormal behavior using a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that briefly summarizes the details of the abnormal behavior," and the generation AI generates the text. The reporting unit reports the text generated by the generation unit. The reporting unit, for example, automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. This enables the crime prevention system according to the embodiment to immediately report an incident when it occurs, improving the crime prevention function. A part or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI, or may be performed without using AI. For example, the reporting unit can input the generated text to an AI model, determine the urgency of the report, and report it.

[0065] The camera unit records the video. For example, the camera unit is a surveillance camera in a store that records video 24 hours a day. The camera unit records continuously and stores the video data. Specifically, the camera unit uses a high-resolution camera and covers a wide range, allowing it to monitor every corner of the store. The camera is equipped with an infrared function so that it can record clear video even in low-light environments. In addition, the camera unit has a motion detection function, and when it detects abnormal movement, it can automatically zoom in and record detailed video. The recorded video data is uploaded to a cloud server in real time and stored for a long period of time. This makes it possible to quickly search past videos and obtain necessary information. Furthermore, the camera unit can link multiple cameras to achieve wide-area monitoring and minimize blind spots. The camera unit is always operated in an optimal state by performing regular maintenance and software updates. This allows the camera unit to ensure safety in the store and contribute to early detection of abnormal situations.

[0066] The analysis unit analyzes the video recorded by the camera unit and detects abnormal behavior. For example, the analysis unit monitors movements and actions in the video in real time and detects abnormal behavior or signs of an incident. For example, the analysis unit analyzes people's movements, facial expressions, and object movements to detect abnormal behavior such as robbery and assault. The analysis unit can also analyze the speed and direction of movement, and the posture and facial expressions of people. Specifically, the analysis unit uses image recognition technology using AI to automatically identify people and objects in the video. AI can distinguish between normal and abnormal behavior patterns based on a data set that has been learned in advance. For example, AI detects a person's sudden running or throwing of an object as abnormal behavior. In addition, the analysis unit can also detect abnormal sounds such as screams and sounds of destruction by using voice recognition technology to analyze the audio data in the video. The analysis unit comprehensively analyzes this information and detects the occurrence of abnormal behavior with high accuracy. Furthermore, when abnormal behavior is detected, the analysis unit can immediately issue an alert and notify relevant parties. This enables the analysis unit to detect abnormal behavior in real time and respond quickly.

[0067] The generation unit generates text based on the abnormal behavior detected by the analysis unit. For example, when a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the store." The generation unit generates text related to the abnormal behavior based on a database that has been trained in advance. The generation unit generates text that concisely summarizes the details of the abnormal behavior. The generation unit generates text related to the abnormal behavior using a generation AI. The generation AI is a text generation AI (e.g., LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that concisely summarizes the details of the abnormal behavior," and the generation AI generates the text. Specifically, the generation unit inputs a prompt to the generation AI based on the data of the abnormal behavior provided by the analysis unit. The generation AI generates detailed text based on information such as the type of abnormal behavior, the location where it occurred, the time, and the characteristics of the person involved. For example, a specific text such as "At 3:15 p.m., a man dressed in black, estimated to be 170 cm tall, entered the store near the entrance. The man had a knife and threatened the store clerk." is generated. The generator automatically formats the generated text and prepares it for transmission to the reporting unit, thereby enabling the generator to quickly and accurately generate detailed information about abnormal behavior and prepare it for reporting.

[0068] The reporting unit reports the text generated by the generating unit. For example, the reporting unit automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. As a result, the crime prevention system according to the embodiment is capable of immediate reporting when an incident occurs, improving crime prevention functions. A part or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI, or may be performed without using AI. For example, the reporting unit can input the generated text into an AI model, determine the urgency of the report, and report it. Specifically, the reporting unit uses an AI algorithm to analyze the generated text and evaluate the urgency and seriousness of the incident. The AI ​​analyzes keywords and phrases in the text and scores the urgency of the incident. For example, if keywords such as "robbery" or "knife" are included, it is evaluated as a high urgency. The reporting unit makes an appropriate report to the police and related organizations based on this evaluation result. The report is automatically performed via the Internet, and the police system is notified immediately. In addition, the reporting unit records the report content so that it can be checked at a later date. This allows the reporting department to report incidents quickly and accurately, and support the rapid response of related agencies. Furthermore, the reporting department can improve and optimize the system by managing the report history and analyzing the content of past reports.

[0069] The analysis unit can monitor movements or actions in the video in real time and detect abnormal behavior. The analysis unit, for example, monitors movements and actions in the video in real time and detects abnormal behavior or signs of an incident. The analysis unit, for example, analyzes human movements, facial expressions, object movements, etc., and detects abnormal behavior such as robbery or assault. The analysis unit can also analyze the speed and direction of movement, the posture and facial expression of a person, etc. The analysis unit, for example, measures the speed of movement by the moving speed in pixels or the amount of change between frames. The analysis unit measures the direction of movement by the change in angle or the direction of a vector. The analysis unit measures the posture by the angle of the joints or the tilt of the body. The analysis unit measures the facial expression by the change in facial feature points or the classification of emotions. This makes it possible to detect abnormal behavior in real time and respond quickly. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input video data into an AI model and cause the AI ​​to detect abnormal behavior.

[0070] The generation unit can generate text related to abnormal behavior based on a database that has been trained in advance. The generation unit generates text related to abnormal behavior based on a database that has been trained in advance, for example. When a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the house." The generation unit generates text related to abnormal behavior based on a database that has been trained in advance. The generation unit generates text that briefly summarizes the details of the abnormal behavior. The generation unit generates text related to abnormal behavior using a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that briefly summarizes the details of the abnormal behavior," and the generation AI generates text. This enables highly accurate reporting by generating text based on a training database. Some or all of the above-mentioned processing in the generation unit may be performed using an AI, for example, or may be performed without using an AI. For example, the generation unit may input data on abnormal behavior to an AI model and cause the AI ​​to generate text.

[0071] The reporting unit can automatically transmit the generated text to a police system via the Internet. The reporting unit, for example, automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. The reporting unit automatically transmits the generated text to a police system via the Internet. The reporting unit automatically transmits the generated text to a police system to notify the occurrence of an incident. This enables a quick report by automatic transmission. A part or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI, or may be performed without using AI. For example, the reporting unit can input the generated text into an AI model, determine the urgency of the report, and make the report.

[0072] The analysis unit can analyze the speed or direction of movement, the posture or facial expression of a person, and the like. For example, the analysis unit measures the speed of movement by the moving speed in pixel units or the amount of change between frames. The analysis unit measures the direction of movement by the change in angle or the direction of a vector. The analysis unit measures the posture by the angle of a joint or the tilt of the body. The analysis unit measures the facial expression by the change in facial feature points or the classification of emotions. For example, the analysis unit measures the speed of movement by the moving speed in pixel units or the amount of change between frames. The analysis unit measures the direction of movement by the change in angle or the direction of a vector. The analysis unit measures the posture by the angle of a joint or the tilt of the body. The analysis unit measures the facial expression by the change in facial feature points or the classification of emotions. This improves the accuracy of detecting abnormal behavior through detailed analysis. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input video data into an AI model and cause the AI ​​to detect abnormal behavior.

[0073] The generation unit can generate text that concisely summarizes the details of the abnormal behavior. For example, when a robbery occurs, the generation unit generates text such as "A man dressed in black, estimated to be 170 cm tall, entered the house." The generation unit generates text regarding the abnormal behavior based on a database that has been trained in advance. The generation unit generates text that concisely summarizes the details of the abnormal behavior. The generation unit generates text regarding the abnormal behavior using a generation AI. The generation AI is a text generation AI (e.g., LLM) or a multimodal generation AI. The generation unit inputs a prompt to the generation AI, such as "Please generate text that concisely summarizes the details of the abnormal behavior," and the generation AI generates the text. This makes the report content clear through the concise text. Some or all of the above-mentioned processing in the generation unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the generation unit may input data of the abnormal behavior to an AI model and cause the AI ​​to generate the text.

[0074] The camera unit can estimate the user's emotion and adjust the camera's recording mode based on the estimated user's emotion. For example, when the user is nervous, the camera unit sets the camera's recording mode to high sensitivity and records detailed video. When the user is relaxed, the camera unit sets the camera's recording mode to normal mode and records standard video. When the user is excited, the camera unit sets the camera's recording mode to wide-angle and records a wide range of video. This allows appropriate video recording by adjusting the recording mode according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the camera unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the camera unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0075] When the camera unit detects abnormal behavior, it can zoom in on a specific area and record detailed video. When the camera detects abnormal behavior, for example, the camera unit zooms in on a person's face and records detailed video. When the camera detects abnormal behavior, the camera unit zooms in on the movement of the person's hands and records detailed video. When the camera detects abnormal behavior, the camera unit zooms in on a specific area (for example, around the cash register) and records detailed video. This makes it possible to record detailed video by zooming in. Some or all of the above-mentioned processing in the camera unit may be performed using, for example, AI, or may be performed without using AI. For example, the camera unit can input abnormal behavior detection data to the generation AI and cause the generation AI to control the zoom-in.

[0076] The camera unit can be added with a function of simultaneously recording audio when abnormal behavior is detected. For example, when the camera detects abnormal behavior, the camera unit simultaneously records surrounding audio. When the camera detects abnormal behavior, the camera unit simultaneously records a person's conversation. When the camera detects abnormal behavior, the camera unit simultaneously records environmental sounds (e.g., alarm sounds). This allows more detailed information to be obtained from audio recording. Some or all of the above-mentioned processing in the camera unit may be performed, for example, using AI, or may be performed without using AI. For example, the camera unit can input audio data to a generation AI and have the generation AI perform an analysis of the audio.

[0077] The camera unit can estimate the user's emotion and adjust the timing of starting recording of the camera based on the estimated emotion of the user. For example, when the user is nervous, the camera unit immediately starts recording of the camera. When the user is relaxed, the camera unit starts recording of the camera at a normal timing. When the user is excited, the camera unit starts recording of the camera with a slight delay. This allows appropriate video recording by adjusting the timing of starting recording according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the camera unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the camera unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0078] When abnormal behavior is detected, the camera unit can automatically adjust the surrounding lighting to improve the quality of the image. When the camera detects abnormal behavior, the camera unit, for example, brightens the lighting to improve the quality of the image. When the camera detects abnormal behavior, the camera unit, dims the lighting to improve the quality of the image. When the camera detects abnormal behavior, the camera unit, adjusts the color temperature of the lighting to improve the quality of the image. This improves the quality of the image by adjusting the lighting. Some or all of the above-mentioned processing in the camera unit may be performed, for example, using AI, or may be performed without using AI. For example, the camera unit can input lighting data to the generation AI and cause the generation AI to adjust the lighting.

[0079] The camera unit can be added with a function of emitting a warning sound when abnormal behavior is detected. For example, when the camera detects abnormal behavior, the camera unit emits a warning sound to notify the surroundings. When the camera detects abnormal behavior, the camera unit emits a warning sound to intimidate the criminal. When the camera detects abnormal behavior, the camera unit emits a warning sound to notify a security guard. This makes it possible to notify the surroundings of abnormal behavior by the warning sound. Some or all of the above-mentioned processing in the camera unit may be performed using, for example, AI, or may be performed without using AI. For example, the camera unit can input abnormal behavior detection data to a generation AI and cause the generation AI to generate a warning sound.

[0080] The analysis unit can estimate the user's emotion and adjust the analysis algorithm based on the estimated user's emotion. For example, when the user is nervous, the analysis unit sets the analysis algorithm to high sensitivity and quickly detects abnormal behavior. When the user is relaxed, the analysis unit sets the analysis algorithm to normal mode and performs standard analysis. When the user is excited, the analysis unit sets the analysis algorithm to a wide range and detects abnormal behavior over a wide range. This enables appropriate analysis by adjusting the analysis algorithm according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0081] When detecting abnormal behavior, the analysis unit can identify a pattern of abnormal behavior by comparing with past data. For example, the analysis unit identifies a pattern of abnormal behavior by comparing with past data. The analysis unit identifies a frequency of abnormal behavior by comparing with past data. The analysis unit identifies a time period of abnormal behavior by comparing with past data. This makes it possible to identify a pattern of abnormal behavior by comparison with past data. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input past data to the generation AI and cause the generation AI to identify a pattern of abnormal behavior.

[0082] When detecting abnormal behavior, the analysis unit can integrate and analyze images from multiple cameras. For example, the analysis unit integrates images from multiple cameras to analyze abnormal behavior. The analysis unit integrates images from multiple cameras to identify the location of abnormal behavior. The analysis unit integrates images from multiple cameras to identify the time of abnormal behavior. In this way, by integrating images from multiple cameras, more accurate analysis is possible. Some or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI or may be performed without using AI. For example, the analysis unit may input video data from multiple cameras to a generation AI and cause the generation AI to perform an integrated analysis of the video.

[0083] The analysis unit can estimate the user's emotion and adjust the display method of the analysis result based on the estimated user's emotion. For example, when the user is nervous, the analysis unit provides a simple and highly visible display method. When the user is relaxed, the analysis unit provides a display method including detailed information. When the user is in a hurry, the analysis unit provides a display method that focuses on the main points. This makes it possible to display an appropriate analysis result by adjusting the display method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0084] When detecting abnormal behavior, the analysis unit can also analyze the voice data to identify the abnormal behavior. The analysis unit, for example, analyzes the voice data to identify the abnormal behavior. The analysis unit analyzes the voice data to identify the location where the abnormal behavior occurred. The analysis unit analyzes the voice data to identify the time when the abnormal behavior occurred. This improves the accuracy of identifying the abnormal behavior by analyzing the voice data. Some or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the voice data to a generation AI and cause the generation AI to perform analysis of the voice.

[0085] When detecting abnormal behavior, the analysis unit can perform analysis based on surrounding environmental data. For example, the analysis unit analyzes abnormal behavior taking into account temperature data. The analysis unit analyzes abnormal behavior taking into account humidity data. The analysis unit identifies the cause of abnormal behavior taking into account environmental data. As a result, the analysis accuracy of abnormal behavior is improved by taking environmental data into account. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input environmental data to the generation AI and cause the generation AI to perform an analysis of abnormal behavior.

[0086] The generation unit can estimate the user's emotion and adjust the expression method of the generated text based on the estimated user's emotion. For example, when the user is nervous, the generation unit uses a concise and clear expression method. When the user is relaxed, the generation unit uses a detailed and polite expression method. When the user is excited, the generation unit uses an emphasized expression method. This enables appropriate text generation by adjusting the expression method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0087] The generation unit can include a detailed timeline of the abnormal behavior when generating the text. The generation unit generates text including, for example, the occurrence time of the abnormal behavior. The generation unit generates text including the duration of the abnormal behavior. The generation unit generates text including the end time of the abnormal behavior. By including a detailed timeline, the report content becomes more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input data of the abnormal behavior to the generation AI and cause the generation AI to generate text including a timeline.

[0088] When generating the text, the generation unit can describe in detail the location where the abnormal behavior occurred. For example, the generation unit generates text that describes in detail the location where the abnormal behavior occurred. The generation unit generates text including the coordinates of the location where the abnormal behavior occurred. The generation unit generates text including a map of the location where the abnormal behavior occurred. As a result, the detailed description of the location makes the report more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, AI, or may be performed without using AI. For example, the generation unit can input data of the abnormal behavior to the generation AI and cause the generation AI to generate text including the location of the occurrence.

[0089] The generation unit can estimate the user's emotion and adjust the length of the generated text based on the estimated user's emotion. For example, when the user is in a hurry, the generation unit generates a short text that is to the point. When the user is relaxed, the generation unit generates a longer text that includes a detailed explanation. When the user is excited, the generation unit generates a text that adds a visually stimulating effect. This makes it possible to provide appropriate information by adjusting the length of the text according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0090] The generation unit can include background information of the abnormal behavior when generating the text. The generation unit generates text including information on similar incidents in the past, for example. The generation unit generates text including background information of the abnormal behavior. The generation unit generates text including the cause of the abnormal behavior. By including the background information, the report content becomes more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input data on the abnormal behavior to the generation AI and cause the generation AI to generate text including the background information.

[0091] The generation unit can describe the range of influence of the abnormal behavior when generating the text. For example, the generation unit generates text describing the range of influence of the abnormal behavior. The generation unit generates text including a map of the range of influence of the abnormal behavior. The generation unit generates text including details of the range of influence of the abnormal behavior. Thereby, the description of the range of influence makes the report more specific. Some or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input data of the abnormal behavior to the generation AI and cause the generation AI to generate text including the range of influence.

[0092] The reporting unit can estimate the user's emotion and adjust the urgency of the notification based on the estimated user's emotion. For example, when the user is nervous, the reporting unit sets the urgency of the notification to high. When the user is relaxed, the reporting unit sets the urgency of the notification to normal. When the user is excited, the reporting unit sets the urgency of the notification to the highest. This allows an appropriate response by adjusting the urgency of the notification according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reporting unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reporting unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0093] The reporting unit can set the priority of the destination when transmitting a text. For example, when transmitting a text, the reporting unit sets the police as the top priority destination. When transmitting a text, the reporting unit sets the security company as the next priority. When transmitting a text, the reporting unit sets the administrator as the last priority. This allows for a quick response by setting the priority of the destination. A part or all of the above-mentioned processing in the reporting unit may be performed using, for example, AI, or may be performed without using AI. For example, the reporting unit can input the priority of the destination into an AI model and have the AI ​​determine the optimal destination.

[0094] The reporting unit can save the transmission history when transmitting text so that it can be referred to later. For example, the reporting unit automatically saves the transmission history when transmitting text. The reporting unit saves the transmission history in the cloud when transmitting text. The reporting unit allows the administrator to refer to the transmission history later when transmitting text. In this way, by saving the transmission history, it becomes possible to refer to it later. Some or all of the above-mentioned processing in the reporting unit may be performed using, for example, AI, or may be performed without using AI. For example, the reporting unit can input the transmission history into an AI model and have the AI ​​manage the history.

[0095] The reporting unit can estimate the user's emotion and adjust the content of the report based on the estimated user's emotion. For example, when the user is nervous, the reporting unit provides concise and clear report content. When the user is relaxed, the reporting unit provides detailed report content. When the user is excited, the reporting unit provides emphasized report content. This enables appropriate information provision by adjusting the report content according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reporting unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reporting unit can input the user's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0096] The reporting unit can add a function of simultaneously making a voice report when sending a text. The reporting unit, for example, makes a voice report when sending a text. The reporting unit records and sends a voice report when sending a text. The reporting unit makes a voice report in real time when sending a text. This allows for a rapid response by making a voice report at the same time. Some or all of the above-mentioned processing in the reporting unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reporting unit can input voice data to a generation AI and have the generation AI execute a voice report.

[0097] The reporting unit can add a function of automatically updating the contact information of the destination when sending a text. The reporting unit, for example, automatically updates the contact information of the destination when sending a text. The reporting unit periodically updates the contact information of the destination when sending a text. The reporting unit allows an administrator to manually update the contact information of the destination when sending a text. This makes it possible to always report with the latest information by automatically updating the contact information. Some or all of the above-mentioned processing in the reporting unit may be performed, for example, using AI or may be performed without using AI. For example, the reporting unit can input contact information to the generation AI and cause the generation AI to update the information.

[0098] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0099] The security system may further include a voice recognition unit. The voice recognition unit may analyze surrounding sounds in synchronization with the video from the surveillance camera and detect signs of abnormal behavior. For example, the voice recognition unit may detect abnormal sounds such as screams or the sound of glass breaking, and transmit the voice data to the analysis unit. The analysis unit may integrate the voice data and video data to more accurately detect abnormal behavior. Furthermore, the voice recognition unit may detect specific keywords (such as "help") and make an immediate report based on the keyword. In this way, by analyzing both voice and video, the accuracy of detecting abnormal behavior may be improved, enabling a quick response.

[0100] The analysis unit can further include a temperature sensor. The temperature sensor can monitor the temperature in the monitored area in real time and detect abnormal temperature changes. For example, if a fire occurs, the temperature sensor detects a sudden rise in temperature and transmits the data to the analysis unit. The analysis unit can integrate the temperature data and the video data to detect the occurrence of a fire and send an immediate report. Furthermore, the temperature sensor can also emit an alarm when an abnormal temperature change is detected. In this way, by monitoring temperature changes, it becomes possible to quickly detect and respond to abnormal situations such as fires.

[0101] The generation unit can further include a function of recording the frequency of occurrence of abnormal behavior. The generation unit records the date, time, and location of the occurrence of abnormal behavior, and accumulates the data. For example, the generation unit can analyze the frequency of occurrence of abnormal behavior in a specific area, and provide the results to an administrator. Furthermore, the generation unit can adjust the surveillance intensity of a specific area based on the frequency of occurrence of abnormal behavior. This makes it possible to grasp the occurrence pattern of abnormal behavior and take effective crime prevention measures.

[0102] The reporting unit may further include a function for generating report content in multiple languages. The reporting unit may translate the generated text into multiple languages ​​and report in each language. For example, the reporting unit may generate report content in major languages ​​such as English, Japanese, and Chinese, and send the report to relevant agencies. Furthermore, the reporting unit may report in an appropriate language based on the language settings of the reporting destination. This allows for multilingual support, enabling international crime prevention measures and the expectation of rapid response.

[0103] The analysis unit can further include a function for predicting abnormal behavior. The analysis unit can learn patterns of abnormal behavior based on past data and predict future abnormal behavior. For example, the analysis unit can detect patterns in which abnormal behavior is likely to occur during certain times of the day or on certain days of the week, and take preventive measures based on that information. Furthermore, the analysis unit can adjust the placement of surveillance cameras and the intensity of surveillance based on the predicted abnormal behavior. This makes it possible to take measures in advance by predicting abnormal behavior, improving crime prevention effectiveness.

[0104] The camera unit can estimate the user's emotions and adjust the viewing angle of the camera based on the estimated user's emotions. For example, if the user is nervous, the viewing angle of the camera is narrowed to record detailed images. If the user is relaxed, the viewing angle of the camera is widened to record a wide range of images. If the user is excited, the viewing angle of the camera can be dynamically adjusted to follow the user's movements. This allows appropriate video recording by adjusting the viewing angle according to the user's emotions.

[0105] The analysis unit can estimate the user's emotions and adjust the method of notifying the analysis results based on the estimated user's emotions. For example, if the user is nervous, the analysis results are notified briefly. If the user is relaxed, the analysis results are notified in detail. If the user is excited, the analysis results are notified with visual emphasis. This makes it possible to provide appropriate information by adjusting the notification method according to the user's emotions.

[0106] The generation unit can estimate the user's emotion and adjust the format of the generated text based on the estimated user's emotion. For example, if the user is nervous, a concise, bullet-point format is used. If the user is relaxed, a paragraph-style format including detailed descriptions is used. If the user is excited, a visually noticeable format using emphasized fonts and colors is used. This makes it possible to provide appropriate information by adjusting the text format according to the user's emotion.

[0107] The reporting unit can estimate the user's emotions and adjust the timing of reporting based on the estimated user's emotions. For example, if the user is nervous, the reporting is made immediately. If the user is relaxed, the reporting is made at the normal timing. If the user is excited, the reporting is made with a slight delay. In this way, the timing of reporting can be adjusted according to the user's emotions, making it possible to respond appropriately.

[0108] The analysis unit can estimate the user's emotions and adjust the parameters of the analysis algorithm based on the estimated user's emotions. For example, if the user is nervous, the sensitivity of the analysis algorithm is set high. If the user is relaxed, the sensitivity of the analysis algorithm is set to normal. If the user is excited, the sensitivity of the analysis algorithm is set to low. This allows for appropriate analysis by adjusting the parameters of the analysis algorithm according to the user's emotions.

[0109] The process flow of the second embodiment will be briefly described below.

[0110] Step 1: The camera unit records video. For example, a surveillance camera in a store records video 24 hours a day, continuously records video, and stores the video data. Step 2: The analysis unit analyzes the video recorded by the camera unit and detects abnormal behavior. For example, it monitors movements and actions in the video in real time, and analyzes people's movements, facial expressions, and object movements to detect abnormal behavior such as robbery or assault. The analysis unit can also analyze the speed and direction of movement, people's posture, facial expressions, etc. Step 3: The generation unit generates text based on the abnormal behavior detected by the analysis unit. For example, if a robbery occurs, it generates text such as "A man dressed in all black, estimated to be 170 cm tall, entered the building." The generation unit generates text about abnormal behavior based on a pre-trained database, and uses a generation AI to generate text that concisely summarizes the details of the abnormal behavior. Step 4: The reporting unit reports the text generated by the generation unit. For example, the generated text is automatically sent to a police system via the Internet to notify the occurrence of an incident. The reporting unit automatically sends the generated text to the police system, enabling immediate reporting when an incident occurs.

[0111] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0112] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0113] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0114] Each of the multiple elements including the camera unit, the analysis unit, the generation unit, and the reporting unit described above is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the camera unit is realized by the camera 42 of the smart device 14, and records video on a 24-hour basis. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and monitors movements and actions in the video in real time to detect abnormal actions and signs of incidents. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates text related to abnormal actions. The reporting unit is realized, for example, by the control unit 46A of the smart device 14, and automatically transmits the generated text to the police system. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0115] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0116] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0117] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0118] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0119] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0120] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0121] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0122] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0123] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0124] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0125] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0126] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0127] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0128] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0129] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0130] Each of the multiple elements including the camera unit, the analysis unit, the generation unit, and the notification unit described above is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the camera unit is realized by the camera 42 of the smart glasses 214, and records video on a 24-hour basis. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and monitors movements and actions in the video in real time to detect abnormal actions and signs of incidents. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates text related to abnormal actions. The notification unit is realized, for example, by the control unit 46A of the smart glasses 214, and automatically transmits the generated text to the police system. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0131] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0132] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0133] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0134] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0135] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0136] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0137] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0138] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0139] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0140] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0141] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0142] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0143] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0144] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0145] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0146] Each of the multiple elements including the camera unit, the analysis unit, the generation unit, and the reporting unit described above is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the camera unit is realized by the camera 42 of the headset type terminal 314, and records video on a 24-hour basis. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and monitors movements and actions in the video in real time to detect abnormal actions and signs of incidents. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates text related to abnormal actions. The reporting unit is realized, for example, by the control unit 46A of the headset type terminal 314, and automatically transmits the generated text to the police system. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0147] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0148] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0149] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0150] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0151] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0152] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0153] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0154] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0155] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0156] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0157] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0158] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0159] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0160] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0161] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0162] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0163] Each of the multiple elements including the camera unit, the analysis unit, the generation unit, and the reporting unit described above is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the camera unit is realized by the camera 42 of the robot 414, and records video on a 24-hour basis. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and monitors movements and actions in the video in real time, and detects abnormal actions and signs of incidents. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates text related to abnormal actions. The reporting unit is realized, for example, by the control unit 46A of the robot 414, and automatically transmits the generated text to the police system. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0164] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0165] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0166] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0167] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0168] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0169] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0170] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0171] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0172] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0173] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0174] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0175] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0176] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0177] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0178] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0179] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0180] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0181] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0182] (Appendix 1) A camera unit for recording video; an analysis unit that analyzes the video recorded by the camera unit and detects abnormal behavior; a generation unit that generates text based on the abnormal behavior detected by the analysis unit; A notification unit that notifies the text generated by the generation unit. A system characterized by: (Appendix 2) The analysis unit is Monitor movements or actions in the video in real time and detect abnormal behavior 2. The system of claim 1, (Appendix 3) The generation unit is Generate text about abnormal behavior based on a pre-trained database 2. The system of claim 1, (Appendix 4) The reporting unit includes: The generated text is automatically sent to the police system via the Internet. 2. The system of claim 1, (Appendix 5) The analysis unit is Analyze the speed or direction of movement, a person's posture or facial expression, etc. 2. The system of claim 1, (Appendix 6) The generation unit is Generate a concise text summary detailing the abnormal behavior 2. The system of claim 1, (Appendix 7) The camera unit includes: To estimate a user's emotion and adjust a camera's recording mode based on the estimated user's emotion. 2. The system of claim 1, (Appendix 8) The camera unit includes: When abnormal behavior is detected, it zooms in on a specific area and records detailed footage. 2. The system of claim 1, (Appendix 9) The camera unit includes: Add a function to simultaneously record audio when abnormal behavior is detected 2. The system of claim 1, (Appendix 10) The camera unit includes: The system estimates the user's emotions and adjusts the camera's recording start timing based on the estimated user's emotions. 2. The system of claim 1, (Appendix 11) The camera unit includes: Automatically adjusts ambient lighting to improve video quality when abnormal behavior is detected 2. The system of claim 1, (Appendix 12) The camera unit includes: Add a function to emit an alarm when abnormal behavior is detected 2. The system of claim 1, (Appendix 13) The analysis unit is Inferring user emotions and adjusting analysis algorithms based on the inferred user emotions 2. The system of claim 1, (Appendix 14) The analysis unit is When detecting abnormal behavior, compare with past data to identify patterns of abnormal behavior 2. The system of claim 1, (Appendix 15) The analysis unit is When detecting abnormal behavior, video footage from multiple cameras is integrated and analyzed. 2. The system of claim 1, (Appendix 16) The analysis unit is Infer user emotions and adjust the way analysis results are displayed based on the inferred user emotions 2. The system of claim 1, (Appendix 17) The analysis unit is When detecting abnormal behavior, audio data is also analyzed to identify abnormal behavior. 2. The system of claim 1, (Appendix 18) The analysis unit is When detecting abnormal behavior, analysis is based on surrounding environmental data 2. The system of claim 1, (Appendix 19) The generation unit is Estimate the user's emotions and adjust the way the generated text is expressed based on the estimated user emotions. 2. The system of claim 1, (Appendix 20) The generation unit is Include a detailed timeline of anomalous behavior when generating text 2. The system of claim 1, (Appendix 21) The generation unit is When generating text, provide detailed descriptions of where abnormal behavior occurs 2. The system of claim 1, (Appendix 22) The generation unit is Estimate the user's emotions and adjust the length of the generated text based on the estimated user emotions. 2. The system of claim 1, (Appendix 23) The generation unit is Include contextual information about anomalous behavior when generating text 2. The system of claim 1, (Appendix 24) The generation unit is Describe the scope of impact of abnormal behavior when generating text 2. The system of claim 1, (Appendix 25) The reporting unit includes: Estimating a user's emotions and adjusting the urgency of a report based on the estimated user's emotions 2. The system of claim 1, (Appendix 26) The reporting unit includes: Set the priority of recipients when sending texts 2. The system of claim 1, (Appendix 27) The reporting unit includes: When sending a text, save your sending history for future reference 2. The system of claim 1, (Appendix 28) The reporting unit includes: Estimate the user's emotions and adjust the content of the report based on the estimated user emotions 2. The system of claim 1, (Appendix 29) The reporting unit includes: Add a feature to send voice messages simultaneously when sending text messages 2. The system of claim 1, (Appendix 30) The reporting unit includes: Add the ability to automatically update contact information when sending a text 2. The system of claim 1, [Explanation of symbols]

[0183] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A camera unit for recording video; an analysis unit that analyzes the video recorded by the camera unit and detects abnormal behavior; a generation unit that generates text based on the abnormal behavior detected by the analysis unit; A notification unit that notifies the text generated by the generation unit. A system characterized by:

2. The analysis unit is Monitor movements or actions in the video in real time and detect abnormal behavior 2. The system of claim 1 .

3. The generation unit is Generate text about abnormal behavior based on a pre-trained database 2. The system of claim 1 .

4. The reporting unit includes: The generated text is automatically sent to the police system via the Internet.

2. The system of claim 1 .

5. The analysis unit is Analyze the speed or direction of movement, a person's posture or facial expression, etc.

2. The system of claim 1 .

6. The generation unit is Generate a concise text summary detailing the abnormal behavior 2. The system of claim 1 .

7. The camera unit includes: To estimate a user's emotion and adjust a camera's recording mode based on the estimated user's emotion.

2. The system of claim 1 .

8. The camera unit includes: When abnormal behavior is detected, it zooms in on a specific area and records detailed footage.

2. The system of claim 1 .

Images